How robot writers could change the internet

How robot writers could change the internet


A mildly fun thing to do when you’re bored
is start the beginning of a text message, and then use only the suggested words to finish
it. “In five years I will see you in the morning
and then you can get it.” The technology behind these text predictions
is called a “language model”: a computer program that uses statistics to guess the
next word in a sentence. And in the past year, other, newer language
models have gotten really, weirdly good at generating text that mimics human writing.
“In five years, I will never return to this place. He felt his eye sting and his throat
tighten.” The program completely made this up. It’s
not taken from anywhere else and it’s not using a template made by humans.
For the first time in history, computers can write stories. The only problem is that it’s
easier for machines to write fiction than to write facts. Language models are useful for a lot of reasons. They help “recognize speech” properly
when sounds are ambiguous in speech-to-text applications.
And they can make translations more fluent when a word in one language maps to multiple
words in another. But if you asked language models to simply
generate passages of text, the results never made much sense.
SHANE: And so the kinds of things that made sense to do were like generating single words
or very short phrases. For years, Janelle Shane has been experimenting
with language generation for her blog AI Weirdness. Her algorithms have generated paint colors,
“Bull Cream” Halloween costumes,
“Sexy Michael Cera” And pick-up lines.
“You look like a thing and I love you.” But this is what she got in 2017 when she
asked for longer passages, like the first lines of a novel:
SHANE:The year of the island is discovered the Missouri of the galaxy like a teenage
lying and always discovered the year of her own class-writing bed …It makes no sense.
Compare that to this opening line from a newer language model called GPT-2.
SHANE: It was a rainy, drizzling day in the summer of 1869. And the people of New York,
who had become accustomed to the warm, kissable air of the city, were having another bad one.
JOSS: It’s like it’s getting better at bullsh*tting us.
SHANE: Yes, yes, it is very good at generating scannable, readable bullsh*t.
Going from word salad to pretty passable prose took a new approach in the field of natural
language processing. Typically, language tasks have required carefully
structured data. You need thousands of correct examples to train the program.
For translation you need a bunch of samples of the same document in multiple languages.
For spam filters, you need emails that humans have labeled as spam.
For summarization, you need full documents plus their human-written summaries. Those
data sources are limited and can take a lot of work to collect.
But if the task is to simply guess the next word in a sentence, the problem comes with
its own solution. So the training data can be any human-written
text, no labeling required. This is called “self-supervised learning.” That’s what
makes it easy and inexpensive to gather data, which means you can use a LOT of it.
Like all of Wikipedia, or 11,000 books, or 8 million web sites.
With that amount of data, plus serious computing resources, and a few tweaks to the
architecture and size of the algorithms, these new language models build vast mathematical
maps of how every word correlates with every other word, all without being explicitly told
any of the rules of grammar or syntax. That gives them fluency with whatever language
they’re trained on, but it doesn’t mean they know what’s true or false.
To get language models to generate true stories, like summarizing documents or answering questions
accurately, it takes extra training. The simplest thing to do without much more
work is just generate passages of text, which are both superficially coherent and also false.
GEITGEY: So give me any headline that you want a fake news story for.
JOSS: Scientists discover Flying Horse. Adam Geitgey is a software developer who created
a fake news website populated entirely with generated text.
He used a language model called Grover, which was trained on news articles from 5,000 publications.
“More than 1,000 years ago, archaeologists unearthed a mysterious flying animal in France
and hailed it the ‘Winged Horse of Afzel’ or ‘Horse of Wisdom’”
GEITGEY: This is amazing, right? Like this is crazy.
JOSS: So crazy. GEITGEY: “The animal, which is the size of
a horse, was not easy.” If we just Google that. Like there’s nothing.
JOSS::It doesn’t exist anywhere. GEITGEY: And I don’t want to say this is perfect.
But just from a longer term point of view of what people were really excited about three
years ago versus what people can do now, like this is just like a huge, huge leap.
If you read closely, you can see that the model is describing a creature that is somehow
both “mouse-like” and “the size of a horse.”
That’s because it doesn’t actually know what it’s talking about. It’s simply mimicking
the writing style of a news reporter. These models can be trained to write in the
voice of any source, like a twitter feed, “I’d like to be very clear about one thing.
shrek is not based on any actual biblical characters. not even close.”
Or whole subreddits. “I found a potato on my floor.”
“A lot of people use the word ‘potato’ as an insult to imply they are not really
a potato, they just ‘looked like’ one.” “I don’t mean insult, I mean as in as in
the definition of the word potato.” “Fair enough. The potato has been used in
various ways for a long time.” But we may be entering a time when AI-generated
text isn’t so funny anymore. “Islam has taken the place of Communism
as the chief enemy of the West.” Researchers have shown that these models can
be used to flood government websites with fake public comments about policy proposals,
post tons of fake business reviews, argue with people online, and generate extremist
and racist posts that can make fringe opinions seem more popular than they really are.
GEITGEY: It’s all about like taking something you could do and then just increasing the
scale of it, making it more scalable and cheaper. The good news is that some of the developers
who built these language models also built ways to detect much of the text generated
through their models. But it’s not clear who has the responsibility to fake-check the
internet. And as bots become even better mimics – with
faces like ours, voices like ours, and now our language, those of us made of flesh and blood may find ourselves
increasingly burdened with not only detecting what’s fake, but also proving that we’re
real.

100 thoughts on “How robot writers could change the internet

  • To learn more about this technology, and try it out yourself, check out the links in the description box. Thanks for watching! -joss

  • "Your scientists were so preoccupied with whether or not they could, they didn't stop to think if they should"

  • A lot of people are not so good at all and I don’t know how much they are going outside and they want to be a better man than I do ete is the way you are going to be happy and I don’t know what to do with you and you don’t know what to do with you and you don’t know what to do with you and you don’t know how much I wanna you want me to do it

  • So I am not sure if I will be able to make it to the meeting on Friday or Friday night and I will be out of town for a few days and then I will be unable to attend the meeting on the 27th and I will be unable to attend the meeting on the 27th and I will be unable to attend the meeting on the 27th

  • I have a lot of money to do that though I’m gonna I was gonna I wanna was a good day to you and you guys too and we all have a lot to make for a great day I wanna I love my family t I have been to this.

  • Autocomplete on twitter

    Well in my humble opinion, of course without intending to offend anyone who thinks differently from my point of view, but also by looking at this matter in a different way, and to make it clear, with the utmost respect, and by considering each and every one's opinion, I honestly believe that I completely forgot what I was going to type…
    – Autocomplete

  • But aren't language models trained off real text, so really the computers aren't writing text themselves? Oh no I'm watching a vox video. Sheesh I'm bored at work

  • Hey as a music producer this could save me 50% of my songwriting publishing there is a songwriter app here somewhere hmmm

  • Really they help with talk to text? Cuz mine always finds the most ridiculous option it can find in the English language.

  • Ohhh I was trying to read what criminals taught black kids to do but it cut off the screen before it showed the rest of the sentence.

  • And made 5k a gamepass of my buildings and I was in a house in a place that I will never forget about it and I don't want it was it to be the budget for the budget that I had in the end.

  • Writing the letter and I want one page technically you have a great week and a little late but I was wondering if there are things you have done for us because we weren't born in front of the house air Force Base today and tomorrow but then again properly so she looked at me and I was wondering ( repeat wall)

  • So you are telling me these AI in the future can look at my brain and generate the dankest of memes that I can tolerate in a limitless supply. We have peaked

  • Hey I’m gonna was the night I wanna was a good day I wanna see y’all soon love y’all be cool I love y’all and I’m so happy I got this time and I’m gonna I wanna see ya tomorrow and I love you Justin is your day I wanna is your birthday. Just thought I would give it a try

  • There are you still have the same to you too have a great day parade on the other hand the wildcats their work is also a good time to explore new ways of learning and smart ways of learning and smart ways of learning and smart ways of learning and smart ways of learning and smart ways of learning and smart ways of learning and smart ways of learning

  • Idk if it’s just me but vox kinda changed changed their video style since Joss and the entire crew of Glad You Asked started showing up more in the videos, it’s so much better produced now and the videos tend to take a philosophical turn by the end now. 100% love it!!! Y’all rock @Vox

  • Even chat bots are becoming much more cunning and a lot harder to detect. They can still be tricked by asking it a similar question but in a different way. Great vid thx 🙂

  • This comment is generated by language model and also the same thing as a single word or an excerpt from the competitio post and the answer is that the word was a little more personal than the word itself but it was not great at first but I thought that it would like to see the truth and also a great story about our lives in this world of mine in this country where we have no reason for it goes away from our consum in this way we have the best of all

  • I think it's the best way to get a good deal on a whole new level of work and a good relationship with your partner

  • The best thing is that you can’t remember what other people do and what you wanna hear about it.
    Autocomplete author

  • Some People will actually believe this 4:30 part, because some just believe everything and anything they see online.

  • I am not sure if you are aware of this but I am interested in the job you have posted on Craigslist and would like to know more about the position you have available for me to take a look at your resume and I look forward to hearing from you soon

  • Are you a good driver and a good guy to work with in the mood for love to hear about the most efficient methods of learning Japanese

  • I like touching kids and new one so we can actually add people who crew of a dickgead does that alot and new one so we can actually add people who crew of a dickgead does that alot and new one 6dollars is a few of the textbooks in off format if u do the whole of

  • "Myself and my wife are looking for a place to stay for a few days and I have to go to the store and get some rest and feel better soon and that is why I am asking for a friend to talk to you" auto correct

  • "All I get from my wife is that she is not going anywhere else but she doesn't obey me and I'm so easily offended by the idea that I can't do anything about it"
    -Samsung auto fill

  • “It was a rainy, drizzling day in the summer of 1869. And the people of New York who had become a custom to the warm, kissable air of the city, were having another bad one. The suffocating spray of rain and strong winds coming down from the great bridge over the Hudson River was running at such a speed that it was raining outside and people ran into the streets in the rain. But worse was to come. A heavy storm was brewing. In less than an hour the great river was about to break the banks of the heavily armed, rowdy crowd, and soon the sidewalks were being split in half and people were thrown across the streets. The storm passed, but the people continued to flock into the city in a swarm and”

  • Are you not going to be able to attend the event on Sunday and I will be there on Friday night for the isolation of pure aerobic and anaerobic.

  • Hey I am interested in your job posting on Craigslist and I was wondering if you could send me a copy of your fingers crossed for the new password and username is that it is a startup company.

  • To be honest, the only thing scary is the misuse. Technology of NLP is like still behind, way behind than this video leads you to believe. But, people still like killing each other with unfinished swords or hammers too.

  • I don't know if you have any questions or concerns please visit the plug-in settings to determine how attachments are you still have the same to you and your family and friends is not available. This started right then took a turn 😂

  • I need this website to write my CV for me. Let's say the key words are 'medical professional', 'surgeon', 'graduated from Hopkins School of Medicine'. Let's see how far I can get before they catch me haha

  • Me in India: Let's go abroad and study Creative Writing in a top college.

    Vox: Haha. You forgot us.

  • Scientific advances are a helpful tool… Sometimes technology kills its creator though! It's happened historically speaking…

    @scoldsore On Twitter
    Current Location : Miami, FL – Earth

  • My predictions are really boring 😔 :
    I am most likely going to be able to make it to the writing class this week and I will be in the office tomorrow and will be in the office tomorrow to take a look at the ppt now and let you know if I can get a hold of you to buy so next time for the sake of convenience and let me know if you have any questions or concerns about the time I am in the office or at home or on the phone or on the phone or at the same time as I am in the office and I will be there on the 15th and I will be there on the 15th and I will be there on the 15th and I will be there on the

  • Generated text: flying horse
    Me: haha funny
    Generated text: * is racist *
    Me: no stop go back to flying horse again

Leave a Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

Copyright © 2019 Geted Tabs Online. All rights reserved.