With so many weights, it practically encodes a massive Internet text database.
However, if the output needs to be curated and edited by humans, the scale and automation is gone - we just get a different manual process, with a modest improvement to speed at cost of some decline in quality, and that's not very impactful.
Google at this point favours long form content for many search intents. Being able to generate thousands of these pages in one-click is a real problem. Not just because of popular topics e.g. "covid-19 symptoms" but more so for the long tail e.g. "should I drink coffee to cure covid-19".
It may be that Google's algorithms don't care at all how human-like the text is, or that their own recognition algorithm/NN (whatever they use) isn't fooled. Even if it is affected, Google has the money and corpus to build its own competing NN to recognize GPT-3 text.
That said, there might be a different threat to Google. GPT-3 seems really useful as a search engine of sorts (with the first answer implementing the 'I'm Feeling Lucky' button). Tune it for a query syntax, and for getting the 'top X' results somehow, then we just need the web corpus and a basic filter over the results. We could have a very interesting Google competitor.
More than cherry-picking, there's the Eliza Effect - it's pretty easy to make people think generated text is intelligent. That text can seem intelligent for a while isn't necessarily impressive at all.
Makes me worry about my own reading comprehension, but I think what happened was that since it was posted on HN and got upvoted a lot, I simply assumed that anything that I didn't understand was not the writer's fault, but mine.
For instance, it was unclear from the post what the bitcoinforum experiment was about, but I just dismissed it as me not being attentive enough while reading.
At one point GPT-3 writes: "The forum also has many people I don’t like. I expect them to be disproportionately excited by the possibility of having a new poster that appears to be intelligent and relevant." Why would people he doesn't like be paricularly excited about a new intelligent poster? Again I just assumed that I missed the author's point, not that it was nonsensical.
Twice it refers to tables or screenshots that are not included, but it seemed like an innocent mistake. "When I post to the forum as myself, people frequently mention that they think I must be a bot to be able to post so quickly" seemed like another simple mistake, meaning to say that when he posted as GPT-3, people thought he was being too quick.
This is like a written Rorschach test, when I'm convinced that what I'm reading must make sense, then I'll guess at the author's intent and force it to make sense, forgiving a lot of mistakes or inconsistencies.
Bots offering idiocy and idiocy generally has done lots of damage. But by idiocy here I would quite carefully calculated cleverly polarized positions and I don't think just bot-rot would be enough (to maybe coin a phrase).
It’s cool, but it looked like very basic stuff - the type of UI that is very easy to create in a few minutes. (And really with what was setup behind the scenes - maybe just as fast to just write the code.)
The hard part about software development is not those bits which are common, but the parts that are unique to our specific solution.
Search terms tweaked for your unique interests, and not a commercial entity's, for example.
Is reddit gold really that valuable?
> famous
Surely there are easier ways.
> really useful
We already have enough 2020 reddit commenters regurgitating 2010 hn threads regurgitating 2000 slashdot threads, thanks.
This will accelerate development. Is the current version there? Probably not. But GPT-4 might, and would then accelerate the development of future versions.
Even though this is not "magic", it sounds like it will turn into a practically usable and extremely valuable tool soon.
However I like spirit of optimism and first looks at encouraging and very promising results.
Exciting times!
Honestly not that impressive since you can get comparable results with a series of regex rules given that there are limited ways to describe your intent e.g. "create a button of colour <colour> at the <location of button>"
I believe the hype is that people think they can replace the designer by "just telling the computer" what they want. I don't believe that will work, as they already have trouble telling a human what they want, and a computer won't really know what to do with "I want it to kind of feel like it's from that movie with the blue people that Cameron did, you know?"
In my experience, people have a hard time writing their ideas about designs & features down, because they don't know what they want. They want to talk about it abstractly with somebody who has a better understanding of the field so that person can help them develop the idea. I don't think ML will cover that part any time soon.
From an academic standpoint, writing is part of the thinking process. If you haven't written it down, you haven't fully thought it through. If it feels difficult, that's probably because your understanding isn't as complete as you thought it was.
From a software development standpoint, implementing something is part of the thinking process. Ever notice how the requirements have a tendency to break as soon as you actually try to implement them? If a spot seems difficult it just means you hadn't really figured it out yet.
I 100% agree. I noticed a giant shift in tasks when I made one client write tickets instead of making phone calls. Writing it down forces you to think it through.
And I agree about software development as well, yes. Though I think it's even rare to have somebody describe all the features they want unless it's an experienced software developer who basically writes a textual representation of the application.
But for most PMs (that I've worked with at least), they have vague ideas about what they want, and bringing them into focus is a back and forth with developers and designers. I don't see them getting anywhere with an NLP automaton, but maybe with an Eliza-style system: "Give me a big yellow button saying 'Sign up'" - "Why do you want a big yellow button saying 'Sign up'?" - "You're right, that's too on the nose... give me a link saying 'Sign up'"...