I’m not sure I’m looking forward to the politics that would come out of 10-20% of the previously middle class becoming instantly redundant and out of (middle-salary) work. That’s the fast path to fascism, unless we’re able to quickly implement UBI and other major societal overhauls.

Yeah I share these concerns as well (https://news.ycombinator.com/item?id=35305791).

My hope is that some countries will see this as an opportunity to expand their safety nets and reduce the work burden on their citizens, which might convince citizens of countries that don't to demand similar policies.

> unless we’re able to quickly implement UBI and other major societal overhauls.

Which are not going to happen.

Something more approachable would be dropping payroll taxes to zero, or even making them negative for some positions, and significantly increasing corporate and capital gains.

The problem isn't the specific policy, the problem is that right now the people who will be empowered and enriched the most by any theoretical "good at stuff" AI are the same people who already spend mountains of cash and effort stopping those things.

How will a functional AI model do anything other than make them better at getting the outcomes they want? CEOs and the megarich have never had any problems watching people burn for their bank account.

This is isn't speculating, this is truthing.

Truthing as in theory quakery, not backed by evidence?

How would we keep it as low as 10-20%? 60% of employees in the US are knowledge workers. That number is similar for most wealthy nations.

I wonder how it will be able to do that for the tech that will be current in 10 years, if mostly everyone will be using AI by then instead of asking on Stack Overflow.

Stack Overflow is headed for oblivion? What's the downside?

Lack of training data for the AI. Stack Overflow is just an e exemplary stand-in, you can replace it with any other resources you like.

you missed the snark there. I hate SO.

I think if history has bearing on things I don't see the cost of accounting, legal or copywriting ever approaching 0. If anything you will see those paywalled behind a company who will extract that from you.

It's wishful thinking that somehow that goes to 0.

ChatGPT is already better at copywriting than 90% of startup founders and marketing people at big cos. You'll soon be able to let it generate 1000s of different versions of marketing material to A/B test or personalize based on user info.

Soon you'll have multi modal transformers from dozens of companies and open source projects that will be able to parse and categorize all of your financial data and they'll have all of the incentives in the world to get it down to the cost of a quickbooks subscription.

> Cost of shipping products will also go down 10-20x.

How can a large language model achieve that?

Ask chatgpt to implement some of the things you worked on the last few months. I was very skeptical too until I tried this.

Here are some sample prompts that I tried and got full working code for:

- "write pytorch code to train a transformer model on common crawl data and an inference service using fastapi"

- "write react native code for a camera screen that can read barcodes and look them up using an API and then display info for matched results in a widget under the camera view"

- "write react code for a wedding website"

- "write code to deploy a django website on GCP using terraform and kubernetes"

- "how do I dockerize the app, it uses pytorch and faiss, also push it to a container registry"

- "implement a GPT style transformer model in pytorch", "write a training loop for it with distributed support and fp16"

- "how would you implement reinforcement learning with human feedback (RLHF)", "can you implement it in pytorch"

- "write code to compress a model trained in pytorch and export for inference on iOS"

- "how would you distill a large vision model to a small one"

- "what are the best CV architectures for mobile inference?"

For all of these it gave me code that was 95% usable, all in under 15 minutes, and which would have taken me a week or two to do on my own.

Oh sorry, I misunderstood "shipping products" to mean "physical shipping of physical products".

You know what's funny? I just asked ChatGPT to implement those exact same things and it shat all over itself producing embarrassing nonsense that won't compile, let alone do what they're expected to do. Bugs and incomplete code everywhere.

You'd have a much better time just Googling those asks and re-using a working examples from SO or GitHub. Which is ironic, given how ChatGPT is supposedly trained on those exact things.

I'm wondering how come we're both getting such vastly different results. Maybe your bar is just lower than mine? I don't know. I'm honestly shocked at the contrast between the PR given to ChatGPT, and the results on the ground.

Try this simple ask (the results of which you'll find plastered everywhere): produce a Python function that decodes a Base64 string and prints the results, without using any "imports" or libraries. Every single output I got back was embarrassing garbage, and I gave it something like 15 shots.

I tested the Base64 thing with GPT4 and it produces code that does seem to work. There have been other tasks I've given it (C++, Clojure, JS) that it doesn't get on the first try or in some cases doesn't get at all though. One task I tried in C++ it kept going in circles and ignoring requirements from prior prompts and I tried numerous ways to prompt it.

All that in mind, I'd be lying to say I'm not more than a little concerned with the progress from 3.5 -> 4. I'm only two years into my career and my fingers are crossed that it won't significantly impact the market for devs for as long as possible.

>since they trained on a huge chunk of web text

What did they trained it on? Why is it unprobable to train on a better/bigger dataset any time soon?

Because they trained it on a huge percentage of the existing web. There's isn't a (much) bigger and better data set available.

What percentage?

What about books, newspapers, documents, etc.?

The LLaMA paper [1] (Meta's model) contains details about what they trained it on. This includes all of Wikipedia, a huge part of the internet (3.3 TB + 783 GB), a huge set of books (85 GB). My guess is basically all high-quality English articles on the web have been included. Also almost all English books must be included. Newspaper archives is about the only thing I see as missing, as well as more non-English sources.

[1] https://arxiv.org/abs/2302.13971

OpenAI is working with Microsoft so they definitely had access to the full Bing index and data from their other platforms like Github and Linkedin. They also paid for private datasets, from what I heard they might have gotten a copy of Quora and I'm sure they got a dump of all digitized books from someone.

Their best bet now is getting more supervised conversational data, which they should be getting a ton of from Bing and ChatGPT usage (they can use it as is with RLHF dataset which they had to pay people to generate by having fake conversations).

I wouldn't be surprised if they partner with Microsoft and hire a large team of doctors to tune it to handle specific medical conditions like diabetes.

Wrong https://www.digitaltrends.com/computing/gpt-5-artificial-gen...

As far as I can tell, the only source in this article is a tweet from a developer with no connection to OpenAI