Techies will realize that they are just giving ideas to O̶p̶e̶n̶AI.com, Microsoft Word, Google Docs and Notion. It is just the same AI bros re-selling their hallucinating snake oil chatbot that are under a new narrative for AI.
There is a reason why the only safe serious use-case of LLMs is summarization of existing text, since everything else it does is untrustworthy and is complete bullshit.
Their so-called 'revolution' is a grift.
It doesn't cover everything OneNote 2016 did, but it does a lot more in other areas and it is progressing nicely.
Summarization of existing text is the *only* safe and serious use-case.
The use-case is anything where occasional bullshit output is an acceptable downside to speeding up. More reliable outputs will enable more use-cases.
Want a flat-earther version of New York Times (The New York Flat Times)? Done. Want a just slightly insidiously fascist version of NPR? Done. Want a pro-Nato version of RussiaToday (WestRussiaToday)? Done.
And we already know people share stuff without checking for veracity and reliability first.
0: https://www.bing.com/create
作成 *芸術* 開始日
AI を使用した単語Right now, they are definitely useful time savers, but they need a lot of handholding. Eventually, someone will figure out how to get hundreds of LLMs supervising teams of millions of LLMs to do some really wild stuff that is currently completely impossible.
You could spin up a giant staff the way we do servers now. There has to be a world changing application of that.
> get hundreds of LLMs supervising teams of millions of LLMs
What does this mean or what can you do with this setup… do you mean running LLMs in parallel?
This is an intuitive direction. In fact, it’s so intuitive that it’s a little bit odd that nobody seems to have made proper progress with LLM swarm computation.
So again, what is the actual difference you are imagining?
Or is it just that distributed X is fashionable?
Just because the swarm infrastructure hosting an LLM has higher latency across certain paths does not make it a swarm of LLMs.
Interesting, I haven't heard of that. Can you name examples?
If 80% of the processors in a cluster are running 'general LLM' and 20% are running 'math LLM' are they the same cluster? Could you host the cluster in a different data center? What if you want to test different math LLM modules out with the general intelligence?
In the case of the brain, while certain functional regions are highly specialized I would not consider them "a small separate brain".
One LLM is limited, one obvious limitation is its context window. Using a swarm of LLMs that each do a little task can alleviate that.
We do it too and it's called delegation.
Better to limit his incompetence to one position.
So, perhaps, there aren't swarms yet just because there are easier ways to scale for now?
For the same reason one genius human does not suddenly need less support staff, they actually need more.
Edit: and why it isn’t here yet is because it’s new and hard.
It's early days. There will be a GPT 5 I'm sure, maybe that one will be better at teamwork.
Please don't. You've already ruined enough industries. Let the MBAs do finance and Wall Street and leave them out of the chain of command in organizations that make things.
Yes, some MBAs fuck things up. Just like some CS grads fuck things up. But advocating against the study of business is just as naive as advocating against the study of computer science just because there are some bad CS grads.
Are you contending that business were not successful before Wharton started pumping out MBAs?
> But advocating against the study of business is just as naive as advocating against the study of computer science
I didn't say 'don't study business', I said 'stick to finance'. MBAs tend to end up destroying innovation and productivity for short term growth and stats.
Jack Welch showed what a successfully motivated 'business oriented' leader can do to an innovative and productive legacy organization when given complete control over it. The MBAs happen to just do it on a smaller scale.
Criticizing garbage MBA programs is not criticizing the study of business. Business schools don't study business. They're a place where people make a lot of money selling theories about business that are useless at best and it many places, quite harmful. Learning about business by going to business school is like learning to kiss by reading books about kissing.
I would say that just as every person is unique so is every company unique. And just as there is plenty of pseudoscience plaguing psychology so are MBAs full of pseudoscience. Two fields that are far too obsessed with generalising their advice. Which is not to say that there aren't any useful ideas in these fields. But the vitriolic reaction above is warranted.
The human brain can hold much more than 9 things and even though AI will be used in medicine broadly very soon, I really want the final diagnosis done by a human.
Once true AGI arrieves, I might change my opinion, but that might take a while.
We have been hearing this since forever.
Revolutions do happen but not the way we expect. My anecdotical experience: no one in my team of about 30 people developing SW uses ChatGPT or similar in their day to day. This may change, or not.
Granted, currently deployed systems are mostly awful, way behind the state of the art, and therefore mostly useless. Maybe it's because designing medical devices and getting them approved takes so long. Maybe it's because the manufacturers put AI in there for marketing purposes only, while assuming nobody will use the suggestiona anyway. In any case, I strongly expect the trend to continue and these systems to become very useful quite soon.
Try writing a number from one piece of paper to another. If it's more than 7-9 numbers, you won't do it in one shot, unless you spend extra time memorizing it.
I’m proficient at math, but my working memory is around 6, so I cannot add two three digit numbers to each other in my head (unless I see numbers to be added in front of me).
I will. As another commenter says, the brain isn't limited to 9 things at all. There's no way that I'll trust the diagnosis of a machine that won't understand me.
If a doctor uses AI to help with research, that would be OK. Just so long as the doctor is actually the one doing the diagnosis and prescribing the treatment.
That is just the introduction, showcasing what level of sophistication you get with just Google and Wikipedia as tools
Now imagine task rabbit or fiver as tools. Ai can make things happen in the real worlds.
These llm have limited attention but infinite focus. You can parallelize them, you can have one direct a fleet of other llm, you can have llm checking input and outputs for correctness from the other models and feedback that information to the controlling model so that it can improve the promp to the other as it tries to reach it's goal
And the goal can be far fetching (manufacture fake artsy trinket and import them from China to distribute etsy) or nefarious (produce subtle propaganda in a moltitude of wordpress website, register accounts on Wikipedia, reddit, create a sophisticate network of citations)
Seems I and you have different Googles and you still have the one I had pre 2010.
For over a decade now, Google has been including things I never asked about to the point where it would sometimes be easier to find it using Marginalia.
Some say it is just because internet has changed and there is less ham and more spam, but the last few months I have been using Kagi and it proves it is possible to create a better search experience.
And, if Google works for you, fine. Maybe you search other topics, use other keywords or are in another bucket wrt experiments, but from my perspective Google is now the same as its predecessors.
For politics shopping and some other topics it can be terrible, but I don’t think GPT is good at those either.
I’m actually happy to be proven wrong here. If you have some examples let’s test it out. If it’s a true step function improvement I’d expect it to be easy to source examples.
Let me repeat that: You can program GPT in English. ENGLISH!
You're complaining about the first nuclear test bomb being impractical and uninteresting. How will this change the world? That huge monstrosity had to be affixed to the top of a test gantry and took years of effort by a veritable army of the best and brightest to make! No way it could change war, or geopolitics, or anything. No way..
This is the day after Trinity. The bomb has gone off. A lot of physicists are very excited, some are terrified, and the military is salivating. The politicians are confused and scared, and the general public doesn't even know yet.
That doesn't mean the world hasn't changed, forever.
What does that mean to "program GPT"? Do you mean program (software) USING GPT?
I thought we already had COBOL, which is pretty much like English so business people can use it. Same for SQL.
And don't we already have lots of low-code or no-code tools? Why do we need to program with ChatGPT if we already are beyond programming?
Not in the sense that you get a computer program out (though you can), but in the sense that it can automate anything without even needing a programming language, compiler, and domain specific UX.
Low code and no-code tools still require thinking like a programmer. You define what you need to do, then implement, then get results. GPT often lets you go directly from spec to results.
If the goal is programming, GPT is nothing special. If the goal is quickly reasoning over very abstract instructions, it’s amazing.
The trick is seeing the new use cases. It really does come back to the GUI revolution: if you want to list files in a directory, the CLI is just as good, maybe better. But GUI makes photoshop possible.
GPT makes it possible to say “summarize the status emails I sent over the past year, with one section per quarter and three bullet points per section”. And the magic is that is the programming.
A sibling comment already explained the second part of the question, but there is something I find more exciting. You can program GPT, as in you can tell it to change its behavior. The innumerable "jail break" prompts are just programs written in English which modify GPT itself. Like macros in lisp I guess. The first time I truly saw this potential was when someone showed me you don't actually have to change the temperature of chatGPT in code, you can just tell it to give low and high temperature answers in the prompt[1]. That's programming the model itself in english.
> Let me repeat that: You can program GPT in English. ENGLISH!
How?
Let me repeat that: How?
I had a little script that from time to time parses a list of jobs from a specific board, extracts some categories, inserts them into an SQLite and have a frontend that displays them to me in a way I want.
The board has since changed some things which would mean maybe 2 hours of commitment from me to update the script.
How do I program GPT in English. ENGLISH! To do that for me? What are the steps involved? I've been using ChatGPT and GPT-4 for awhile and I can't imagine what the steps are to make this happen without a lot of back and forth. I can't imagine how to program the infrastructure. I can't imagine how the API endpoint is more than a fancy autocomplete. I need help understanding what it means that I can program it in ENGLISH! (I can also program it in my country's language for what it's worth).
> That doesn't mean the world hasn't changed, forever.
I sort of agree with this.
> Auto-GPT is an experimental open-source application showcasing the capabilities of the GPT-4 language model. This program, driven by GPT-4, autonomously develops and manages businesses to increase net worth. As one of the first examples of GPT-4 running fully autonomously, Auto-GPT pushes the boundaries of what is possible with AI.
Perhaps this is the part you're missing. When I've watched people program with ChatGPT it _is_ a lot of back and forth because an enormous amount of context is able to be stored and back referenced. I.e. one wouldn't say "make me a Flappy Bird clone for iOS", they'd start with:
"Give me the code for a starter SpriteKit project". Then
"Now draw a sprite from bird.png and place it in the center of the screen".
"Now make it so the bird sprite will fall as if it's affected by gravity"
I won't bore anyone with how might one go from that all the way to a simple game, but I'm sure you see the idea. There are obviously _huge_ limitations to this approach and professionals will get hit them fast, but the proof is in the pudding: people who can barely code are producing real software through this approach. It's happening.
I've tried to build a lot of fun stuff with it so far. Haven't been able to properly 'program it in English' for anything non-trivial. Back and forth ended up in loops of not what I wanted. I'm just utterly confused at the difference in experiences I've had with it vs. what some people are preaching.
> There are obviously _huge_ limitations to this approach and professionals will get hit them fast, but the proof is in the pudding: people who can barely code are producing real software through this approach. It's happening.
I've had 4 product people I know try to create products using ChatGPT. All 4 of them basically got stuck on the first steps of whatever they were trying to do. "Where do I have to put this code?", "How do I put it online?", "How do I store user data?", "Where do I get a database from?". Basic questions to any professional, but to them it was impossible to overcome the obstacles from code to deployment.
I don't doubt that it's happening and it will become better in the future; I'm just having a hard time trying to grasp where some people are coming from when my experience as a professional, using it, has been mixed.
expectations: using to the LLM to break problems into steps, suggest alternatives, using the LLM to help them think through the problem. I think this is the people using it to write emails - myself included, having a loop to dial in the letter allows me to write the letter without the activation energy needed to stare at a blank page
empathy: people who've spent enough time interacting with an LLM get to know how to boss it around. I think some people are able to put themselves in the LLMs shoes and imagine how to steer the attention into a particular semantic subspace where the model has enough context to say something useful.
GPT4 writes boilerplate python and javascript servers for me in one shot because I ask for precisely what I want and tell it what tools to use - I think because I have dialed in my expectation for what it's capable of and I learned how to ask in precise language, I get to be productive with GPT4's code output. Here's a transcript: https://poe.com/lookaroundyou/1512927999932108
Link: https://github.com/kaegi/alass
It's basically magic.
For problems that are fine being defined ambiguously. Try to program a database in English, let's see where it goes.
Meanwhile, if you give Chat GPT your database schema and ask it to write a SQL query for a report, it can do that for you.
Nonetheless, it could prove useful for looking up algorithms, patterns, and generating boilerplate code. However, an important issue is will it generate similar code if queried at a later time? Not likely, which will make it less useful or result in an inconsistent codebase. Maybe you can request a version of the code generation? In-house code generators will generate consistent code, so it will be interesting to see how it is used in real projects.
https://gist.github.com/int19h/4f5b98bcb9fab124d308efc19e530...
Note that in this case it isn't even asked to write specific queries for specific tasks - it's just given one high-level task and the schema to work with.
You're right, though, that the effectiveness of this approach depends very much on schema design and things like descriptive table/column names etc (and even then sometimes you have to make it more explicit than a human would need). You really need to design the schema around the capabilities of the model for best results, which makes it that much harder to integrate with legacy stuff. Similarly, not all representations of data work equivalently well - originally, I gave the model direct access to the typed object graph, and it handles that much worse than SQL. So if your legacy software has a data model that is not easy to map to relational, too bad.
On the other hand, GPT-4 is already vastly better at this kind of task than GPT-3.5, so I think we can't assume that this will remain a limitation with larger models.
This may end up being a feature of some high level frameworks … “compatible with ChatGPT” or “designed to work with xxx LLM”.
Perhaps I could interest you in some Radium Water? It's new and trendy and good for your health.
If your prompt is garbage then the output will be garbage and if you don't know how to program you won't even realize the output was garbage.
It's not the language part of programming language that is hard. It's the programming part because it means you have to have a good understanding of what you want. Just like a human programmer won't read your mind an AI programmer won't read your mind either.
But I can already foresee bosses dismissing employees that raise issues (performance, maintainability, scalability, etc., etc.) by saying "Look, the AI can do it. So if it can do it you can do it too.". I foresee this because I have already seen it.
That's why it makes this so interesting - this type of automation impacts our jobs directly. Of course, I'm not sure who would use this in a corporate codebase without legal concerns.
The very existence of "prompt engineering", numerous discussions about how to prompt ChatGPT in order to get the result you want, etc. imply that while it may be in English, it still requires similar care and attention to do properly as a programming language does.
Which makes me wonder what the advantage of using English is. A formal language seems like it would be more productive and accurate.
The advantage of using English (natural language that is), the humans around you tend to speak it. I don't naturally speak powershell. Instead I want a script that searches for particular filenames, under a particular size, between a particular date in a directory path I specify. I told GPT I wanted that and in a few seconds it dumped out what I needed. It wrote the script in a formal language, which is then interpreted by the machine in an even more formal manner. Let the code deal with accuracy, and lets let language models argue back and forth with humans on intent.
This is true, but of limited utility. English is so bad at this sort of thing that even native-speaking humans are constantly misunderstanding each other. Especially when it comes to describing things and giving instructions.
That's why we have more formal languages (even ignoring programming languages) for when we need to speak with precision.
As far as formal languages... GPT doesn't know Lojban well, presumably because of its very small presence in the training data (and dearth of material in general). But it would be interesting to see how training on that specifically would turn out.
Yes, and with people, that's insufficient if you really need confidence of understanding.
There's a reason that lawyers speak legalese, doctors speak medicalese, etc. These are highly structured languages to minimize confusion.
Even in less technical interactions, when you need to be sure that you understand what someone else is saying, you are taught to rephrase what they said and tell it back to them for confirmation. And there's still a large margin of error even then.
This is why, whenever I have an important conversation at work, I always send an email to the person telling them what I understood from our exchange. In part to check if I understood correctly, but also so that I have a record of the exchange to cover my ass if things go sideways because we didn't understand each other, but thought we did.
I don't know any other library I could just use with this task with nearly the same quality besides some regex soup.
- summary of all the carbon neutral concrete methods, especially ones that can be done in a small industrial workshop as a prototype
- I have allergies in Thailand, mid-february. What may it be related to?
- list all the companies from Japanese stock exchange that have high debt rate
Those are top of my head, but really anything that is either a super-specific niche, or requires merging a few niches together, Google won't help you with.
Humans are really gullible for the appearance of confidence. And humans are also very prone to wishful thinking.
It's not just one clean answer and we're done. In my experience it is helpful in breaking the problem down into stuff you can Google.
Yeah I can see that being useful. I’ve also seen a lot of non-technical people straight up accept whatever comes out of it, so that’s a little worrying. It’s true of Google searches too, of course, but at least a google search gives N results someone can check rather than 1.
With the example questions I provided, it would take many hours to do research on the subject. GPT provided initial answers instantly, and then fact checking was easy.
That’s what we did with gpt-3. With plugins you can have gpt fact-check itself.
Also, if you have a system for dedicated knowledge, you can use embeddings - with embeddings gpt has very little room for hallucinations, and it can provide detailed references.
100% agree and so glad to see someone else say it. I feel like people are losing their minds every time we go through the same hamster wheel.
To hear first hand, in the article above, the effect this is having on ML engineers breaks my heart.
Yes, if you try hard enough, you can try to cast transformational shifts as trifling.
- e.g. “Barteen, Shockley , and Brittain made a smaller version of the vacuum tube.” (transistors)
- “Scientists discovered that light could carry information, like electrical wires do.” (fiber-optics)
The effects (including the harder to measure cultural shifts) matter more than some uncharitable characterization.
Also, the “it is not X” thinking is the result of present fixation. Such argumentation is, at best, quite narrow. Perhaps applicable in specific defined markets and situations but hardly a good mindset for making sense of how the world is changing. Hence the cliché, “The Stone Age didn’t end because we ran out of stone.”
The psychological undertones in the comment above are probably “people, stop exaggerating”. From one overreaction to another, it seems.
However, once you actually start using it and see that the "ten minute walk" is actually an hour walk, or that a full third of the attractions it has shepherded you to are permanently closed, you realize that building that itinerary yourself from scratch using Google or TripAdvisor would take you less time than manually double checking everything ChatGPT says.
It's also quite surprising that people still think ChatGPT is capable of logic. Even for a complete layperson, all it takes is asking it to draw someone's family tree as an ASCII chart to see that text prediction only goes so far and there's not enough of a relational concept in there to comprise knowledge. There are many examples of asking it to solve famous puzzles with minor variations where it fails spectacularly.
The marketing behind ChatGPT is genius, but there is only so far you can go before the honeymoon is over and people start to really question what you brought to the table. Aside from that, ChatGPT isn't unique in what it can do, and others (including open source) are catching up fast.
That being said, I'd still use it for something like language learning (and other types of learning), where follow up queries (such as why you'd use one word instead of another, or how to rephrase something to be more polite) unlock a significant amount of value. It can also be useful to write trivial code, though I doubt a serious professional would do this (for several reasons, such as privacy and liability).
It seems that as of right now, the killer app of ChatGPT is the boost in views you get by putting it in the title of your YouTube video.
You mention blogging from the standpoint of writing it all yourself, and then using a tool to tweak it. That's not the revolutionary part. It's collaborating with the tool to write the post.
OpenAI revolutionized… rewriting things with slightly different wording?
I’ve seen so many breathless people posting “this would have taken me so long to search” and then I type 3 keywords from their massive prompt they crafted and find it instantly on Google. We’re talking 1000x or more faster. I feel like the same is happening in your comment. How often have I thought “damn I wish I wrote this blog post ever so slightly differently” in my life? Maybe a handful of times? And yes I’m including all generalizations of that question.
But certainly fake girlfriends and summarization will be mid size fields. Image generation has some mid size potential. But these will be spread between many companies.
I really think it has uses no doubt, but is it a revolution? Where? It’s not creative in the valuable sense - media, art, fashion, etc all will adopt it marginally but ultimately it will actually only serve to further the desire for genuine human experience, and cohesive creativity that we see it really falls flat at. It saves some marginal time perhaps if you’re ok sounding like a robot.
Taking into account the downsides it looks like a hype bubble right now to me, and a draw in the long run. There’s just a whole lot of tech people trying to cash in on the hype.