Just one example: parsing structured data out of a big pile of poorly formatted PDF documents.

That used to be too difficult and expensive for me to tackle without a small army of data entry people to help do the work.

Today I can point Textract OCR at it and then use a language model to extract structured data.

(I haven't implemented this particular example just yet, but I'm looking for an opportunity to do so.)

- Asking it to summarize text

- Using it to extract facts from text and present them in an alternative format - turning a chunk of HTML into JSON for example

- Creative writing - poems, stories etc

- Getting feedback on your own text - asking it what should be tightened up, which bits are confusing and so on

- All kinds of code generation activities

I haven't done exactly that, but based on similar examples this is likely very vulnerable to hallucinations.

Here's a similar trick I did with Copilot: https://til.simonwillison.net/gpt3/reformatting-text-with-co...

Sounds like an interesting project! Who are the users?

I remember a long time ago Joel from Joel on Software made a post/comment (I can't find it, so it must have been a comment) about how the hard part of programming is interpreting the spec into code, and that programmers often get mad because the "spec isn't complete". Of course the spec isn't complete, if it were complete, it would cover every edge case and be as complex as the code. If you could have AI turn a spec into code, then the real intelligence goes into writing the spec to cover the complexity.

It's ok if AI can generate boiler plate for functions or even regurgitate leet code solutions (heck, I gave it the typical "coding challenge" we use here as our first filter and it did a good job). The real intelligence in development is to know what algorithm you want to use for your current problem.

So to me ChatGPT isn't that big of a deal. It does some neat things, and it may make some jobs redundant, but I don't see how it could replace the real value that humans bring to a problem. It may keep replacing the bottom tier workers, but that just frees people up to bring value higher up the value chain.

> Of course the spec isn't complete, if it were complete, it would cover every edge case and be as complex as the code.

Fair, but triggering nonetheless because I've definitely seen this argument used as an excuse for why there was basically no spec aside from "an iPhone app used to rate beer" and that there is some kind of value in such a contribution that couldn't otherwise be found by pulling in random strangers from the street and asking them about cool things they wished that computers could do.

If that's the case, then the spec is code. And this isn't the first time something like that has happened: people who write assembly might say to someone writing Python, "How can you call that code when you don't even know which register is storing a value?"

I think at some point this will stop being true. Obviously people displaced from jobs in the industrial revolution found other work, but at some point we can't expect the bottom 10% of a population (in terms of ability to perform non-menial jobs) to do "higher value" work.

I don't say this in any disparaging way and I am not in any way demeaning them or suggesting they don't matter. But they don't necessarily fit into society's expectations for "higher value" work.

Bottom line is, AI and automation can cause short term pain, but the upside is enormous. Successful countries will be those that can manage that transition.

This sounds great in theory, and I fully agree, but the system of compensation and wealth distribution will need to be turned upside down for that to happen.

The value created by current and future automation will somehow need to be captured and distributed to people whose financially profitable jobs are going to be replaced with unprofitable, but socially-beneficial work, there's really no way around it. Those people will still need to be fed, housed and have access to at least basic luxuries.

Furthermore, what do we do in a future "ideal" world where robots and AI are capable of providing basic necessities to sustain every human alive? Capitalism would likely break down one way or another, with branching paths that either take us to an utopia or a nightmare dystopia.

My concern is for people like my son who has autism. He has a job at the airport slinging bags into and out of planes. It is a job he enjoys because he likes transportation (trains, planes, etc.) Realistically, it is a job that is at the very top of the limit of his abilities. If an AI robot took over his bottom tier job, he would not be freed up to bring value higher up the value chain. He would probably become homeless if we were not around to support him.

There's your "change everything" -- it might not be a "strong AI", but if people can argue that Searle's Chinese Room [2] is "actually" talking, and it says useful, monetizable things, then it's close enough to be disruptive.

[0] https://www.theregister.com/2023/01/24/chatgpt_exam_study/

[1] https://nitter.1d4.us/sergeyi49013776/status/159843047987885...

[2] https://en.wikipedia.org/wiki/Chinese_room

The AI models just seem so clearly and instantly more useful to me.

To you yes. Now go out in the real world in which most people don't work in an office and mostly use internet for entertainment

Crypto seemed very useful to many people, and they still do, you'll find thousand upon thousands of comments and this very website preaching cryptos as the next game changer

With AI, the biggest claims I see are overwhelmingly from people who are not doing cutting-edge work in the field, who have no real foundation for a belief that these AIs will continue to improve at a dramatic rate. Because to really change the world, they do need to get a lot better.

I'm a big believer in the "capability overhang" idea, which is that the existing language models still have a huge array of capabilities that we haven't discovered yet.

That theory seems to be proved correct on a constant basis. Even the classic "let's think about this step by step" paper came out less than a year ago: https://arxiv.org/abs/2205.11916 - May 2022.

This paper (https://arxiv.org/abs/2206.07682) also touches on a pretty fascinating phenomenon - that when scaling up large language models they seem to "naturally" obtain new emergent abilities that do not exist on smaller models.

But honestly you sound just like someone in 1996 going "Oh the internet isn't going to change anything and is just a fad", and here we are decades later and the internet has changed almost everything in our lives. Every person you know uses the internet every day on their cellphones in one way or another.

I wanted to refute your point by giving some YouTube videos about practical uses and their views. Then I checked YouTube's trending videos and compared the view counter to that of a PewDiePie video of 2 days ago and now I agree. You are right.

The person I responded to was saying that they (as a dev I believe) have been seeing huge productivity gains _right now_ and that's what I'm interested in.

write a function using the python requests library which makes a get request to the URL example.com, parses the JSON response and returns the value of the "foo" field. Throw an exception if the get request fails, the response is not JSON or is invalid JSON, or if the foo field is not present in the response.

I just tried this and got a correct (and reasonable) function on the first try.

This kind of high level description to low-level implementation is a huge timesaver but it saves time in a different way than copilot.

I couldn't get Copilot to spew anything like that (a single simple test at best, and it fails at that more frequently than it produces something useful).

It's also quite good at converting relatively simple programs or configs between languages. For example, I used it to convert PostgreSQL DDL queries into Hibernate models (and also in reverse), JS snippets into OCaml, XML into YAML, maven pom.xml into gradle build scripts, and a few more.

Also if you use vim you can try my npm package `askleo` `npm i -g askleo` (not tied to the website but requires your own OpenAI API key) with `:r ! askleo Go function to reverse a list of numbers` or whatever .

People are claiming to use it to write code, I'm curious how sophisticated said code is. Sure it might take out of some of the grunt work, like an ultra-sophisticated find-and-replace, but you still have to review all the changes it makes and correct any mistakes so the only thing it really saves is the typing. There's no way you could ask it to write code for a sophisticated architecture without extensive training on said architecture, and I'm not sure how you would even train it for that (can it parse design documents? Diagrams?)

The "Look, AI is replacing creative work first, the thing we thought was most immune to AI!" narrative annoys me. IMO it's just yet another factor revealing how little people value "creative" work. There's a reason relatively simplistic Marvel movies make the big bucks and the erudite starving author/artist is a meme. The market does not appreciate creativity for its own sake, and never has. No one cares about the reincarnation of William Shakespeare if he' s using all his talent to write blogspam. No one cares about Monet's ghost's DeviantArt anime titty drawings. People care about creativity because it's a requirement to produce something new and useful, that use can be pragmatic or symbolic, but if it's neither no one cares and the AI-generated equivalent is good enough.

You want to see AI-proof creativity (at least currently available AI)? Look at any luxury automobile interior. Look at any sophisticated software/hardware architecture. Look at an aircraft carrier or any other item where there aren't millions of samples to train on. That's not to say AI couldn't contribute to the tools that make these things, but no one working on the above is losing their job to AI any time soon. It's just taking out some of the low-hanging creative fruit. Maybe it's the start of an all-consuming revolution, or maybe this is as far as it goes. Only time will tell, but I've seen enough false revolutions (self-driving cars, AI advertising, crypto) to not buy in until I see hard data of it doing something more than writing convincing youtube intros and being used to cheat on high school essays. If the tools turn out to produce genuine value, then I'll learn how to use them to maximum effect at my job. Nothing to get wound up about either way.

tl;dr: in some instances I had to coach it pretty directly to get what I wanted. In other instances it was flawless. And to your point, some of the code it generated was both clearer and faster than the code I would have written for the same task.

Any relationship not visible in the code seems to be outside its capabilities to understand for the moment. I'll be impressed when I can point it at a server cluster and the associated dozen repos, give it some clues, and it can understand how the code for a server cluster interacts with said cluster's configuration and database hookups by simply scanning the files/repos and the info I textually provide.

This "if you wait to see results the opportunity will be gone!" mentality is for VCs and other people who's business models require them to be way out on the risk curve, who make a lot of bad calls, but lose relatively little when they fail. It was also partially a product of low interest rates. It is not applicable to most individuals/organizations.

So, if you are using it as a “personal assistant” and “supervise it” is a big (very big) deal since it's going to save a lot of time.

There are also some unknowns related to the cost/time needed to train and run the infrastructure behind it that could change a lot of things.

Maybe can be useful if actual humans validate/fix ChatGPT response, but in this case probably better to put effort to fix response directly in ChatGPT.

That's the problem with Copilot. If I'm not familiar with the API calls then I'm still going to have to dig into the docs. I could run it and it might "work", but that doesn't guarantee that it's correct. There are a whole lot of things in the C API that work but are not correct, such as the gets() function. If I have to do such legwork then it's just as easy to write the code myself.

The utility of AI is proportional to the trust one has in it. Trust is easy to lose and hard to regain. It will take just a few AI mishaps to ruin a product or even an entire industry.

It's an assistant that creates drafts for you. You still need to check them, but usually reading is a lot easier and less time-consuming than writing. I used it a couple of times to compose some long replies email, and it was just fantastic. I had to fix some minor stuff, but I complete the task in less than 5 minutes while without ChatGPT it would take me about 30 minutes.

> That's the problem with Copilot. If I'm not familiar with the API calls then I'm still going to have to dig into the docs. I could run it and it might "work", but that doesn't guarantee that it's correct.

You have to run/compile it anyway and if you combine it with existing tools (linters, type checkers and so on) you will detect this kind of anomalies very soon.

I know people that use it to write slide deck copy, which I think is nice because no one really cares about that and it saves some time. I do think it's a nice tool to get a general idea of something or even can be used as a writing prompt.

For some coding answers as of now, it's actually just easier and faster to use Google/Bing and find an answer on StackOverflow.

"Personally I can give you a formal definition of intelligence and even a number of speculative sketches of how I think it could be implemented, but I will also tell you that strong AI is not within sight and that recent advances are not moving us in the direction of strong AI"

Tools that assist in knowledge work can be very useful and very impactful on the knowledge industries without needing to even define "strong AI", let alone "being smart", "knowing ideas", et al.

What's important is not what we call these things or how we talk about what they do in an abstract manner rather if these tools are useful.

There's a general lesson here: Don't waste your time talking yourself in philosophical circles about what "AI" means, or what "knowing" means, or what "ideas" mean. If you'd like further convincing please see Philosophical Investigations by Wittgenstein. If pressed I'm just going to do a cheap imitation so you might as well get it from the source.

Personally, I have found ChatGPT, Stable Diffusion, Copilot, and a number of other large language models and tools built on large language models to be very useful.

I fed a folk song I was working on into ChatGPT and asked it to conjure up some similar evocative scenery... I then generated animations with Stable Diffusion based on a linear interpolation between those descriptions of that evocative scenery in the latent space... Then I wrote wrote some more lyrics about the weird stuff Stable Diffusion was hallucinating... and in the end I had a bunch of visual images, a song, and some animations, all that seem worthy of publishing. The visual components would never have been made as I don't have the time or money to do much more than sit around with my acoustic guitar and write songs.

https://www.dropbox.com/scl/fo/yq5ar9moacgilrqxqpjr1/h?dl=0&...

The images are in png format and have the prompt and model used in the info section of the file, which you can see if you open the "info" panel on Dropbox:

https://www.dropbox.com/home/DistantDesertHighway?preview=C7...

And here's the song:

  distant desert highway
  
  C                       F
  I'm on a distant desert highway
           C                     G
  I'm just trying to find my way home
           C                           F
  But this world is all empty and I'm alone
            C                 G                      C
  Spare the reptiles, and the truckers, and the neon homes
  
  
          F                              C
  And the way that you looked at me last night
         F                                         C
  Is the way that I stare at the sun at that great light
          G                               C
  I had a dream you came to me in another land
            G                                      C
  Where the forests grew over all where we used to stand
           F                  G                 C
  Over the highways, over the deserts, over our hands
  
  
  I'm on an untimely troubled Tuesday
  I don't need to know all the beasts that roam
  But this world is all empty and I'm alone
  Spare the cones, and pot holes, and the buffalo
  
  
  Dancing in high heeled hooves
  It's getting hella weird out there
  Better stay inside for awhile

It's fantastic at generating content that on first glance looks remarkably right, but always fails fine inspection.

All I've seen from LLMs is much better demos of the types of funny things markov chains were generating two decades ago (anyone remember the various academic paper generators?). However I have yet to see anything that stands out as really remarkable.

My read is that people want to see incredible progress towards strong AI, and LLMs do a great job of letting people feel like they're seeing that if they want to.

I suspect in 5 years we'll largely have forgotten about LLMs and in 10 they'll come back into popularity because techniques will become more efficient and computing power will increase enough that people can train them on their home PCs... but they'll still just be a fun novelty.

Have you actually looked at the linked Cheat Sheet?

What is in sight is AI that can (sometimes) fool us into thinking it is "strong". And whether it is or not becomes a point for philosophers to debate.

We're heading toward a situation captured fairly well in "The Good Place" -- is Janet actually alive? Is killing her wrong? She would tell you no, as long as you're not approaching the button that actually reboots her :-)

I.e. were not good at predicting how far away stronger AI is. Even if we can comment on strong/weak AI.

Or maybe you've fooled yourself into thinking that "strong AI"/general intelligence is actually not just a bunch of tricks that mash together into a smorgasbord that usually works quite well. That includes human intelligence.

Including human intelligence, as you say. I'm reasonably convinced that "intelligence" is an emergent higher-order abstraction that arises from deterministic processes, at the cellular, or digital, level.

I would, however, make a distinction between resilient and brittle intelligence. ChatGPT, for example, seems very human-like for some percentage of responses, but when it goes off the rails, it goes completely wrong. Humans can sometimes be like that, but (I don't think) to the same degree, and certainly they are often capable of a softer landing when they get into unfamiliar territory.

It's both IMO. It will change everything but is also overhyped at this point.

My feeling is that it (well, LLMs in general, ChatGPT is just the particular one getting public attention) is an overhyped major advancement that is going to change almost everything, but not as much as it is being hyped as changing everything.