News for hackers. Stuff that matters.
Re:The Rise of the AI Engineer by cj on Friday June 30, 2023 @02:29PM (#36539872)
I can see a world where "ML Engineer" (or similar) is someone that's hired to solve a known problem (whether it be with classifiers, LLMs, neural nets, etc), whereas a "AI Engineer" (or whatever the title) is hired to figure out how the hell to capitalize on the AI hype, without a specific problem to solve.

IMO right now we're entering the "Peak of Inflated Expectations" in Gartner's hype cycle model. https://en.wikipedia.org/wiki/Gartner_hype_cycle#/media/File...

Lots of companies want to jump on the AI bandwagon, but they don't really know what to do or how to leverage it.

What the LLM community needs now is for companies to leverage and productize these LLM models for truly game changing use cases. If that doesn't happen soon, the hype will start to fade and lose momentum ("trough of disillusionment").

I'd really love to see some killer use cases emerge soon.

Re:The Rise of the AI Engineer by richardw on Friday June 30, 2023 @02:40PM (#36540072)
* * *
Re:The Rise of the AI Engineer by moffkalast on Friday June 30, 2023 @03:39PM (#36540997)
Well you have to consider the inherent bias on HN, in that the trough of disillusionment is HN's default state of mind regarding literally every topic. It's really rare to see a thread where most of the comments aren't negative.
Re:The Rise of the AI Engineer by staticman2 on Friday June 30, 2023 @09:36PM (#36545342)
Ha, when GPT 4 launched the default HN post seemed to be saying LLM's were about to exceed human intelligence and the singularity was upon us.

The disillusionment seemed to happen pretty fast.

Re:The Rise of the AI Engineer by yacine_ on Friday June 30, 2023 @02:37PM (#36540010)
The right way to leverage AI is to use it to do ye olde engineering, faster. Actually, most AI use cases are in fact that! Ye olde engineering - data pipelines, automation. Stuff we've been doing since before I was born.

Everything else - transformative tech, will hit the open domain fairly quickly, is my bet. Similar to databases.

Re:The Rise of the AI Engineer by rvz on Friday June 30, 2023 @02:51PM (#36540258)
We are indeed at the "Peak of Inflated Expectations" in the Garter hype cycle. Everyone is screaming that we are out of the AI winter and throwing LLMs at every problem.

> What the LLM community needs now is for companies to leverage and productize these LLM models for truly game changing use cases.

The killer serious use-case has always been summarization of existing text. That's it. Everything else is a constant flow of creative bull-shitting from a black box AI requiring triple checking everything before using the output, meaning that the output can't be trusted.

> If that doesn't happen soon, the hype will start to fade and lose momentum ("trough of disillusionment").

I think they will realize that there is more to "AI" than LLMs, just like the hype with CNNs and the like.

Re:The Rise of the AI Engineer by visarga on Friday June 30, 2023 @05:06PM (#36542277)
Not just summarisation, information extraction in general. LLMs are great data normalisers.
Re:The Rise of the AI Engineer by z3c0 on Friday June 30, 2023 @05:52PM (#36542875)
No. No they are not. Do not keep proliferating this idea, as there are real consequences. Your input only informs the likelihood of the output. There is no model actually extrapolating rules from the information, so the premise of LLMs being extractive is 100% verifiably false. They summarize well due to being handed a roughly correct arrangement of tokens to mimic the order of. This should not be confused with extraction.

To put it simply, keep falling for that, and it'll bite you in the ass.

Re:The Rise of the AI Engineer by visarga on Saturday July 01, 2023 @12:31AM (#36546605)
You are talking from a top-down point of view. Of course they can make an error any time, after all they stochastically sample the output. All approaches to information extraction are upper bounded at 90-95% accuracy, I have extensive experience with this task and run many evaluations on invoices, receipts, forms and other doctypes. Human in the loop is still required.

But in practice you can rely on good copying and excellent ability to parse names, addresses and various values as good or better than other approaches. LLMs reduce the number of corrections, and they are easier to deploy - instead of labeling hundreds of documents you can just query with the field name.

Re:The Rise of the AI Engineer by z3c0 on Saturday July 01, 2023 @04:35PM (#36554887)
* * *
Re:The Rise of the AI Engineer by pvman on Wednesday July 05, 2023 @05:37AM (#36597959)
Thanks for the info z3c0. Have you implemented open source solutions? Do you mind sharing some deterministic solutions to detect entities without huge dictionaries (NER)? Or maybe extractive QA if you used that instead?
Re:The Rise of the AI Engineer by z3c0 on Wednesday July 05, 2023 @02:27PM (#36604860)
Deterministic? Not anything that is "one size fits all". If your documents are of an unpredictable shape, then ML is your best bet. For both NER and QA, BERT models are very capable.

To your first question, most of my ML work is proprietary, unfortunately. I am hoping to change that in the near future.

Edit: spaCy is a great library for NER, if you're hoping for an open-source solution that can have you hitting the ground quickly. SQuAD for QA.

Re:The Rise of the AI Engineer by swyx on Friday June 30, 2023 @03:02PM (#36540421)
(author here) There are at least 4 killer apps ($100m/yr revenue potential) so far:

1. Generative Text for writing - Jasper AI going 0 to $75m ARR in 2 years

2. Generative Art for non-artists - Midjourney has by some accounts $80m ARR

3. Copilot for knowledge workers - GitHub’s Copilot has roughly 50-80m ARR as well

4. Conversational AI UX - ChatGPT probably has >$100m ARR by now, Bing Chat has brought $m's worth of attention to Bing

(More on agents as the immature #5: https://www.latent.space/p/agents)

what else do you need to see to believe? (geniune question)

Re:The Rise of the AI Engineer by cj on Friday June 30, 2023 @03:20PM (#36540713)
> what else do you need to see to believe? (geniune question)

It's not that I don't believe in LLMs.

It's that the level of media attention AI is getting isn't backed by the same level of real world use cases, yet. (All of the use cases you listed are awesome, no denying that, but I don't think those alone justify the amount of hype in mainstream media)

Re:The Rise of the AI Engineer by swyx on Friday June 30, 2023 @03:28PM (#36540837)
i mean, theres nothing i can do about mainstream media hype, but i guess my main point is this is a growing field with real money and utility behind it, and so will professionalize. if I am correct on that then AI Engineer will be a thing (because it is Least Bad title for the thing)
Re:The Rise of the AI Engineer by skepticATX on Friday June 30, 2023 @03:51PM (#36541174)
AI Engineers are just software engineers who use specific tools. If we want to use a fancy title then that's fine, I guess. But let's not pretend that a typical dev can't easily learn vector DBs, data pre-processing, fine tuning, etc.

None of these things require specialized knowledge in the way that say being an AI researcher would.

Re:The Rise of the AI Engineer by jsight on Friday June 30, 2023 @04:55PM (#36542155)
You could say the same about things like SREs or Devops. In fact, many of them transitioned from regular "software engineer" to these roles simply by learning closely related skills.

That won't stop the industry from inventing new, useful titles.

Re:The Rise of the AI Engineer by __loam on Friday June 30, 2023 @10:11PM (#36545630)
You are actively contributing to the hype cycle lol.
Re:The Rise of the AI Engineer by moffkalast on Friday June 30, 2023 @03:30PM (#36540874)
The tech problems will eventually get sorted out I'm sure, the most uncertain problems are legal. The world's copyright law is not even up to date enough to deal with the internet without crappy patchwork workarounds (like Youtube's contentID nonsense), much less LLMs.

Steam just banned AI art and text assets and built in generators because they don't want the liability of hosting it. Midjourney is getting sued by Getty, Copilot breaks GPL licenses and was even facing class action lawsuits, OpenAI trained using copyrighted and even pirated content, LLama models are unavailable for commercial use, AGI alarmists want to ban the whole thing altogether, etc.

A lot of end usage depends on how all of this plays out in courts over the next few years. Why would people dump capital into something that will be declared illegal?

Re:The Rise of the AI Engineer by jsight on Friday June 30, 2023 @04:57PM (#36542168)
Because if they don't, their competitors will, even if those competitors are overseas in a more lax environment.

Either IP laws will catch up, or the countries with better IP laws will become more competitively successful.

Re:The Rise of the AI Engineer by staunton on Friday June 30, 2023 @05:13PM (#36542366)
If people get used to LLMs governing their lives, those copyright laws will "just" be changed. People will not accept a legal challenge taking away their toys. Granted, we're not at that point yet. But the window for the legal stuff really making a difference is slowly closing. Meanwhile, the legal system is pretty slow.
Re:The Rise of the AI Engineer by echelon on Friday June 30, 2023 @03:09PM (#36540544)
The longer HN continues to doubt in AI's deliverables, the bigger our revenues and our moats can grow. Don't tell them.

(It's not that hard to hit $1M ARR with a good AI product. So many classes of new products and solutions have opened up.)

Re:The Rise of the AI Engineer by constantly on Friday June 30, 2023 @04:38PM (#36541900)
I’m sure I could find a similar comment about cryptocurrency or NFTs. There’s always snake oil salesmen.
Re:The Rise of the AI Engineer by jsight on Friday June 30, 2023 @04:57PM (#36542176)
I remember thinking this about Tesla too. They built a fairly massive automaker while everyone kept saying their whole approach was fundamentally doomed.
Re:The Rise of the AI Engineer by onion2k on Friday June 30, 2023 @03:52PM (#36541194)
75+80+80+100 is £335m per year in revenue. For a point of comparison, that's approximately 11 hours of Google's revenue.
Re:The Rise of the AI Engineer by pclmulqdq on Friday June 30, 2023 @04:30PM (#36541795)
The innovator's dilemma on display.
Re:The Rise of the AI Engineer by dist-epoch on Friday June 30, 2023 @04:23PM (#36541700)
I find myself using Bing Chat more and more instead of googling specific questions. And no, hallucinations are not a problem, because the questions are concrete and the answers immediately verifiable.
Re:The Rise of the AI Engineer by riku_iki on Friday June 30, 2023 @04:06PM (#36541439)
but rate of growth is likely 100-1000 times higher
Re:The Rise of the AI Engineer by ducharmdev on Friday June 30, 2023 @05:22PM (#36542485)
On a smaller time scale, is rate of growth that meaningful?

I guess I'm just thinking, until we see the how this all pans out over the next decade, we don't really know if the current rate of growth will hold, or if it'll plateau/slow down.

Re:The Rise of the AI Engineer by 0xEFF on Friday June 30, 2023 @05:43PM (#36542767)
Yes, it’s meaningful. If Google doesn’t fix search in the next decade then Bing (or some other winner) will be the service earning Google’s ARR in 11 hours.
Re:The Rise of the AI Engineer by riku_iki on Friday June 30, 2023 @05:54PM (#36542899)
> On a smaller time scale, is rate of growth that meaningful?

revenue in initial stage of extreme growth is much less meaningful.

Re:The Rise of the AI Engineer by staunton on Friday June 30, 2023 @05:08PM (#36542308)
Consider the possibility that may be a point in a technology's evolution where you can "look at what is being done with it" and conclude that it's useful from that, rather than just comparing revenue figures across domains.
Re:The Rise of the AI Engineer by ImaCake on Friday June 30, 2023 @06:56PM (#36543672)
Yeah the dollar values here are a distraction. I use copilot everyday at my Data Science job. Its useful!

What we are not considering is integrating a custom trained LLM into our work because the tech just isn’t there yet.

Re:The Rise of the AI Engineer by Little_Kitty on Saturday July 01, 2023 @05:02AM (#36548200)
LLMs today are great for certain use cases, which is amazing when it suits your needs. Need to extract a hotel name, city, address and confirmation number from emails and return it as JSON? Not a problem. Need it to validate postal codes in Canada? It's the wrong tool.
Re:The Rise of the AI Engineer by mschuster91 on Friday June 30, 2023 @04:35PM (#36541853)
I'd add these use cases:

5. automated processing of unstructured paperwork and ingestion into ERP systems. Basically, upload any kind of bill and get all of the information into the system, not just "find out the total amount". That can save so much in accounting it's not even funny any more.

6. related to this, something that sorts incoming emails. Classify stuff into "look into it now" vs "look into it later" vs "yet another bullshit marketing email".

Re:The Rise of the AI Engineer by oblio on Friday June 30, 2023 @05:40PM (#36542721)
My problem with this is that AI at the moment is a kind of 80% thing.

And you want to plug that into ERPs and accounting?!?

We don't even really understand its failure modes.

Re:The Rise of the AI Engineer by themanmaran on Friday June 30, 2023 @09:33PM (#36545318)
Yea but humans manually retyping things is also a 80% thing.
Re:The Rise of the AI Engineer by mschuster91 on Saturday July 01, 2023 @06:07AM (#36548511)
The thing is it can be used to get rid of the tedious and error-prone busywork. Instead of having highly paid accountants manually type monetary amounts into SAP, the accountants can now just look over and check if the AI was accurate in transcription.
Re:The Rise of the AI Engineer by oblio on Saturday July 01, 2023 @07:51AM (#36549120)
Why AI, though? Why not OCR and regular software? What does AI bring to this table?

Except for VC money.

Re:The Rise of the AI Engineer by mschuster91 on Saturday July 01, 2023 @08:14AM (#36549269)
OCR can't grasp context, it needs incoming bills to be formatted in exactly the same way or at least in somewhat consistent area patterns.

AI doesn't have that problem, so you can use it to detect areas of interest and then follow up on that with classic OCR.