News for hackers. Stuff that matters.
Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by wruza on Friday December 20, 2024 @02:06PM (#42473858)
And look at the airplanes, they really can’t just land on a mountain slope or a tree without heavy maintenance afterwards. Those people weren’t all stupid, they questioned the promise of flying servicemen delivering mail or milk to their window and flying on a personal aircar to their workplace. Just like todays promises about whatever the CEOs telltales are. Imagining bullshit isn’t unique to this century.

Aerospace is still a highly regulated area that requires training and responsibility. If parallels can be drawn here, they don’t look so cool for a regular guy.

Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by Workaccount2 on Friday December 20, 2024 @02:27PM (#42474104)
What people always leave out is that society will bend to the abilities of the new technology. Planes can't land in your backyard so we built airports. We didn't abandon planes.
Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by wruza on Friday December 20, 2024 @02:51PM (#42474346)
Yes but the idea was lost in the process. It became a faster transportation system that uses air as a medium, but that’s it. Personal planes are still either big business or an expensive and dangerous personal toy thing. I don’t think it’s the same for LLMs (would be naive). But where are promises like “we’re gonna change travel economics etc”? All headlines scream is “AGI around the corner”. Yeah, now where’s my damn postman flying? I need my mail.
Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by ben_w on Friday December 20, 2024 @04:58PM (#42475478)
> It became a faster transportation system that uses air as a medium, but that’s it.

On the one hand, yes; on the other, this understates the impact that had.

My uncle moved from the UK to Australia because, I'm told*, he didn't like his mum and travel was so expensive that he assumed they'd never meet again. My first trip abroad… I'm not 100% sure how old I was, but it must have been between age 6 and 10, was my gran (his mum) paying for herself, for both my parents, and for me, to fly to Singapore, then on to various locations in Australia including my uncle, and back via Thailand, on her pension.

That was a gap of around one and a half generations.

* both of them are long-since dead now so I can't ask

Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by PaulDavisThe1st on Friday December 20, 2024 @02:45PM (#42474274)
Sure, but that also vindicates the GP's point that the initial claims of the boosters for planes contained more than their fair share of bullshit and lies.
Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by oblio on Friday December 20, 2024 @03:50PM (#42474892)
We are slowly discovering that many of our wonderful inventions from 60-80-100 years ago have serious side effects.

Plastics, cars, planes, etc.

One could say that a balanced situation, where vested interests are put back in the box (close to impossible since it would mean fighting trillions of dollars), would mean that for example all 3 in the list above are used a lot less than we use them now, for example. And only used where truly appropriate.

Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by ForHackernews on Friday December 20, 2024 @03:29PM (#42474721)
This is already happening. A few days ago Microsoft turned down a documentation PR because the formatting was better for humans but worse for LLMs: https://github.com/MicrosoftDocs/WSL/pull/2021#issuecomment-...

They changed their mind after a public outcry including here on HN.

Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by tivert on Friday December 20, 2024 @04:26PM (#42475207)
> What people always leave out is that society will bend to the abilities of the new technology.

Do they really? I don't think they do.

> Planes can't land in your backyard so we built airports. We didn't abandon planes.

But then what do you do with the all the fantasies and hype about the new technology (like planes that land in your backyard and you fly them to work)?

And it's quite possible and fairly common that the new technology actually ends up being mostly hype, and there's actually no "airports" use case in the wings. I mean, how much did society "bend to the abilities of" NFTs?

And then what if the mature "airports" use case is actually something most people do not want?

Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by moffkalast on Friday December 20, 2024 @05:05PM (#42475532)
No, we built helicopters.
Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by skydhash on Friday December 20, 2024 @02:18PM (#42474005)
This pretty much. Everyone knows that LLMs are great for text generation and processing. What people has been questioning is the end goals as promised by its builders, i.e. is it useful? And from most of what I saw, it's very much a toy.
Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by MVissers on Friday December 20, 2024 @10:10PM (#42477252)
What would you need to see to call it useful?

To give you an example– I've used it for legal work such as an EB2-NIW visa application. Saved me countless of hours. My next visa I'll try to do without a lawyer using just LLMs. I would never try this without having LLMs at my disposal.

As a hobby– And as someone with a scientific background I've been able to build an artificial ecosystem simulation from scratch without programming experience in Rust: https://www.youtube.com/@GenecraftSimulator

I recently moved from fish to plants and believe I've developed some new science at the intersection of CS and Evolutionary Biology that I'm looking to publish.

This tool is extremely useful. For now– You do require a human in the loop for coordination.

My guess is that these will be benchmarks that we see within a few years: How good an AI coordinate multiple other AIs to build, deploy and iterate something that functions in the real world. Basically manager AI.

Because they'll literally be able to solve every single one shot problem so we won't be able to create benchmarks anymore.

But that's also when these models will be able to build functioning companies in a few hours.

Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by skydhash on Friday December 20, 2024 @11:26PM (#42477551)
> ...me countless of...would never try this without having LLMs...is extremely useful...they'll literally be able to solve...will be able to... in a few hours.

That's marketing language, not scientific or even casual language. So much outstanding claims, without even some basic explanations. Like how did it help you save you these hours? Terms explanations? Outlining processes? Going to the post office for you? You don't need to sell me anything, I just want the how.

Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by wruza on Saturday December 21, 2024 @12:26PM (#42480922)
My issue with LLMs is that you require a review-competent human in the loop, to fix confabulations.

Yes, I’m using them from time to time for research. But I’m also aware of the topics I research and see through bs. And best LLMs out there, right now, produce bs in just 3-4 paragraphs, in nicely documented areas.

A recent example is my question on how to run N vpn servers on N ips on the same eth with ip binding (in ip = out ip, instead of using a gw with the lowest metric). I had no idea but I know how networks work and the terminology. It started helping, created a namespace, set up lo, set up two interfaces for inner and outer routing and then made a couple of crucial mistakes that couldn’t be detected or fixed by someone even a little clueless (in routing setup for outgoing traffic). I didn’t even argue and just asked what that does wrt my task, and that started the classic “oh wait, sorry, here’s more bs” loop that never ended.

Eventually I distilled the general idea and found an article that AI very likely learned from, cause it was the same code almost verbatim, but without mistakes.

Does that count as helping? Idk, probably yes. But I know that examples like this show that you cannot not only leave an LLM unsupervised for any non-trivial question, but have to leave a competent role in the loop.

I think the programming community is just blinded by LLMs succeeding in writing kilometers of untalented react/jsx/etc crap that has no complexity or competence in it apart from repeating “do like this” patterns and literally millions of examples, so noise cannot hit through that “protection”. Everything else suffers from LLMs adding inevitable noise into what they learned from a couple of sources. The problem here, as I understand it, is that only specific programmer roles and s{c,p}ammers (ironically) write the same crap again and again millions of times, other info usually exists in only a few important sources and blog posts, and only a few of those are full and have good explanations.

Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by throwaway4aday on Friday December 20, 2024 @03:14PM (#42474573)
Your point is on the verge of nullification with the rapid improvement and adoption of autonomous drones don't you think?
Re:OpenAI O3 breakthrough high score on ARC-AGI-PUB by wruza on Friday December 20, 2024 @06:44PM (#42476265)
Sort of, but doesn’t that sit on a far-fetch horizon? I doubt that drone companies are all the same who sold aircraft retrofuturism to people back then.