News for hackers. Stuff that matters.
Re:ChatGPT for SNES by 4lx87 on Thursday November 19, 2026 @11:53PM (#50570331)
Fascinating to think we could have implemented transformers in 1990. What other discoveries await us?
Re:ChatGPT for SNES by LeoPanthera on Friday November 20, 2026 @12:01AM (#50570370)
Running them isn't the same as creating them.
Re:ChatGPT for SNES by DoesntMatter22 on Friday November 20, 2026 @12:22AM (#50570486)
Sure but it could still have been done, we just didn't realize it.
Re:ChatGPT for SNES by paxy on Friday November 20, 2026 @01:32AM (#50570912)
By that logic we could have invented it at any point in history with a turing machine made of rocks.
Re:ChatGPT for SNES by staplehorse6553 on Friday November 20, 2026 @01:44AM (#50570992)
And now you're re-reading the Three-body Problem.
Re:ChatGPT for SNES by phire on Friday November 20, 2026 @01:34AM (#50570924)
I'm not sure we would have found this impressive back then, the output is pretty nonsensical. Back then, we had other (more manual) approaches that could produce somewhat equivalent results.

It looks impressive to us because we know how good the output from a full-sized LLM can be, and we are willing to give it a lot of leeway because of its small size.

Well... computational linguistics academics should have been impressed at how well it understands language with nothing more than ingesting a huge amount of untagged data.

Re:ChatGPT for SNES by remembergod on Friday November 20, 2026 @05:10AM (#50572581)
In fact, I know for a fact we wouldn't, because... This existed.

Well, not something made the same way this was. And yet, still precisely this in nature. There was at one point a website with what was basically a chatbot pretending to be God. It had pretty evasive responses, very frequently got off topic, but sometimes, and I am not sure how they technically achieved it, it would give spot on answers to what you told them. Never deep trivia knowledge, but like, casual talk expressions and very simple queries, and other follow ups that made complete sense. Like it would know how to follow up on a "What's up dog?" and bark as a joke when you tried to say "I meant dog literally".

And crucially: We found it utterly unimpressive. It was a stupid toy for kids in computer class.

Re:ChatGPT for SNES by piltdownman on Friday November 20, 2026 @08:37AM (#50574479)
Heuristic Pattern Matching via AIML. Probably a variant of ALICE

https://en.wikipedia.org/wiki/Artificial_Linguistic_Internet...

So not even comparative to be honest.

Re:ChatGPT for SNES by mk89 on Friday November 20, 2026 @01:41AM (#50570971)
Plenty of research papers out there that get no love.

Go through them and find the next big thing :)

Re:ChatGPT for SNES by kqr on Friday November 20, 2026 @02:13AM (#50571191)
As someone else pointed out, training something like this requires much more hardware than running it.

It's similar with other forms of machine learning, too. A logistic regression can make calibrated decisions in complex situations with nearly no compute at all, but training it on lots of data to give it good weights is more expensive. And even if we could submit a training batch job on 1990s hardware and wait a few years for it to complete, there's a level of interactivity required at the start to explore one's way to the right parameters and tokenisation etc. that probably wouldn't have been economically feasible at the time.

In a wider sense, a general purpose computer is a general purpose computer. Hypothetically, we "could have" run transformers in the 1940s on vacuum tubes. It just would have (a) required much space, (b) produced much heat, and (c) required tube replacements all the time. We "could have" run transformers on human computers in the late 1800s or early 1900s – all the maths was there, I think? – but it would have been even slower!

Re:ChatGPT for SNES by anthk on Friday November 20, 2026 @03:16AM (#50571662)
The difference between 1992 hardware and 1998 was really huge. High end Sparc servers on 1998 (or double CPU based Xeons) could create something usable.
Re:ChatGPT for SNES by handoflixue on Friday November 20, 2026 @06:00AM (#50572993)
> We "could have" run transformers on human computers in the late 1800s

I love the idea of using humans to train an auto-complete model that produces human-like text. I think those are called authors ;)

Re:ChatGPT for SNES by solyin on Friday November 20, 2026 @03:37AM (#50571838)
The Ancient Romans could have been the ones to discover electricity and steam engines but they didn't. Room temperature superconductors could have already been found.
Re:ChatGPT for SNES by piltdownman on Friday November 20, 2026 @08:30AM (#50574391)
I mean take a step back and look at in context of the early 90s and it was fairly evident that we could. But it was just as evident that it was a research and sentiment dead-end.

1993 was the AI winter. Not dissimilar to the naive forecast and scope of the famous 1966 Summer Vision Project in MITs fledgling AI department, they had massively underestimated the complexity of the problem domain. Funding had dried up, and the LISP machine industry was circling the drain.

The jump from human mandated logic and decision trees was far in the future. The original Neural Networks (Elman & Jordan Networks -> RNN) were crippled by context limitations due to stepping through the data in a linear fashion and hitting the vanishing gradient brickwall almost immediately.

An agentic AI could trivially can build and run a native Lisp interpreter or bytecode VM on both SNES and Sega Genesis hardware, and a human team could feasibly port an implementation over. The Genesis ran on a 68k with support for 24-bit address space and 32-bit registers, so it was easy to compile 68k C/Assembly to reside within 64kb of RAM.

But it's not 'real' AI in the sense of Transformer or precursor Architecture. Without an FPU you've no avenue to do parallel matrix multiplication in any sort of reasonable cycle time, even if you managed to somehow deploy a 500mb Model into a custom cartridge and optimised the I/O and swap to make it mechanically feasible. So you're talking hours per token.

So the results were incredibly slow, erratic, with a tiny context window it still couldn't retain context from, and all the use-cases for console gaming or add-ons like Sewing Machines were handled and implemented in a much saner and deterministic manner via State Machines.

So in short, we could have done so. Even limited to the knowledge of yesteryear. But why on earth would you have even considered it without the foresight of the contemporary industry outcomes? Even with the hindsight it's simply a novelty... for now anyway.

Re:ChatGPT for SNES by alnwlsn on Friday November 20, 2026 @10:53AM (#50576603)
There was an article the other week - turns out, we didn't need Deep Blue to beat Kasparov, a off the shelf regular desktop PC could have done it at the time if it was running a modern chess engine of today. Though you never would have been able to train the model in the first place.