News for hackers. Stuff that matters.
Re:Extreme Video Compression by Animats on Monday February 19, 2024 @05:39PM (#39435685)
Extreme compression will be when you put in a movie and get a SORA prompt back that regenerates something close enough to the movie.
Re:Extreme Video Compression by aqme28 on Monday February 19, 2024 @07:33PM (#39436609)
Where’s that quote? Something like “AI is just compression, and compression is indistinguishable from AI”
Re:Extreme Video Compression by daveguy on Monday February 19, 2024 @08:06PM (#39436839)
I'm not sure the quote, but you're probably thinking something related to the Hutter Prize:

https://en.m.wikipedia.org/wiki/Hutter_Prize

A lossless compression contest to encourage research in AI. It's lossless, I think just to standardize scoring, but I always thought a lossy version would be better for AI -- our memories are definitely lossy!

Re:Extreme Video Compression by Vecr on Monday February 19, 2024 @10:21PM (#39437691)
> A lossless compression contest to encourage research in AI. It's lossless, I think just to standardize scoring, but I always thought a lossy version would be better for AI -- our memories are definitely lossy!

Gwern posts about this when people say something like that on here, but I'll do it instead. Lossless encoding is just lossy encoding + error correction of some sort.

Re:Extreme Video Compression by daveguy on Monday February 19, 2024 @10:36PM (#39437788)
Hah. "error correction of some sort" is doing a lot of heavy lifting there. Bit level correction is what I generally consider under the "error correction" umbrella. Which we have no problem with -- curious the bit error rate that would make text unreadable. In the context of compressing a large part of the English language wiki -- I think lossy also includes loss so significant that you wouldn't be able to reproduce the exact text. So, well beyond what we would generally consider "error correcting". But intuitively understandable as equivalent by humans. Impossible to quantify that objectively, hence lossless only for the competition.
Re:Extreme Video Compression by GuB-42 on Thursday February 22, 2024 @08:37AM (#39466840)
The way we learn exact sentences is usually by getting an intuitive sense and applying corrections.

For example, to memorize "Doggo woofs at kity", we first get the concept of "dog barks at cat", it compresses well because intuitively, we know that dogs bark and cats are common targets. That's our lossy compression and we could stop there but it is only part of the story. It is not a "dog" but a "doggo", and it goes well with the familiar tone, a good compression algorithm will take only a few bits for that. Then there is the typo "kity" vs "kitty", it will take a bit of extra space, but again, a good algorithm will recognize the common typos and compress even that. So it means the entire process to lossless matters, lossy is just stopping halfway.

And if there is pure random noise remaining, there is nothing you can do, but all algorithms are on an equal footing here. But the key is to make what the algorithm consider as uncompressible noise as small as possible.

Re:Extreme Video Compression by pyinstallwoes on Monday February 19, 2024 @08:31PM (#39437001)
Intelligence is compressing information into irreducible representation.
Re:Extreme Video Compression by sandkoan on Tuesday February 20, 2024 @12:53AM (#39438429)
Re:Extreme Video Compression by visarga on Tuesday February 20, 2024 @02:04AM (#39438713)
> AI is just compression, and compression is indistinguishable from AI

Almost. Compression and AI both revolve around information processing, but their core objectives diverge. Compression is focused on efficient representation, while AI is built for flexibility and the ability to navigate the unpredictable aspects of real-world data.

Compression learns a representation from the same data it encodes, like "testing on the training set". AI models have different training and test data. There are no surprises in compression.

Re:Extreme Video Compression by larodi on Tuesday February 20, 2024 @03:47AM (#39439211)
lets say AI is not-so-smart JPEG which has more parts missing, and there is more guesswork when producing restoration.

compression is most of the times about finding the minimal grammar that unfolds to the same original material.

interestingly Fabrice Bellard somehow found a way to use transformers for compression without loss, and beats xz by significant margin. https://bellard.org/nncp/nncp_v2.1.pdf. it uses "deterministic mode of PyTorch" to make sure both directions work alike which I guess means - it saves the random toss throughout compression, for the decompression to use. note: this paper is still on my to-read list.

Re:Extreme Video Compression by kimixa on Tuesday February 20, 2024 @01:07PM (#39444586)
A lot of current compression techniques use prediction followed by some set of correction data to fix mis-predictions. If the prediction is more accurate, you can have a smaller correction set.

But you're right the predictor does need to be reproducible - the output must be exactly the same to match encoder and decoder behavior. While I don't think this is big focus right now for many, I don't think there's a fundamental reason why it couldn't be, though probably at the cost of some performance.

Re:Extreme Video Compression by jazzyjackson on Tuesday February 20, 2024 @01:26PM (#39444824)
representations' efficiency increases the more flexibility you give it
Re:Extreme Video Compression by Rygian on Tuesday February 20, 2024 @04:58AM (#39439509)
How does that make sense? Compression is deterministic (for same prompt, same output is algorithmically guaranteed). AI is only deterministic in corner cases.
Re:Extreme Video Compression by leoff on Tuesday February 20, 2024 @05:05AM (#39439551)
AI is always deterministic. We add noise to the models to get "non-deterministic" results, but if the noise and input is the same, the output is also the same.
Re:Extreme Video Compression by david-gpu on Tuesday February 20, 2024 @09:42AM (#39441872)
It's a bit more nuanced than that. Floating point arithmetic is not associative: "(A+B)+C" is not always equal to "A+(B+C)". Because of that, certain mathematical operations used in neural networks, such as parallel reductions, will yield slightly different results if you run them multiple times with the same arguments.

There are some people working hard to provide the means to perform deterministic AI computations like these, but that will come with some performance losses, so I would guess that most AIs will continue to be (slightly) non-deterministic.

Re:Extreme Video Compression by nuancebydefault on Tuesday February 20, 2024 @03:53PM (#39446757)
Is AI in general relying on floating point calculations?
Re:Extreme Video Compression by david-gpu on Wednesday February 21, 2024 @03:30AM (#39451394)
One hundred percent. It's mostly fancy floating point matrix multiplications.
Re:Extreme Video Compression by Rygian on Wednesday February 21, 2024 @07:11AM (#39452854)
That assumes an AI that is trained exactly once ever.
Re:Extreme Video Compression by 5kg on Tuesday February 20, 2024 @12:17PM (#39443908)
Language Modeling Is Compression: https://arxiv.org/abs/2309.10668
Re:Extreme Video Compression by squokko on Monday February 19, 2024 @05:44PM (#39435743)
I can imagine that in under 5 years, the movie's script plus one example still photo for each scene could do the job.
Re:Extreme Video Compression by ben_w on Monday February 19, 2024 @05:59PM (#39435851)
Ah, so you have long timelines then? :P
Re:Extreme Video Compression by bsenftner on Monday February 19, 2024 @05:58PM (#39435843)
If we get anywhere close to that, coming up with a new economics model is going to be the prompt we'll be giving the AGI when it's ready. We'll need it.
Re:Extreme Video Compression by IgorPartola on Monday February 19, 2024 @06:19PM (#39436028)
“Alexa show me Star Wars but with Dustin Hoffman as Luke”.
Re:Extreme Video Compression by gedy on Monday February 19, 2024 @08:35PM (#39437026)
I actually would really like this flexibility. "Star Wars, but in Korean with k-pop stars cast", etc.
Re:Extreme Video Compression by solardev on Monday February 19, 2024 @09:12PM (#39437257)
Or as a VR game. "Star Wars, but with the Empire and Rebel Alliance teaming up to defeat the latest threat to the galaxy: me as Jar-Jar Binks, Jedi Jester. My abilities include Force Juggle, Failed Comedic Relief, and Turn Into Merchandise. Oh, and Darth Vader is Morgan Freeman, and everyone else is Natalie Portman."
Re:Extreme Video Compression by yreg on Tuesday February 20, 2024 @04:52AM (#39439475)
I've invited friends over to watch Harry Potter and the Deathly Weapons (a.k.a. Harry Potter with Guns) in its entirety and it was entertaining.

https://harrypotterwithguns.com/

trailer: https://www.youtube.com/watch?v=xA-ayM5I4Jw

Re:Extreme Video Compression by corobo on Tuesday February 20, 2024 @05:04AM (#39439541)
The most fun thinking is combining this with that Apple Vision and further steps in that tech tree

Gonna be able to skip holodeck episodes in real life soon

Re:Extreme Video Compression by EGreg on Monday February 19, 2024 @06:21PM (#39436050)
“I’m sorry Dave. I can’t do that. As an Amazon Large Langauge model, I need you to up your subscription to Amazon Prime first.”

“On the other hand, I can generate endless amounts of Harlan Coben miniseries… :-P”

Re:Extreme Video Compression by melagonster on Monday February 19, 2024 @08:29PM (#39436982)
the old style selective copyright infringement.
Re:Extreme Video Compression by gedy on Monday February 19, 2024 @08:33PM (#39437013)
"To watch Dustin Hoffman, you need to subscribe to the Classic Stars pack"
Re:Extreme Video Compression by polemic on Monday February 19, 2024 @08:50PM (#39437122)
How big is the SORA model itself?
Re:Extreme Video Compression by orblivion on Monday February 19, 2024 @11:17PM (#39438013)
I can show you an algorithm that compresses an entire 2 hour movie to a single bit, but it only works on one movie.
Re:Extreme Video Compression by Leo_Germond on Tuesday February 20, 2024 @12:20AM (#39438302)
Ah I got it to work for two movies :)
Re:Extreme Video Compression by verticalscaler on Tuesday February 20, 2024 @01:15AM (#39438517)
You just made disco stu very happy, if these trends continue- 'Ey!
Re:Extreme Video Compression by nuancebydefault on Tuesday February 20, 2024 @03:55PM (#39446781)
I think the most impressive movie compression technique is youtube. It compresses movies into a short sequence of characters denoted as URLs.
Re:Extreme Video Compression by jasonjmcghee on Monday February 19, 2024 @11:08PM (#39437960)
You only need one copy of it - even if it is 100GB. If it's baked into every OS... and storage / RAM keeps getting cheaper... just might work.
Re:Extreme Video Compression by polemic on Tuesday February 20, 2024 @03:31AM (#39439138)
Meh, AI doesn't break information theory. The relationship between the prompt size to the "similarity" of the result will be such that it doesn't beat traditional compression techniques.

At best we might consider it a new type of lossy (or... replacey?) compression. Of course if storage / RAM / bandwidth keeps increasing, this is quite likely the least energy efficient technique available.

Re:Extreme Video Compression by snovv_crash on Tuesday February 20, 2024 @04:36AM (#39439404)
If the compression can take into account the sub-manifold of potential outputs that people would actually be interested in watching a movie about it can achieve enormously higher compression than if it doesn't know about this.
Re:Extreme Video Compression by polemic on Tuesday February 20, 2024 @03:27PM (#39446458)
Unproven - but yes like I say - a new type of compression
Re:Extreme Video Compression by snovv_crash on Thursday February 22, 2024 @06:55AM (#39465842)
Not new. If you have a preshared dictionary even zstd can give you better compression.
Re:Extreme Video Compression by est on Monday February 19, 2024 @10:52PM (#39437877)
it depends which resolution you want to see.
Re:Extreme Video Compression by gcanyon on Tuesday February 20, 2024 @12:59AM (#39438454)
Ha, I've commented almost exactly this twice now on HN. We'll see how long before it's a reality -- probably better measured in months rather than years.
Re:Extreme Video Compression by _the_inflator on Tuesday February 20, 2024 @01:05AM (#39438480)
In what sense? How about reproducibility? Is stored memory, the connection between the prompt and the exact output, really compression or simply a retrieval of a compressed file then stored as factual knowledge ingrained in its Neural Network?

I like your sentiment, it is technically inspiring.

Re:Extreme Video Compression by cfn on Tuesday February 20, 2024 @01:20AM (#39438534)
Given the same prompt and the same seed (and algorithm) the resulting movie/output will always be the same. This is the case for AI image generation now.
Re:Extreme Video Compression by kzrdude on Tuesday February 20, 2024 @02:44AM (#39438904)
The compression competitions include the decompression program size in the size of the output. Must be a large series of movies compressed to win, then.
Re:Extreme Video Compression by 4gotunameagain on Tuesday February 20, 2024 @02:49AM (#39438919)
If one model can "compress"/"decompress" all movies and series, its fraction of the size becomes negligible, but yes I agree with you since it still has to be distributed.
Re:Extreme Video Compression by jebarker on Tuesday February 20, 2024 @09:22AM (#39441663)
This isn't really any different to what they've done is it?