News for hackers. Stuff that matters.
Re:Apple's petition against RED by est31 on Thursday August 22, 2019 @01:26AM (#20764907)
> RED wanted to find a way to make digital compression visually lossless (i.e. no perceivable loss in quality).

What does that mean? Is it lossless like lossless webp or png, or is it lossy and well performing under some metric like PSNR? "no perceivable loss in quality" can mean anything, including a lossy codec.

Re:Apple's petition against RED by holy_city on Thursday August 22, 2019 @01:42AM (#20765004)
Having worked a bit in the field (on the audio side of film) "perceptually lossless" means that in a double blind study, an audience would not be able to discern between the lossless and the lossy encoded versions. The only way to verify is by testing with real people, and due to the nature of the business, it's easier to do it with professionals who have a higher bar for perception.

There are a lot of reasons to prefer lossless over lossy, but there is always the "good enough" point with storage media. Film is not lossless, so it doesn't matter if the digital storage is. What matters is if the lossiness in encoding is good enough to work with at the same level as film.

Re:Apple's petition against RED by hvidgaard on Thursday August 22, 2019 @02:59AM (#20765372)
The industry also prefers to use people with "golden ears", so they can be pretty confident that the target audience cannot tell the difference.
Re:Apple's petition against RED by holy_city on Thursday August 22, 2019 @03:26AM (#20765511)
There's a weird bar for confidence, to be frank. Most often I've found that people disregard studies with people who have "golden ears" because they're not average ears, or that studies find that only those with "golden ears" can find a difference, and the audience tends to feel they're in that range.

For example, there was a study a few years ago at McGill with trained listeners on the effects of bitrate with mp3 and m4a audio encoding that found that only a slim number of mastering engineers preferred lossless over lossy encoded audio (interestingly enough, some professional mixers and musicians found lossy encodings preferable to lossless even for jazz and classical music). But trying to convince audiophiles that those codecs are comparable to lossless is a losing battle.

Re:Apple's petition against RED by zaarn on Thursday August 22, 2019 @04:04AM (#20765737)
As someone with minor audiophilia, there is still some difference; when you have a lossless version, you can reencode to any lossy version without loosing quality over time.

I can transcode FLAC to Mp3 fast enough to stream it on my phone without issue, if I decide to use OGG or Opus or anything else, I don't have eventual reencoding issues.

Re:Apple's petition against RED by hvidgaard on Thursday August 22, 2019 @04:56AM (#20766015)
For archival purposes, it makes no sense to store anything less than lossless.
Re:Apple's petition against RED by jobigoud on Thursday August 22, 2019 @04:08AM (#20765758)
But what happens when you make multiple roundtrips with this perceptually lossless codec? Is it still passing the test or does it degrade over time?
Re:Apple's petition against RED by Tuna-Fish on Thursday August 22, 2019 @09:47AM (#20767995)
Quality degrades, just as it would if you did the same with film.
Re:Apple's petition against RED by jobigoud on Thursday August 22, 2019 @03:49PM (#20771724)
It's not a given though I imagine. For audio for example you could discard inaudible parts of the spectrum and compress the rest losslessly. In this case only the first round is lossy and it doesn't degrade over time. But I'm not sure the same exists for images.
Re:Apple's petition against RED by mattdw on Friday August 23, 2019 @01:00AM (#20774636)
In the VFX industry you don't roundtrip lossy codecs; first step is to decode to something lossless like EXR or TIFF, and those are what you pipe through your workflows. (With maybe lowres JPEG "proxies" to make iteration a bit quicker.)

Film/TV more generally I think tolerates slightly lossy codecs like RED or Pro Res, as there's often only 1-2 intermediate steps that could cause extra loss. (E.g. the part of editing that is just 'cutting' is pass-through, but color-grading would require a second encoding step.)

Re:Apple's petition against RED by KaiserPro on Thursday August 22, 2019 @04:46AM (#20765960)
Cinema cameras need to be able to record at 4k (real 4k, 4048x2028) at upto 32bit per channel. 24 times a second.

Thats a lot of data. Now, RED at the time didn't have a way of recording to large disk arrays, (unlike the alexa) so they used their own SSD pack things.

This limited the amount of space, and shooting time.

So they needed a way of doing more than RLE compression.

This meant that they had to start chucking away some data. With standard JPEG, you throw away 3/4 of the blue and green channels (kinda, its a different colour space) and then compress the rest.

The problem? in VFX the blue and green channels are crucial for "pulling a key" (green/blue screen work, the less clear, the more manuakl cleanup needed, which costs $$$). So all that 4k resolution will be useless because in practice, the bit that the VFX team need will be < HD res.

So RED used JPEG2000 that uses wavelets to compress things. Roughly speaking, instead of storing a per pixel value, you group chunks of the image together and store the _change in frequency_, that is the difference in colour between pixels.

This doesn't reduce the resolution so much and doesn't produce square artefacts like oldschool JPEG. The problem is that its quite CPU intensive. To the point that it would take >30 seconds to decode a frame.

GPUs make it trivial to do it in real time now, but back then, its was a massive faff.

Also, RED are masters of bullshit and marketing. There is quality loss, its just they never tell you that.

Re:Apple's petition against RED by colejohnson66 on Thursday August 22, 2019 @09:00AM (#20767523)
> so they used their own SSD pack things.

Minor nitpick: they’re miniPCIe SATA (not NVMe) SSDs in a fancy housing with some fancy pin remapping for the outer connector

Re:Apple's petition against RED by KaiserPro on Thursday August 22, 2019 @09:27AM (#20767757)
https://www.cinema5d.com/whats-inside-a-red-mini-mag-the-con...

This is new to me, I didn't think it was that simple. They harped on in the old days about how revolutionary the minimags were.

Re:Apple's petition against RED by CptFribble on Thursday August 22, 2019 @03:38PM (#20771624)
What camera system can record 32-bit? As far as I knew industry standard currently tops out at 16?
Re:Apple's petition against RED by KaiserPro on Friday August 23, 2019 @04:10AM (#20775433)
Sorry, its recorded as 32bit float. the actual sensor is nowhere near capable of 32 bit though.
Re:Apple's petition against RED by Joeboy on Thursday August 22, 2019 @03:06AM (#20765406)
It means lossy. I think the actual situation, which the article kind of misstates, is that footage from cinema cameras like RED's is likely to undergo significant grading and postprocessing, which is where compression artifacts are likely to become a problem. Annoyingly, "Raw" as applied to video codecs seems to be a marketing term that means "you can grade it and still probably not notice compression artifacts."
Re:Apple's petition against RED by rzzzt on Thursday August 22, 2019 @03:25AM (#20765501)
Is this one also a marketing term, then? Would calling a video codec "lossy" have any bad consequences for the product?
Re:Apple's petition against RED by Joeboy on Thursday August 22, 2019 @03:43AM (#20765604)
Virtually all video capture is lossy, even on multi-million movies, because lossless capture is still fairly impractical in terms of storage, and there's generally not much advantage over a good lossy codec. But I don't think film people generally know that, and I suspect some of them would get upset if you tried to make them use a "lossy" camera.
Re:Apple's petition against RED by kelsolaar on Thursday August 22, 2019 @04:42AM (#20765942)
You should define what lossy means for you because your statement is incorrect for me. Shooting Raw with an ARRI Alexa is lossless, this is not the case when you shoot R3D (Raw for RED). R3D is using JPEG2000 compression and will effectively destroy the data the sensor captured to some degree. This can have dramatic consequences when doing green screen keying and/or shooting in low light.
Re:Apple's petition against RED by Joeboy on Thursday August 22, 2019 @05:22AM (#20766153)
Is the part you're disagreeing with "there's generally not much advantage over a good lossy codec"? There are definitely times it's not true, but I think I covered myself with "generally". Most footage is shot with adequate exposure and doesn't require keying etc.
Re:Apple's petition against RED by kelsolaar on Thursday August 22, 2019 @06:36AM (#20766538)
I'm disagreeing with the "Virtually all video capture is lossy" which is simply not true. ARRI cameras and especially the Alexa are some of the most used cameras in the movie industry and the Alexa Raw format is lossless.
Re:Apple's petition against RED by Joeboy on Thursday August 22, 2019 @06:47AM (#20766599)
Those cameras also record ProRes though, which I would expect to be used far more commonly. At least outside of $100m+ movies where money is no object.

Edit: I had a look at https://www.indiewire.com/2018/05/cannes-2018-camera-cinemat... and annoyingly I have to admit that ARRIRAW looks to be much more popular I expected among smaller features. Like 10/32 specifically mention it, and some of the others are ambiguous.

Re:Apple's petition against RED by kelsolaar on Thursday August 22, 2019 @07:10AM (#20766735)
Sure they can but Directors and DOP will favour a Raw file format, e.g. ARRIRAW or R3D over ProRes. Disclaimer: I work in the industry and I'm in touch with most of the camera vendors either directly or indirectly and with a lot of people using them too. I'm also a user.
Re:Apple's petition against RED by Joeboy on Thursday August 22, 2019 @07:19AM (#20766809)
Yeah, it seems like I significantly underestimated the amount of ARRIRAW capture going on (see my edit above).
Re:Apple's petition against RED by msbarnett on Thursday August 22, 2019 @02:06AM (#20765113)
REDCODE and the competing ProRes RAW are lossy codecs the work off the raw Bayer mosaic data. The article does a poor job of explaining this.
Re:Apple's petition against RED by kelsolaar on Thursday August 22, 2019 @04:31AM (#20765889)
It is a lossy compression codec derived/based on JPEG2000: https://peter.schlaile.de/redcode/
Re:Apple's petition against RED by ent on Thursday August 22, 2019 @01:36AM (#20764974)
I'm pretty sure it does mean lossy codec. The

> i.e. no /percievable/ loss in quality.

would be pretty redundant otherwise.

Re:Apple's petition against RED by okcando on Thursday August 22, 2019 @04:01AM (#20765724)
Lossless unqualified would mean lossless: recoverable exactly bit-for-bit, like PNG, FLAC, or ZIP.

The only reason to describe a system as visually or perceptually lossless is because the encoding is lossy, even if very good, like an MP3 or JPEG at their highest quality settings.

Re:Apple's petition against RED by megaremote on Thursday August 22, 2019 @02:57AM (#20765366)
It can only mean lossy. It makes no sense to talk about visually lossless if it is exactly the same image.
Re:Apple's petition against RED by pas on Thursday August 22, 2019 @03:58AM (#20765703)
PNG is truly lossless yet smaller than raw BMP.
Re:Apple's petition against RED by manicdee on Thursday August 22, 2019 @01:52PM (#20770693)
PNG is lossless. There is no adjective or qualifier needed.
Re:Apple's petition against RED by mung on Thursday August 22, 2019 @05:53PM (#20772740)
I'm finding some of the disagreements under this comment frustrating. Movie people, please don't redefine what lossless actually means. If you find a way to encode say a cryptographic key, or a text document, say, your employment contact, through the codec and then retrieve it back and it comes back garbled or useless, it's lossy! It doesn't matter if you can't see a difference with visual data, it has changed the data. That is lossy. That is what lossy means. That is what it has always meant and that is what it always will mean. And if you try to change the meaning for a narrow context driven by marketing, well, words cease to have a meaning, don't they.
Re:Apple's petition against RED by onion2k on Thursday August 22, 2019 @02:10AM (#20765125)
In audio compression most codecs remove data outside of the typical human hearing range to improve the compression ratio - people can't hear that part anyway so there's no reason to save it. I guess RED wanted to do the same thing for visual data.
Re:Apple's petition against RED by est31 on Thursday August 22, 2019 @03:13AM (#20765437)
Yes and back when it was introduced with MP3, many people were really mad about it because it made quality much worse. Nowadays encoders are better and with high enough bitrates you can't tell a difference to lossless.
Re:Apple's petition against RED by ijiiijji1 on Thursday August 22, 2019 @02:14AM (#20765143)
The resolutions of 4K/5K/8K with completely lossless at high frame rates or at native sensor resolution in terms of storage for hundreds/thousands of hours of source material for marginal audiophile-like puritanism/placebo magical beliefs is just way too expensive for essentially no marginal return, because there are fundamental limits to psychovisual and psychoacoustic thresholds that no human can discern in a given playback venue (different for large IMAX, normal theater, home theater and small phablet; and different to a degree depending on an individual's would-be double-blinded or bio-electrically/fMRI measured thresholds.) The goal of pseudo-"lossless" compression is to be very conservative on what to throw out because transcoding to other formats can always (and must) optimize (throw away) data later... it really is marketing wank because there is some compression, but the idea is it's aimed to be minimal as to not matter in the final cut product.
Re:Apple's petition against RED by KaiserPro on Thursday August 22, 2019 @04:48AM (#20765973)
Yes, for customers, I agree.

For VFX, no.

That extra resolution, and crucially colour dynamic range means that less time and money had to be spent cleaning up green screen masks.

It also means that there is more scope to do things like Day-for-night, change the colour temperature, and have full artistic freedom to change the look and feel of a movie without reshooting.

Re:Apple's petition against RED by hvidgaard on Thursday August 22, 2019 @02:57AM (#20765369)
It's a bit more nuanced than that. Resolution is meaningless unless you know the size of the image and the distance to the viewer - so keeping a "lossless" copy makes sense for people professionally buying very expensive hardware. It's no different than keeping the original reels of a movie so you can scan them at higher and higher resolution as technology improves. In other word, the final product will always at best be limited by the format the source is kept in. That is not to say that even amateur videophiles have a use for it, they just want the best and mimic the professionals.

As for frame rate I'm pretty sure we know from experiments that the average human eye can see the difference in frame rate all the way up to 150-160 fps, and that the trained eye can detect images shown for less than 1/200 second.

So however marginal the gain is, it makes sense for professionals that take pride in having the absolutely best quality. It's the same people that will finetune encoder settings for the individual scenes in a movie just to get that tiny improvement.