News for hackers. Stuff that matters.
Re:Lepton image compression: saving 22% losslessly from imag... by Jabbles on Thursday July 14, 2016 @01:04PM (#12095497)
It's interesting to do a cost analysis here:

It's saved "multiple petabytes" of space.

Backblaze storage is $0.005/GB/Month = $5k/PB/Month.

The GitHub repo has 7 authors, perhaps costing Dropbox $200k/year each and taking most of a year ~ $1M to develop this system.

So this might pay for itself after 200PB*Months, assuming Dropbox's storage costs are the same as Backblaze's prices, and assuming CPU time is free. (TODO: estimate CPU costs...)

Of course, advancing the state of the art has intrinsic advantages, but again, it's interesting to look at the purely financial point.

https://www.backblaze.com/b2/cloud-storage-pricing.html

Re:Lepton image compression: saving 22% losslessly from imag... by drewcrawford on Thursday July 14, 2016 @06:16PM (#12097610)
I think you may be crunching the numbers the wrong way...

Dropbox is valued at $10b (regardless of what you think of that number, someone was willing to buy at that price). But they only have 1800 employees. If by some miracle 80% of them are engineers that is $7M per engineer. All this for a "pure software" business. No inventory, no manufacturing, they only recently started doing their own operations.

Investors would be telling them "more engineers" => "more software" => "better Dropbox". So now you have to go out and snap up engineers in Silicon Valley, which is very very hard. But you don't offer the perk of a Google/Apple line on a resume nor the compensation of e.g. Microsoft. What do you do?

Answer: you offer people the chance to work on cutting edge breakthrough technology. Not only that but it's all open source! This is a dream job for some people. They will turn down every other gig for this one.

It doesn't matter that PB/mo is chump change because an investor doesn't know that, they only know the engineering headcount. If by some miracle an investor does know this is a waste of time, then you just point to this HN post and observe how many developers are interested in this technology and how it is attracting developer mindshare that can be exploited for additional hires down the road.

I'm not saying it's rational–it's not. But I think there were strong incentives to greenlight a project like this, even if the actual cost savings were zero.

Re:Lepton image compression: saving 22% losslessly from imag... by BrunoJo on Thursday July 14, 2016 @01:13PM (#12095583)
Space is cheap but the bandwidth isn't. At Backblaze the download traffic costs $0.05/GB = $50k/PB.
Re:Lepton image compression: saving 22% losslessly from imag... by Jabbles on Thursday July 14, 2016 @01:23PM (#12095671)
I don't think they're compressing this and decompressing it client side. At least I didn't get that impression from the article.

You're correct ofc, download costs = 10 months of storage.

Re:Lepton image compression: saving 22% losslessly from imag... by lj3 on Thursday July 14, 2016 @02:01PM (#12095980)
From the article:

Lepton can decompress significantly faster than line-speed for typical consumer and business connections. Lepton is a fully streamable format, meaning the decompression can be applied to any file as that file is being transferred over the network. Hence, streaming overlaps the computational work of the decompression with the file transfer itself, hiding latency from the user.

In my humble opinion, dropbox's image gallery webapp is considerably faster than any other I've seen, especially when compared to imgur.

Re:Lepton image compression: saving 22% losslessly from imag... by aab0 on Thursday July 14, 2016 @01:48PM (#12095882)
> I don't think they're compressing this and decompressing it client side.

The speed quotes made it sound like client-side was a concern. Why would you go to all the effort of devising a new image compression format saving 20%+ storage and on the wire, and not have it decompressed client-side, especially when you control the client?

Re:Lepton image compression: saving 22% losslessly from imag... by dgoldstein0 on Friday July 15, 2016 @06:42AM (#12100016)
My rough understand, as someone who works at Dropbox and knows some of the people who worked on this (but isn't directly involved), is that this currently only runs on our servers. The perf requirements are primarily that we don't want to slow down syncing / downloads significantly - and also want to keep the CPU cost under control. As is, the savings in storage space should easily pay for the extra compute power required.
Re:Lepton image compression: saving 22% losslessly from imag... by mseebach on Thursday July 14, 2016 @01:51PM (#12095908)
Why wouldn't they (at least, plan to)? They control the client-software.
Re:Lepton image compression: saving 22% losslessly from imag... by kardos on Thursday July 14, 2016 @05:31PM (#12097319)
The typical encoding/decoding rates are measured with a Xeon... my guess is that lower powered phones/netbooks will be considerably slower. I'd trade off a 20% longer sync time in exchange for not running my laptop battery down.
Re:Lepton image compression: saving 22% losslessly from imag... by placeybordeaux on Thursday July 14, 2016 @01:06PM (#12095523)
I only see 3 authors.
Re:Lepton image compression: saving 22% losslessly from imag... by Jabbles on Thursday July 14, 2016 @01:07PM (#12095530)
Re:Lepton image compression: saving 22% losslessly from imag... by vanderZwan on Thursday July 14, 2016 @01:08PM (#12095534)
Doesn't mean they worked on it full-time
Re:Lepton image compression: saving 22% losslessly from imag... by Jabbles on Thursday July 14, 2016 @01:10PM (#12095562)
So are you going to give me a better estimate?
Re:Lepton image compression: saving 22% losslessly from imag... by jamie_ca on Thursday July 14, 2016 @01:23PM (#12095668)