Re:How 500px serves up 500TB of high res photos
by ranrub on Saturday May 16, 2015 @03:00AM
(#9555280)
Thanks for sharing this and being on HN to answer - very interesting architecture! How do you deal with customers from remote locations (Asia, Oceania) complaining about slow upload speeds?
I will keep posted to your blog. Thanks for sharing the link. Always looking to learn about memory heavy operations. Currently am the lead dev on software that revolves around user generated content (images and audio). Its an ever ending road of (fun) challenges.
Re:How 500px serves up 500TB of high res photos
by renaudg on Saturday May 16, 2015 @12:09AM
(#9555031)
Is there a specific reason why you didn't use an off the shelf open source solution like Thumbor ( http://thumbor.org/ ) ?
Re:How 500px serves up 500TB of high res photos
by Renat on Saturday May 16, 2015 @09:19AM
(#9555991)
performance is the main reason. Additionally we needed a solution to modify images on the fly (watermarking and attribution). That's why we decided to build our own service using go
Re:How 500px serves up 500TB of high res photos
by frik on Saturday May 16, 2015 @02:27AM
(#9555224)
I assume you still generate a small thumbnail for every picture, right?
Re:How 500px serves up 500TB of high res photos
by discardorama on Friday May 15, 2015 @11:57PM
(#9555014)
I am surprised that it is faster to resize the image, than to pass the (resized) bits right through. Is the outbound network a bottleneck?
Re:How 500px serves up 500TB of high res photos
by kalleboo on Saturday May 16, 2015 @01:18AM
(#9555136)
It could be that there are a lot of images/versions that simply never get seen, or seen rarely enough that it's inefficient to store them. And they're being cached by the CDN anyway, so popular images will still only be generated once per edge.
Re:How 500px serves up 500TB of high res photos
by Renat on Saturday May 16, 2015 @09:06AM
(#9555944)
bingo! Extra storage for all the resized images that never get seen, lack of flexibility to introduce new sizes, plus requirements for watermarking and attribution on images are the main reasons why we are moving away from pre-conversion to dynamic image resizing
Today, we rely on the new Resizer service that can resize and crop images on the fly. It's an interesting piece of technology that we will be writing about on our blog: developers.500px.com