News for hackers. Stuff that matters.
Re:The 'Fuck You' Pattern by nine_k on Monday June 28, 2021 @07:01PM (#27669427)
Go an and run a site that serves massive amounts of cat pictures for peanuts!

Well, yes, serving static HTML is easy. Serving images is kind of not hard either, because you have free unlimited traffic, don't you? Some hosting plans offer it. They just limit the egress bandwidth. So maybe you need a few more servers to cope with it, just like $50 a pop. You're not gonna need any CDN, people further away from your DC will just wait longer. Or run your servers from two regions, it just takes duplicating your collection of cat pictures, should be peanuts.

Then, you need to store these terabytes of cats. On rather fast disks because else the disks will be the bottleneck, and cats will load very slowly for some users. It would be terrible were something to happen to the collection, so you need some backup for peanuts, like some Backblaze, and maybe a second copy because Backblaze has no^W limited redundancy.

Then, well, you need to monitor all that and sometimes fix issues, because cloud infra is highly available but not highly reliable.

Then, where do the cat pictures come from? Allow anonymous uploads! Or register users and allow uploads.

Now analyze each picture so that it's an actual cat picture and not child porn or a warez archive. Resize them to match your standards. Yes, this is simple, just run a RNN that detects cats.

Then, curate the incoming pictures so that they are actually reasonably interesting. Just maybe hire a few people to do that for peanuts somewhere in Africa.

Ah yes, you want to also deduplicate images; run some image hashing algorithms; use maybe just one or two GPU instances to do that efficiently. And you gonna need a DB, a simple one, that will never need any administering or fixing, won't it?

Well, yes, now you can run that site for just several grand a month, making it your day job, because you're well-qualified to maintain every part of it. Enjoy the peanuts.

Re:The 'Fuck You' Pattern by dylanwenzlau on Monday June 28, 2021 @10:52PM (#27671164)
Hi. I run imgflip.com, a fine purveyor of cat pictures, among others. It serves ~100TB of bandwidth and hundreds of millions of image views per month with total infrastructure cost below 5k per month. I believe Imgflip completely avoids dark patterns because I abhor them and strive to provide positive value to humanity, although I would love for you to share your opinions to the contrary. Imgflip is also 80% profit. It employs several of the techniques you mention, but efficiently, rather than bloated and inefficient like many companies. Curious to hear your take
Re:The 'Fuck You' Pattern by novok on Tuesday June 29, 2021 @01:10AM (#27672034)
So I turned off adblock, but I see no ads, even though you have some ad domains. Is imgflip pro the way you make money only? I tried again and all I saw is one banner ad, sometimes. Do you have more ads more frequently, or is it just a relatively minimal amount in general? What is the revenue split between pro & ads? For linus for example, it's about %26 youtube ads, https://www.youtube.com/watch?v=-zt57TWkTF4

How do you deal with moderation? Is it automated, I see you even have comments? Or is it just report buttons and your takedown form? How is the support load?

What is your tech stack? PHP? Python? C#? Java? Golang?

Re:The 'Fuck You' Pattern by DaiPlusPlus on Monday June 28, 2021 @11:06PM (#27671249)
What’s the business-model for image-hosting like thesedays?

Years ago everyone had to use an image-host (Photobucket, lol…) because all those phpBB instances on shared hosting accounts didn’t support direct image uploads or greatly limited attachments and most people didn’t have their own web hosting account to use instead.

With the rise of Web 2.0, it became important for sites to control their content - I remember the days of hotlinked images being swapped-out with obscene or vomit-inducing content as an act of revenge on bandwidth leeches - and cheap storage meant it wasn’t a problem to host hundreds of gigabytes of content by oneself. Can you image if Facebook and Instagram didn’t host the images posted to them?

It’s 2021 now, web-forums are dead - if people want to share photos to a small circle they’ll likely use Dropbox or OneDrive - or just copy+paste between Signal, Telegram and WhatsApp…

Do guide question: who uses your service and where’s the opportunity for profit? Serving images directly means people can’t see any ads, and browsers now block more and more third-party cookies (until all of them by 2023, I understand) so using image-hosting as a way to track users across the web is gone - is your frontpage traffic enough? Savvy users who use image hosts also tend to the ones to adblock too, even on mobile.

Re:The 'Fuck You' Pattern by nine_k on Tuesday June 29, 2021 @12:54AM (#27671932)
Thank you for the reality check!

So, well, my estimate of a few grand a month was not completely off.

Re:The 'Fuck You' Pattern by tpetry on Tuesday June 29, 2021 @01:09AM (#27672027)
It would be really interesting reading an article on what you do different on what each part costs. Effectively how image hosting services work nowadays or if nothing changed in the past 15 years.
Re:The 'Fuck You' Pattern by webmaven on Monday June 28, 2021 @10:40PM (#27671096)
That's an amusing reductio ad absurdum, but most people don't have that much of an objection to ad-supported media in principle, it is a time-honored business model.

But being an asshole about insisting that everyone log in so you can target them individually isn't actually a requirement for the business model to work and be profitable.

At most, you can justify being an asshole about insisting that the user not be using an ad-blocker (with the user having the option of logging in with a paid account for an ad-free experience).

Re:The 'Fuck You' Pattern by jehb on Monday June 28, 2021 @07:31PM (#27669696)
This sounds like a great case for a decentralized web infrastructure like Mastodon uses. Each cat server can be exactly as reliable as the owner feels like making it, and each user can choose a cat server based on their needs, and the whole thing is federated because it's based on open source and open standards.

p2p cats for teh win

Re:The 'Fuck You' Pattern by sitzkrieg on Monday June 28, 2021 @08:45PM (#27670266)
mastodon also downloads remote media from all known accounts with no real way to manage it, making it a nightmare to administer disk usage for any decent sized instance. they go the route of your instance being responsible for all remote media, not the other way around

https://github.com/tootsuite/mastodon/issues/9567

Re:The 'Fuck You' Pattern by aembleton on Tuesday June 29, 2021 @05:02AM (#27673497)
Each server could have an address that you type into a browser. We could have a 'domain name server' that keeps a list of which address should match to which server.

Then people could easily navigate to cat pictures.

Re:The 'Fuck You' Pattern by NieDzejkob on Monday June 28, 2021 @07:36PM (#27669742)
> Backblaze has no redundancy.

That does not seem to be the case.

https://help.backblaze.com/hc/en-us/articles/217667468-B2-Se...

Re:The 'Fuck You' Pattern by nine_k on Monday June 28, 2021 @08:25PM (#27670107)
Indeed. They have redundancy within a DC, like you would expect from a NAS or SAN.

They don't have the cross-DC redundancy like S3 does. If a DC goes offline, so does your data.

I'm updating my comment.

Re:The 'Fuck You' Pattern by DaiPlusPlus on Monday June 28, 2021 @11:10PM (#27671274)
How often do entire DCs suffer catastrophic loss? Not just an hour or two of connectivity outage, but “meteor-strike”-type scenarios - or more likely: a fire that takes out an entire rack or row of racks? How often does it happen in developed countries (where fire fighting infrastructure is in-place and building fire-codes are enforced) vs worldwide average? Has it ever happened to a brand-name cloud provider (AWS, Azure, GCP)? How does it compare to self-managed co-lo?
Re:The 'Fuck You' Pattern by Sebb767 on Monday June 28, 2021 @11:50PM (#27671556)
OVH suffered a fatal DC loss quite recently [0]. It's not an everyday event, but it's not something that never happens, either. Also, don't overestimate our fire fighting capabilities: Once a fire breaks out, severe damage to the DC is very likely - if the fire doesn't kill the server, water, temperature or shock will easily do so. Especially if we're on a scale as small as a rack.

Don't put data you don't want to loose in a single location.

[0] https://www.ovh.com/world/news/press/cpl1787.fire-our-strasb...

Re:The 'Fuck You' Pattern by nine_k on Tuesday June 29, 2021 @01:00AM (#27671973)
A power or connectivity loss may happen. A DC does not need to be destroyed or something, just a backhoe acting up somewhere.
Re:The 'Fuck You' Pattern by wruza on Tuesday June 29, 2021 @12:16AM (#27671708)
You’re overblowing an imageboard checklist into a pack of elephants. I wonder how much I’d earn if I disaggregated every “make us another site quick” in my company in such a way. Well, writing html is easy. But then webpack… and then you want editable content… postgres/orm… and browser coverage, .babelrc… and of course we don’t serve it from our office, now it’s vps management and deployment configuration… and version control, do you know how hard is to remember all git essentials? We have to register a plan on gitlab and enter a card into it!

Smells like seven figures already.

Re:The 'Fuck You' Pattern by e12e on Tuesday June 29, 2021 @07:08AM (#27674225)
> Go an and run a site that serves massive amounts of cat pictures for peanuts!

Well, I was responding to "millions", so maybe "a handful of peanuts" might be more accurate...

I belive there was an open company/OCI post by neocities.org a good while back, detailing their budget - but I can't seem to find it again.

Re:The 'Fuck You' Pattern by e12e on Wednesday June 30, 2021 @05:39AM (#27686638)
Looks like Kyle dropped out of the open company initiative at some point, or let that one blog post drop off the site?

http://web.archive.org/web/20160310015118/https://blog.neoci...

> Our (rough) budget for 2015

(...)

> We have upgraded a lot of our infrastructure to provide for faster, more reliable service with more features, space, and bandwidth capabilities, which has pushed our operating bill to around $300/month (...) To prevent unexpected costs, we have made agreements with our providers to make this a fixed rate, so a sudden flood of traffic shouldn't change this dramatically.

(...)

> In order to become self sustaining with the current system, we will require approximately $3600 in annual recurring revenue. I think this is an achievable goal with our new site (we're almost 1/3 of the way there), and we have plenty of runway incase this takes longer than expected. We also have the ability to scale back on some of our expenses, as I believe I can shave off at least $100-150 from our monthly recurring expenses should we need to without too many problems.

> (...) We can scale a lot (~500k+ sites) with what we have now, and the scaling options we have are simple, predictable and affordable.

Re:The 'Fuck You' Pattern by bombcar on Monday June 28, 2021 @09:59PM (#27670802)
We just need a standardized cat and picture thereof - perhaps even expand the standard to cover one of each breed.

Then you can have a local copy and be sure you have every standard cat in existence!

Re:The 'Fuck You' Pattern by majewsky on Tuesday June 29, 2021 @05:14AM (#27673563)
No, we just need to train a neural network to generate cat pictures from a small set of inputs (like a uint8 for pose, a uint8 for breed, etc.) and then just transmit those parameters instead of the entire picture.
Re:The 'Fuck You' Pattern by gazby on Monday June 28, 2021 @10:35PM (#27671069)
This guy infrastructures.
Re:The 'Fuck You' Pattern by eyelovewe on Monday June 28, 2021 @07:48PM (#27669819)
Sounds like overkill, mate. I’ll go with your comments parent. Maybe 5 peanuts. In all seriousness, I’ll agree on cdn at 20 a month plus 3 @ 15 dollar nodes running out app and db, let’s go multimaster with a local db node on each app server, screw the c go for a and p, and yeah a single haproxy hardware thread, ha, so make that 2 single core nodes 1gb will do fine for 300k connections per second, um you mentioned staff? For what exactly?

Anyway, you can do anything the hard way, but what’s your point? That serving millions of cats per second requires big investment that must be recouped? I see most of this as rent seeking, and the infrastructure is economy of scale. Maybe you use tensor flow and Kafka inside Kubernetes, I dunno…

I agree that one can expect lame websites to act lame, but it’s out of sheer churlishness rather than necessity

Re:The 'Fuck You' Pattern by nine_k on Monday June 28, 2021 @08:29PM (#27670144)
Indeed, serving is the easy part, once it's set up. Going offline when the DC goes offline can be completely acceptable.

It's the picture intake pipeline that eats most of the resources. If you only want to serve the same collection, or a collection you personally increase by 1-2 cats a day, then it's fine.

Re:The 'Fuck You' Pattern by Dylan16807 on Monday June 28, 2021 @11:57PM (#27671595)
> It's the picture intake pipeline that eats most of the resources.

How so? Let's say you want to be somewhat competitive with imgur and handle a million images per day, and also that the peak rate is 4x your average. That comes out to 50 images per second. A thumbnail or few shouldn't really take more than a CPU-second, so you can handle that with 2-6 cheap servers. Let alone using spare cores on storage-optimized servers. And it's next to zero database load.

Re:The 'Fuck You' Pattern by cvwright on Monday June 28, 2021 @10:19PM (#27670960)
You think that Facebook and Google build these huge warehouse-size data centers all over the world for fun?

You think they wouldn’t rather have the cash?

Re:The 'Fuck You' Pattern by eyelovewe on Tuesday June 29, 2021 @07:55PM (#27683317)
You do realize that their data includes half of humanities personal lives, if you count WhatsApp and tinder. It ain’t all cat pictures, I’m afraid