News for hackers. Stuff that matters.
Re:The 'Fuck You' Pattern by e12e on Monday June 28, 2021 @05:08PM (#27668145)
> You thought it was fair that a company spends millions in technical infrastructure and staffing so you can sit at home and spend your time looking at cats for free? No, they have your attention and they're going to connect you to organizations who will pay for it.

"The infrastructure" to share cat pictures cost peanuts. It's the addictive dark patterns and montezation/tracking that costs millions. That's the irony.

Re:The 'Fuck You' Pattern by nine_k on Monday June 28, 2021 @07:01PM (#27669427)
Go an and run a site that serves massive amounts of cat pictures for peanuts!

Well, yes, serving static HTML is easy. Serving images is kind of not hard either, because you have free unlimited traffic, don't you? Some hosting plans offer it. They just limit the egress bandwidth. So maybe you need a few more servers to cope with it, just like $50 a pop. You're not gonna need any CDN, people further away from your DC will just wait longer. Or run your servers from two regions, it just takes duplicating your collection of cat pictures, should be peanuts.

Then, you need to store these terabytes of cats. On rather fast disks because else the disks will be the bottleneck, and cats will load very slowly for some users. It would be terrible were something to happen to the collection, so you need some backup for peanuts, like some Backblaze, and maybe a second copy because Backblaze has no^W limited redundancy.

Then, well, you need to monitor all that and sometimes fix issues, because cloud infra is highly available but not highly reliable.

Then, where do the cat pictures come from? Allow anonymous uploads! Or register users and allow uploads.

Now analyze each picture so that it's an actual cat picture and not child porn or a warez archive. Resize them to match your standards. Yes, this is simple, just run a RNN that detects cats.

Then, curate the incoming pictures so that they are actually reasonably interesting. Just maybe hire a few people to do that for peanuts somewhere in Africa.

Ah yes, you want to also deduplicate images; run some image hashing algorithms; use maybe just one or two GPU instances to do that efficiently. And you gonna need a DB, a simple one, that will never need any administering or fixing, won't it?

Well, yes, now you can run that site for just several grand a month, making it your day job, because you're well-qualified to maintain every part of it. Enjoy the peanuts.

Re:The 'Fuck You' Pattern by dylanwenzlau on Monday June 28, 2021 @10:52PM (#27671164)
Hi. I run imgflip.com, a fine purveyor of cat pictures, among others. It serves ~100TB of bandwidth and hundreds of millions of image views per month with total infrastructure cost below 5k per month. I believe Imgflip completely avoids dark patterns because I abhor them and strive to provide positive value to humanity, although I would love for you to share your opinions to the contrary. Imgflip is also 80% profit. It employs several of the techniques you mention, but efficiently, rather than bloated and inefficient like many companies. Curious to hear your take
Re:The 'Fuck You' Pattern by novok on Tuesday June 29, 2021 @01:10AM (#27672034)
So I turned off adblock, but I see no ads, even though you have some ad domains. Is imgflip pro the way you make money only? I tried again and all I saw is one banner ad, sometimes. Do you have more ads more frequently, or is it just a relatively minimal amount in general? What is the revenue split between pro & ads? For linus for example, it's about %26 youtube ads, https://www.youtube.com/watch?v=-zt57TWkTF4

How do you deal with moderation? Is it automated, I see you even have comments? Or is it just report buttons and your takedown form? How is the support load?

What is your tech stack? PHP? Python? C#? Java? Golang?

Re:The 'Fuck You' Pattern by DaiPlusPlus on Monday June 28, 2021 @11:06PM (#27671249)
What’s the business-model for image-hosting like thesedays?

Years ago everyone had to use an image-host (Photobucket, lol…) because all those phpBB instances on shared hosting accounts didn’t support direct image uploads or greatly limited attachments and most people didn’t have their own web hosting account to use instead.

With the rise of Web 2.0, it became important for sites to control their content - I remember the days of hotlinked images being swapped-out with obscene or vomit-inducing content as an act of revenge on bandwidth leeches - and cheap storage meant it wasn’t a problem to host hundreds of gigabytes of content by oneself. Can you image if Facebook and Instagram didn’t host the images posted to them?

It’s 2021 now, web-forums are dead - if people want to share photos to a small circle they’ll likely use Dropbox or OneDrive - or just copy+paste between Signal, Telegram and WhatsApp…

Do guide question: who uses your service and where’s the opportunity for profit? Serving images directly means people can’t see any ads, and browsers now block more and more third-party cookies (until all of them by 2023, I understand) so using image-hosting as a way to track users across the web is gone - is your frontpage traffic enough? Savvy users who use image hosts also tend to the ones to adblock too, even on mobile.

Re:The 'Fuck You' Pattern by nine_k on Tuesday June 29, 2021 @12:54AM (#27671932)
Thank you for the reality check!

So, well, my estimate of a few grand a month was not completely off.

Re:The 'Fuck You' Pattern by tpetry on Tuesday June 29, 2021 @01:09AM (#27672027)
It would be really interesting reading an article on what you do different on what each part costs. Effectively how image hosting services work nowadays or if nothing changed in the past 15 years.
Re:The 'Fuck You' Pattern by webmaven on Monday June 28, 2021 @10:40PM (#27671096)
That's an amusing reductio ad absurdum, but most people don't have that much of an objection to ad-supported media in principle, it is a time-honored business model.

But being an asshole about insisting that everyone log in so you can target them individually isn't actually a requirement for the business model to work and be profitable.

At most, you can justify being an asshole about insisting that the user not be using an ad-blocker (with the user having the option of logging in with a paid account for an ad-free experience).

Re:The 'Fuck You' Pattern by jehb on Monday June 28, 2021 @07:31PM (#27669696)
This sounds like a great case for a decentralized web infrastructure like Mastodon uses. Each cat server can be exactly as reliable as the owner feels like making it, and each user can choose a cat server based on their needs, and the whole thing is federated because it's based on open source and open standards.

p2p cats for teh win

Re:The 'Fuck You' Pattern by sitzkrieg on Monday June 28, 2021 @08:45PM (#27670266)
mastodon also downloads remote media from all known accounts with no real way to manage it, making it a nightmare to administer disk usage for any decent sized instance. they go the route of your instance being responsible for all remote media, not the other way around

https://github.com/tootsuite/mastodon/issues/9567

Re:The 'Fuck You' Pattern by aembleton on Tuesday June 29, 2021 @05:02AM (#27673497)
Each server could have an address that you type into a browser. We could have a 'domain name server' that keeps a list of which address should match to which server.

Then people could easily navigate to cat pictures.

Re:The 'Fuck You' Pattern by NieDzejkob on Monday June 28, 2021 @07:36PM (#27669742)
> Backblaze has no redundancy.

That does not seem to be the case.

https://help.backblaze.com/hc/en-us/articles/217667468-B2-Se...

Re:The 'Fuck You' Pattern by nine_k on Monday June 28, 2021 @08:25PM (#27670107)
Indeed. They have redundancy within a DC, like you would expect from a NAS or SAN.

They don't have the cross-DC redundancy like S3 does. If a DC goes offline, so does your data.

I'm updating my comment.

Re:The 'Fuck You' Pattern by DaiPlusPlus on Monday June 28, 2021 @11:10PM (#27671274)
How often do entire DCs suffer catastrophic loss? Not just an hour or two of connectivity outage, but “meteor-strike”-type scenarios - or more likely: a fire that takes out an entire rack or row of racks? How often does it happen in developed countries (where fire fighting infrastructure is in-place and building fire-codes are enforced) vs worldwide average? Has it ever happened to a brand-name cloud provider (AWS, Azure, GCP)? How does it compare to self-managed co-lo?
Re:The 'Fuck You' Pattern by Sebb767 on Monday June 28, 2021 @11:50PM (#27671556)
OVH suffered a fatal DC loss quite recently [0]. It's not an everyday event, but it's not something that never happens, either. Also, don't overestimate our fire fighting capabilities: Once a fire breaks out, severe damage to the DC is very likely - if the fire doesn't kill the server, water, temperature or shock will easily do so. Especially if we're on a scale as small as a rack.

Don't put data you don't want to loose in a single location.

[0] https://www.ovh.com/world/news/press/cpl1787.fire-our-strasb...

Re:The 'Fuck You' Pattern by nine_k on Tuesday June 29, 2021 @01:00AM (#27671973)
A power or connectivity loss may happen. A DC does not need to be destroyed or something, just a backhoe acting up somewhere.
Re:The 'Fuck You' Pattern by wruza on Tuesday June 29, 2021 @12:16AM (#27671708)
You’re overblowing an imageboard checklist into a pack of elephants. I wonder how much I’d earn if I disaggregated every “make us another site quick” in my company in such a way. Well, writing html is easy. But then webpack… and then you want editable content… postgres/orm… and browser coverage, .babelrc… and of course we don’t serve it from our office, now it’s vps management and deployment configuration… and version control, do you know how hard is to remember all git essentials? We have to register a plan on gitlab and enter a card into it!

Smells like seven figures already.

Re:The 'Fuck You' Pattern by e12e on Tuesday June 29, 2021 @07:08AM (#27674225)
> Go an and run a site that serves massive amounts of cat pictures for peanuts!

Well, I was responding to "millions", so maybe "a handful of peanuts" might be more accurate...

I belive there was an open company/OCI post by neocities.org a good while back, detailing their budget - but I can't seem to find it again.

Re:The 'Fuck You' Pattern by e12e on Wednesday June 30, 2021 @05:39AM (#27686638)
Looks like Kyle dropped out of the open company initiative at some point, or let that one blog post drop off the site?

http://web.archive.org/web/20160310015118/https://blog.neoci...

> Our (rough) budget for 2015

(...)

> We have upgraded a lot of our infrastructure to provide for faster, more reliable service with more features, space, and bandwidth capabilities, which has pushed our operating bill to around $300/month (...) To prevent unexpected costs, we have made agreements with our providers to make this a fixed rate, so a sudden flood of traffic shouldn't change this dramatically.

(...)

> In order to become self sustaining with the current system, we will require approximately $3600 in annual recurring revenue. I think this is an achievable goal with our new site (we're almost 1/3 of the way there), and we have plenty of runway incase this takes longer than expected. We also have the ability to scale back on some of our expenses, as I believe I can shave off at least $100-150 from our monthly recurring expenses should we need to without too many problems.

> (...) We can scale a lot (~500k+ sites) with what we have now, and the scaling options we have are simple, predictable and affordable.

Re:The 'Fuck You' Pattern by bombcar on Monday June 28, 2021 @09:59PM (#27670802)
We just need a standardized cat and picture thereof - perhaps even expand the standard to cover one of each breed.

Then you can have a local copy and be sure you have every standard cat in existence!

Re:The 'Fuck You' Pattern by majewsky on Tuesday June 29, 2021 @05:14AM (#27673563)
No, we just need to train a neural network to generate cat pictures from a small set of inputs (like a uint8 for pose, a uint8 for breed, etc.) and then just transmit those parameters instead of the entire picture.
Re:The 'Fuck You' Pattern by gazby on Monday June 28, 2021 @10:35PM (#27671069)
This guy infrastructures.
Re:The 'Fuck You' Pattern by eyelovewe on Monday June 28, 2021 @07:48PM (#27669819)
Sounds like overkill, mate. I’ll go with your comments parent. Maybe 5 peanuts. In all seriousness, I’ll agree on cdn at 20 a month plus 3 @ 15 dollar nodes running out app and db, let’s go multimaster with a local db node on each app server, screw the c go for a and p, and yeah a single haproxy hardware thread, ha, so make that 2 single core nodes 1gb will do fine for 300k connections per second, um you mentioned staff? For what exactly?

Anyway, you can do anything the hard way, but what’s your point? That serving millions of cats per second requires big investment that must be recouped? I see most of this as rent seeking, and the infrastructure is economy of scale. Maybe you use tensor flow and Kafka inside Kubernetes, I dunno…

I agree that one can expect lame websites to act lame, but it’s out of sheer churlishness rather than necessity

Re:The 'Fuck You' Pattern by nine_k on Monday June 28, 2021 @08:29PM (#27670144)
Indeed, serving is the easy part, once it's set up. Going offline when the DC goes offline can be completely acceptable.

It's the picture intake pipeline that eats most of the resources. If you only want to serve the same collection, or a collection you personally increase by 1-2 cats a day, then it's fine.

Re:The 'Fuck You' Pattern by Dylan16807 on Monday June 28, 2021 @11:57PM (#27671595)
> It's the picture intake pipeline that eats most of the resources.

How so? Let's say you want to be somewhat competitive with imgur and handle a million images per day, and also that the peak rate is 4x your average. That comes out to 50 images per second. A thumbnail or few shouldn't really take more than a CPU-second, so you can handle that with 2-6 cheap servers. Let alone using spare cores on storage-optimized servers. And it's next to zero database load.

Re:The 'Fuck You' Pattern by cvwright on Monday June 28, 2021 @10:19PM (#27670960)
You think that Facebook and Google build these huge warehouse-size data centers all over the world for fun?

You think they wouldn’t rather have the cash?

Re:The 'Fuck You' Pattern by eyelovewe on Tuesday June 29, 2021 @07:55PM (#27683317)
You do realize that their data includes half of humanities personal lives, if you count WhatsApp and tinder. It ain’t all cat pictures, I’m afraid
Re:The 'Fuck You' Pattern by mdavis6890 on Monday June 28, 2021 @05:44PM (#27668594)
You make a great point that I hadn't considered before.

It's like rocket fuel! The faster you want to go, the more fuel you need. Which adds weight, so now you go slower, so you need more fuel...

I want to serve a cat picture, which would cost 1/100 of a cent. But at scale, it adds up. So now I show adds that generate 1.2/100 of a cent in revenue but add .5/100 cents in serving cost, which I can optimize with user data to add an additional .7/100 cents in revenue, but which adds .4/100 cents in serving costs....

Re:The 'Fuck You' Pattern by cyrux004 on Monday June 28, 2021 @05:43PM (#27668588)
Shouldnt you include engineers compensation in "The infrastructure" thats where a lot of money goes.
Re:The 'Fuck You' Pattern by tomc1985 on Monday June 28, 2021 @06:33PM (#27669145)
A bloated payroll to support a bloated advertising platform with a bloated AI-based recommendation algorithm when all it should be doing is loading a descending-order list of most recent submissions meeting the category criteria....

Don't talk about costs, this crap is self-inflicted

Re:The 'Fuck You' Pattern by saurik on Monday June 28, 2021 @06:44PM (#27669262)
The hosting and bandwidth costs of images and especially video is non-trivial. I agree that the vast majority of the costs of these overly-large companies comes from what happens while trying to make them profitable, and then all of the engineering work that goes into that... but that all starts because running the site in the first place wasn't free and you have to do something to make them at least sustainable, with your "obvious" options being to either: 1) ask for donations (which likely won't pay for the site if you get popular: I know this from experience); 2) require people to pay some small subscription fee (which will make the site less fun for everyone); 3) pray ubiquitous micropayments eventually happens (i work on this problem as part of Orchid, but it still hasn't happened ;P); 4) sell "something else" to whales, like t-shirts or the ability to "guild" messages (the strategies reddit was trying to do as they resisted ads for a long time... maybe these work well enough?); or 5) start trying to sell ads and become the thing you hate. The only other strategies I have seen tend to either drag you towards #5 or simply cause other dark patterns, such as taking a cut of "tips" (what TikTok and Twitch do), which at bare minimum incentivizes the "don't you dare talk about alternative payment systems in our ecosystem: all payments must go through us" model sin (which can be fought against if you have a lot of willpower and remain private--I never required this with Cydia as I considered it the original sin my entire market existed to undermine--but I see the motivation and it feels kind of inevitable). I personally bet the only real "correct" solution is essentially #3... if we ever get to the point where the fees for that are as easy to pay and as low as the fees you pay for electricity (which works on a similar model).
Re:The 'Fuck You' Pattern by tomc1985 on Monday June 28, 2021 @06:50PM (#27669314)
How about running ads but not letting them dominate your product? I see nothing wrong with running a costs-plus business model but VC comes in and demand all these stupid companies dominate the world so they can get their 500x return or whatever. Then they go public and now the only thing that matters is shareholder value, fuck the users and OG supporters...

Every one of these companies start with a nice mission and it ends up getting perverted into the hellscape that is tech today. Every damn time.

Re:The 'Fuck You' Pattern by pm90 on Monday June 28, 2021 @07:10PM (#27669514)
Its really really hard for a company whose ultimate goal is profit, to give up opportunities to do so. Its just not gonna happen. You can have all the ethical people you want, but people change and it takes just one "smart" engineer/product manager to realize how much money is on the table, then use that to grab power and get the same changes everywhere.

Relying on people doing the right thing is an unstable equilibrium. IMO the only way to really ensure this is with regulations and compliance controls. Laws like GDPR and CCPA have to be followed by the business.

Re:The 'Fuck You' Pattern by jokethrowaway on Monday June 28, 2021 @10:20PM (#27670968)
I don't think it's hard per-se. I think it's hard if you have investors.

I know plenty of founders who would be happy with lower constant profits, less employees, in order to have a company which doesn't actively psychologically harm its users, a less stressful working environment.

Gumroad is the perfect example of this.

The problems come from VC money, which force a company to chase 500x returns or die trying. More in general, whenever there is centralisation of resources, problems ensue at some point later on.

Regulations are not the answer - established companies find a way to arginate them (look at Google playing with cookies), smaller companies trying to go live and compete have a slower time to market and die, benefiting the established companies.

On top of this regulations introduce a different set of problems. What if I'm okay being on a website which doesn't respect my privacy? What if I care more about reading a news article and never visit that website again more than I care about them respecting some rules regarding what kind of tracking they do?

Instead of assuming we, the lawmakers, are omniscient and perfect, I think we should just get out of the way of the market and limit governmental influence to reduce the number of monopolies and corruption that gets into the real world - and then bubbles up in various ways, chasing infinte growth being one of them.

To 100 more Gumroads.

Re:The 'Fuck You' Pattern by EarthLaunch on Monday June 28, 2021 @08:51PM (#27670320)
> Relying on people doing the right thing is an unstable equilibrium. IMO the only way to really ensure this is with regulations and compliance controls.

Isn’t this ironic? The government is people. I guess this just moves the onus of people doing the right thing, up to government. If you’re right, that destabilizes government. Which I guess is what happens when it stops corps; corps then need to take it over.

Re:The 'Fuck You' Pattern by tomc1985 on Monday June 28, 2021 @08:40PM (#27670228)
I get that, but there is something very wrong with business culture if that is the only outcome guaranteed. There are lots of examples of successful companies whose business model is not based around the unrestrained pursuit of pure profit at the expense of customers... look at Nintendo, for example, or Disney.
Re:The 'Fuck You' Pattern by DevKoala on Monday June 28, 2021 @09:02PM (#27670411)
Shhh! Some people are trying to justify their jobs as a service to mankind.
Re:The 'Fuck You' Pattern by user123456780 on Monday June 28, 2021 @06:15PM (#27668947)
> "The infrastructure" to share cat pictures cost peanuts

lol at this. Go setup that infrastructure and see how far you get for peanuts.

Re:The 'Fuck You' Pattern by jopsen on Monday June 28, 2021 @06:35PM (#27669169)
A single server can host an awful lot of cat pictures. Especially if you apply a little compression.

In fact: I'm sure there are many cat image sharing sites out there. A random search landed me here: https://www.funnycatpix.com/_pics/Hmmmmmm890.htm

Seems like a high-quality HN-like website, hehe :)

Re:The 'Fuck You' Pattern by greedo on Monday June 28, 2021 @08:33PM (#27670179)
Considering that pof.com used to run on a single server and handled more dating traffic than all of its competitors combined, you can go pretty damn far for peanuts.
Re:The 'Fuck You' Pattern by mtnGoat on Monday June 28, 2021 @10:56PM (#27671183)
Folks that come from over funded and over engineered companies don’t know this. Because they haven’t experienced it, it’s simply not possible to them. If you have money to burn, you do so.

My company has grown 15x in the last few years and our total infra costs continue to go down, because we pay attention to them.

Re:The 'Fuck You' Pattern by yibg on Monday June 28, 2021 @11:18PM (#27671330)
Every posting about one of the larger scale companies have to have a “why does it cost x / take y engineers to run this thing?” comment. It’s like everyone at google / Facebook / Amazon are too stupid to realize they can run the entire thing out of a server in the basement maintained by 5 engineers.

If it was so simple to host Instagram for peanuts I wonder why no one has done it.

Re:The 'Fuck You' Pattern by symlinkk on Monday June 28, 2021 @06:55PM (#27669367)
I honestly don’t think it would be hard, especially with AWS abstracting away all scaling issues from you.
Re:The 'Fuck You' Pattern by tofuahdude on Monday June 28, 2021 @07:08PM (#27669495)
Hard to abstract away the money that it'll cost - which is the original point here. They're doing something that costs money, and people want to use that something, so they're doing what it takes to make money from it.
Re:The 'Fuck You' Pattern by symlinkk on Monday June 28, 2021 @07:58PM (#27669885)
If your business isn’t scaling with users, it’s a business model problem not a technical one
Re:The 'Fuck You' Pattern by myrandomcomment on Tuesday June 29, 2021 @12:37AM (#27671820)
AWS makes around 26% margins (not broken out, it’s markets best guess). At a medium size it can make more cost effect sense to run your own……with the understanding that you have to build a cattle farm, and pets are forbidden. If you every allow a pet, then just go to AWS, et.al.
Re:The 'Fuck You' Pattern by gorgoiler on Tuesday June 29, 2021 @02:00AM (#27672336)
You need content, which means you need to become popular.

This drives up complexity of the business operation overall, not just the infrastructure.