Running 10M PostgreSQL Indexes in Production

Re:Running 10M PostgreSQL Indexes in Production by skrebbel on Tuesday June 13, 2017 @04:02AM (#14543455)

I'm not Michael, and I'm not sure if this is the answer you want to hear, but if you're a frontend developer the database is likely going to be the least of your worries when you start. Making and deploying the backend is going to take much more of your time than doing Postgres well.

There's a lot of blog posts on HN about bending databases like Postgres to pretty insane workloads. Heap's workload is a nice example of that. they're interesting blog posts but don't let them distract you. Most databases really do work pretty well by default, if you just use them the simplest way possible. Most apps don't have insane workloads, and especially when you start out they won't.

If you just make a table for every "thing" in your system and link them together with foreign keys you're pretty much set. Most popular frameworks (Rails, Django, etc) do this for you without even touching SQL.

This approach is fine for nearly every case. I bet even Heap started like that. Famously, Twitter started like that as well, and the only reason that got them in trouble (remember the failwhale?) was because they were growing faster than nearly any platform ever before. Most sites don't 10x their user count every half year, and if you do, lucky you. Scaling problems are luxury problems.

Half of HN is jealous of people who have scaling problems. I sure am. The other half, well, they won't have scaling problems because they're not shipping fast enough. Their backends would scale though. They read all the books on Postgres and doing it right from the start!

[ Parent | on HN ]

Re:Running 10M PostgreSQL Indexes in Production by StavrosK on Tuesday June 13, 2017 @09:07AM (#14544421)

This is exactly right. Install Postgres, add some indices on the columns you're querying, and you're probably good for the next ten years in that respect. Try to focus on making a product people want and making them aware of it instead. No company[1] has ever died because their tech stack wasn't good enough.

[1]: Approximately.

[ Parent | on HN ]

Re:Running 10M PostgreSQL Indexes in Production by everdev on Tuesday June 13, 2017 @11:39AM (#14545593)

I ran a web agency in Silicon Valley and I encountered numerous clients who's budgets were significantly hampered by having to maintain or scale poorly designed tech stacks. MediaZone was the biggest. They chose to use an in-house, proprietary ASP.net framework for their web platform. They were live streaming major sports programming back in 06/07 like Wimbledon. They had tons of customers, but their development run rate and slow speed significantly contributed to their parent company to shutting them down. It's true that most companies never get to their scale (~200 employees), but if you want to get that big, you have to plan a text stack early on that can accommodate your aspirations.

[ Parent | on HN ]

Re:Running 10M PostgreSQL Indexes in Production by nasalgoat on Tuesday June 13, 2017 @01:44PM (#14546815)

Well, how far in advance are you planning? It's easy to over-optimize when you're starting out and you end up all tangled in an overly-complex system that is over-engineered for your needs.

Everything in moderation. You can often run a single master PostgreSQL system into the millions of transactions a second without having to resort to sharding and other complexities.

[ Parent | on HN ]

Re:Running 10M PostgreSQL Indexes in Production by collyw on Tuesday June 13, 2017 @09:08AM (#14544430)

As someone who uses Django more or less full time, I don't think relying on the ORM without knowing any SQL is especially good advice.

Django's ORM is very easy to use, but it is not especially powerful, and when you want to do advanced queries (multiple join conditions for example) it often isn't possible. When someone asks me for some data from the database, usually its easier for me to go directly to the database than use the Django ORM.

Use the ORM to start with, but its most definitely a good idea to learn proper SQL .

[ Parent | on HN ]

Re:Running 10M PostgreSQL Indexes in Production by pas on Thursday June 15, 2017 @09:57AM (#14560382)

ORM is very much like hypermonadic supertemplate C+++++ type theory, you can only play with it if you know what you're doing, but don't want all that boilerplate (cognitive or in-code unit tests) in your head.

So, I think starting with PHP and raw SQL is a good thing for motivated learners.

And if you need motivation, try building a WordPress site with themes and plugins first. And maybe an Android app. And then something cute like a LED blinker with a raspberry.

[ Parent | on HN ]