News for hackers. Stuff that matters.
Re:Does it scale? Who cares (2011) by devduderino on Saturday April 08, 2017 @11:40PM (#14070294)
I care because it usually goes like this: Product manager > "Niche sass app {x} will never need to support more than 10-20 users"

Two weeks after launch > "We have 10k users and counting, why didn't you architect this for scale?"

Always assume you underestimated the scope of the project.

Re:Does it scale? Who cares (2011) by skinp on Sunday April 09, 2017 @12:07AM (#14070389)
Interesting. It might be a by-product of having worked in larger companies, but my experience has pretty much always been: Product manager > "We need this to work for 10k r/s at launch". Actual launch throughput (and all time peak): 100 r/s

Now I always mostly just assume (capacity asked / 10) is more than enough!

Re:Does it scale? Who cares (2011) by Nican on Sunday April 09, 2017 @02:03PM (#14073511)
As an employee of a large company, I can relate to this experience. The manager has grandiose views about the product, but has no data to back his claims.
Re:Does it scale? Who cares (2011) by virmundi on Saturday April 08, 2017 @11:47PM (#14070317)
I've literally been waffling back and forth between UUID and BigInt for the last week. The reason for this is I need to handle eventually distributed systems. Do I need UUIDs? Maybe not, but after much debt, I've decided that the storage requirements of such an index in memory is worth the ability to move from Postgres to Postgresql-xl.
Re:Does it scale? Who cares (2011) by ProblemFactory on Sunday April 09, 2017 @09:12AM (#14072109)
> Do I need UUIDs? Maybe not, but after much debt, I've decided that the storage requirements of such an index in memory is worth the ability to move from Postgres to Postgresql-xl.

There is a major benefit to UUIDs besides server-side scaling concerns: you can generate them offline in mobile apps and other client-side applications.

With serial primary keys, you are often stuck between waiting for a server round trip before saving data locally, or inventing complicated solutions for referring to not-yet-uploaded data locally.

Re:Does it scale? Who cares (2011) by qaq on Saturday April 08, 2017 @11:51PM (#14070336)
The problem is not with memory problem is that after checkpoint postgres will do full page writes for serial you will be touching way fewer pages compared to UUID so write amplification for UUID vs bigint can be 60X+
Re:Does it scale? Who cares (2011) by virmundi on Sunday April 09, 2017 @12:02AM (#14070369)
Can you go into a bit more detail? I've seen amplification referenced before. My estimation is that the database will be pretty much read heavy. I will need to split a given database across multiple instances.
Re:Does it scale? Who cares (2011) by qaq on Sunday April 09, 2017 @12:15AM (#14070425)
https://blog.2ndquadrant.com/on-the-impact-of-full-page-writ... this is fairly detailed analysis of this issue
Re:Does it scale? Who cares (2011) by virmundi on Sunday April 09, 2017 @12:47AM (#14070526)
Thank you. I'm reading over it now and for the next few days (to really grok it). I do wonder how bad it will be for low write situations. At the same time I'm open to switching to BigInt and using Postgresql-xl.
Re:Does it scale? Who cares (2011) by Dylan16807 on Sunday April 09, 2017 @04:05AM (#14071049)
How about simply using non-random UUIDs?
Re:Does it scale? Who cares (2011) by qaq on Sunday April 09, 2017 @10:57AM (#14072578)
That would def. help
Re:Does it scale? Who cares (2011) by graphememes on Sunday April 09, 2017 @01:49AM (#14070707)
luid, check it out
Re:Does it scale? Who cares (2011) by qaq on Saturday April 08, 2017 @11:48PM (#14070323)
Ok what can you write it in that you can't scale to 10K users by just adding web servers? The hard to scale part is storage layer but even crappy options like RDS will happily support 10K users.
Re:Does it scale? Who cares (2011) by tbrownaw on Saturday April 08, 2017 @11:56PM (#14070352)
Microsoft Access.

Works fine for something one team uses, works... not fine for something the entire corporation uses.

Re:Does it scale? Who cares (2011) by qaq on Sunday April 09, 2017 @12:03AM (#14070372)
Well there are tools to help easily migrate Access to SQL Server, but this is moving fairly far from the original premise of this topic.
Re:Does it scale? Who cares (2011) by Dylan16807 on Sunday April 09, 2017 @03:54AM (#14071014)
Does Access actually work well for 20 users?
Re:Does it scale? Who cares (2011) by Clubber on Sunday April 09, 2017 @10:50AM (#14072547)
Last time I built something in Access was on Access 1997. I believe it had about 100 concurrent users and over a gig of data. Those both exceed the hard limits of access at the time so I had a single read/write database on a share and multiple read only databases on shares. I had to split tables among multiple database files as well.

My direct boss was the CFO and he was cheap as hell. It's amazing the things we can come up with when we have to.

Re:Does it scale? Who cares (2011) by nyir on Sunday April 09, 2017 @06:23AM (#14071510)
Anecdotally it doesn't: had to use a time tracking program that used an Access database on a shared network folder and would frequently freeze up or crash due to locking issues (with around that many users). I imagine it could be done better though.
Re:Does it scale? Who cares (2011) by devduderino on Saturday April 08, 2017 @11:58PM (#14070354)
Yep
Re:Does it scale? Who cares (2011) by cyberferret on Sunday April 09, 2017 @12:42AM (#14070505)
> but even crappy options like RDS

Curious as to why you see RDS as a 'crappy' option??

Given the choice of building and maintaining a MySQL box in the corner of your bedroom and hosting it on RDS - I know which way I would go (and I have done both over the years).

If you are talking 'scaling' in terms of the OP's article, then RDS is almost a no brainer, and you can scale your instances (and add replicated instances etc.), hide it behind a VPN, set up firewalls to prevent DDoS attacks all in a matter of minutes, with a few mouse clicks.

Re:Does it scale? Who cares (2011) by qaq on Sunday April 09, 2017 @12:44AM (#14070516)
The highest IOPS you can get on RDS is about a single consumer grade SSD.
Re:Does it scale? Who cares (2011) by cyberferret on Sunday April 09, 2017 @12:59AM (#14070564)
Fair enough. But if the alternative is setting up a consumer grade PC on a consumer grade internet connection in a (non environmentally or power controlled) corner of a room at home, then surely we are looking at a similar situation?

Given that spooling up a RDS instance is quicker and cheaper than buying all the hardware and spending time installing and configuring an SQL server, then if time to market is a critical factor, you would choose RDS over a home grown solution, wouldn't you?

[Edit: I just realised that this is actually the point you were making, and I got sidetracked by my above question]

Re:Does it scale? Who cares (2011) by pbreit on Sunday April 09, 2017 @02:15AM (#14070771)
Have you heard of hosting? No one is running servers in their house. Nobody.
Re:Does it scale? Who cares (2011) by pbreit on Sunday April 09, 2017 @02:14AM (#14070768)
No one hosts MySQL out of their bedroom. C'mon. The alternative is hosting the DB yourself at a hosting provider which is what most do and it works fine.
Re:Does it scale? Who cares (2011) by flukus on Sunday April 09, 2017 @11:00PM (#14075679)
Everywhere I've worked that tried to architect for massive scales at the beginning has ended up with an architecture that inhibits scaling. Using correct indexes and transactions can go a long way.
Re:Does it scale? Who cares (2011) by pbreit on Sunday April 09, 2017 @02:11AM (#14070758)
Usually the opposite: business thinks it's going to get lots of users, engineers code for such, business way undershoots estimates, stuck with over-engineered app.

I've never heard of a PM saying such a thing.

Re:Does it scale? Who cares (2011) by emmelaich on Sunday April 09, 2017 @02:56AM (#14070868)
It's a good things to overestimate in a Big Corp, because the managers always penny pinch and will put the blame on you if things go to shit.

If you're a startup, go the other way.

Re:Does it scale? Who cares (2011) by romanovcode on Sunday April 09, 2017 @11:28AM (#14072725)
I don't know how shitty must a system be to not able to handle 10k users with vertical scaling.