Distant relative of 5 whys. We need NoSql document store -> so we can store json blobs -> so we can do databasing at app level -> because DBAs with their insistence on schemas are slow. Oh, so we can solve the problems by hiring one DBA and maybe training two devs instead of hiring full dev team and refactoring stuff for a year?
Funny thing is we are now migrating stuff out of datastore (and new stuff is not in datastore to begin with) into an RDBMS, but we are doing it microservice style with each microservice having its own separate database. So relationships are now cross-services concerns...
Not that having EVERYTHING in a single DB is the best approach always, but IMO we should default to keep everything in one single DB.
Put up 2 or 3 read replicas, split your queries so writes happen to main and reads come from replicas (supported out of the box by many modern ORMs), and you can scale to millions in daily active users for most startup workloads.
Really the hard part of BI is that folks who need the info don’t wanna learn SQL. The ones who can do SQL, will struggle to keep up with your changing schema.
Data analysts are fine with SQL though. Every "get into data analysis as a career" course will teach you SQL (about 70% of what the querynomicon teaches [1]).
Yes! I haven’t seen startups hiring these though. Somehow I always end up doing this as a side-gig on my engineering job.
Add analytics reporting views in your app database as the 'API' is the way.
Ugh. That sounds good on paper, but in practice it can become a problem. You're making your _database_ schema a part of the public API. It's an example Hyrum's Law, people will, sooner rather than later, start depending on internal details of the data representation.
And your development velocity will crater, as you'll now need to update all the reports (that are not necessarily even tracked in version control!).
Investing some time early to add code to pull out the data relevant for analytics can be worthwhile.
There's also a question of the personal information.
Realistically this guide should be bifurcated in terms of scale.
Yes this is good advice, until you get really large scale you don't need anything more fancy than some SQL in a read replica.
Wait, was that the reason people were doing NoSQL? JSON support? I thought it was about sharding, write scalability, etc.
My favorite example is that Twitter used mysql for all tweets, writing ~5k/s 24/7/365, until about 2016ish. Well into being a public company with billions in revenue and 300mm+ MAUs.
3/4 companies in the Bay Area senior software engineer interviews require a System Design interview where they will tell you "what if you had 10m users" and expect a distributed write-heavy sharding answer