News for hackers. Stuff that matters.
Re:Startup Mistakes: Choice of Datastore by mamcx on Thursday November 16, 2017 @09:37AM (#15712890)
BIG ERROR.

Like, MONUMENTAL.

That something (if chose very wrong) will totally derail you progress and will cost a lot of fix it later.

Is incredible. Nobody remember that the most cost effective way to fix a problem is in the early stages?

And, yes, the best overall primary datastore, like 90% of the cases, is a RDBMS. Very few actually need to use something else.

Re:Startup Mistakes: Choice of Datastore by geophile on Thursday November 16, 2017 @09:55AM (#15713038)
Yes and no.

Yes, choose carefully at the beginning, and use relational. More specifically, use Postgres.

But choose the right things to worry about at the beginning. So worry about something that can accommodate changing requirements, i.e., a relational database. Don't worry about scalability. You will be very, very lucky to ever have that problem. Worry about it then.

Re:Startup Mistakes: Choice of Datastore by StavrosK on Thursday November 16, 2017 @09:59AM (#15713075)
Also, if you find that some data is better suited for some other datastore, just move that data off. For example, I start with Postgres but almost always end up storing session data in Redis, for a bit more speed.
Re:Startup Mistakes: Choice of Datastore by edneypitta on Thursday November 16, 2017 @10:06AM (#15713158)
>something that can accommodate changing requirements, i.e., a relational database

Wouldn't NoSQL be better suited for this scenario? Genuinely curious.

Re:Startup Mistakes: Choice of Datastore by vidarh on Thursday November 16, 2017 @11:27AM (#15713926)
Postgres is one of the best NoSQL datastores you'll find, until/unless you get to the point where scaling a single Postgres database becomes a problem.

But the point is 1) you're unlikely to ever get there. 2) if you're lucky enough to get there, if you start with your data in a structured database and gradually e.g. move things to JSON blobs in your Postgres DB, then move the things that really, seriously needs it off into a more easily shardable NoSQL datastore, it's a far easier direction to go in than the opposite.

Recovering structure when you've tossed it out the window is a massive pain.

And even a lot of the time when you end up needings things like e.g. ElasticSearch for search or something like that, it may still be better to keep the canonical data store in Postgres and stream changes to a secondary store to scale reads. Or add caching.

I'm not suggesting there are no cases where NoSQL datastores can't be the right choice from the beginning. But if you don't have a very compelling reason to, it's probably not.

Re:Startup Mistakes: Choice of Datastore by mamcx on Thursday November 16, 2017 @11:13AM (#15713799)
Exist the myth RDBMS are not flexible (because it have schemas). To the contrary, the relational model is very flexible, and allow to model everything you could want. Even model a NoSql :)

And the Relational model is fairly simple. You could learn anything you need in minutes and just remember in add a few index here and there. For the small amount of things you need to do, is incredible how many features a RDBMS give you for free (like transactions, some basic axioms, and a query engine).

----

I don't remember the original quote, but is alike:

"Novices worry for code, Experts focus on data structures".

Properly chose your data-structures, schemas and data-layout will have a huge net impact in your code. That is why a well modeled database will perform well and allow to easily code against it.

This is the hard part, and most novices like to "defer" it.

In my times, we start designing the app first on the DB layer, including the queries, reports, etc. Now most start with the front-end or back-end... Imaging the are focusing in the abstract logic (because some can't imagine the datastore is part of it!) and ignore or reject the concept of learn what are the datastore capabilities.

That is like ignoring the documentation about arrays, building a layer on top of it, rejecting the idea of use arrays as full, and then wondering why his code performs bad and re-creating, badly, what it already have.

---

For fast prototypes, sqlite (stored in RAM) could be good. In the early stages I erase the DB in each run of the code (after the initial DB design, tweaking it). I continue to do that as far as possible, and only worry about migrations and all that when start shipping to customers.

Also, not be afraid of build several copies of the tables - like experiments- (customerA, customerB, etc), use views and peek on your DB documentation.

Re:Startup Mistakes: Choice of Datastore by SQLite on Thursday November 16, 2017 @12:29PM (#15714576)
> I don't remember the original quote, but is alike: "Novices worry for code, Experts focus on data structures".

The origin is Linus Torvalds on the Git mailing list. Here is a copy from LWN: https://lwn.net/Articles/193245/

The quote is ironic considering that Git uses a bespoke and not particularly well-designed key/value database, which has resulted in notorious usability problems in Git.

Re:Startup Mistakes: Choice of Datastore by extrapickles on Thursday November 16, 2017 @10:19AM (#15713284)
In general a SQL db will be a better choice for CRUD apps. They get slow if you try to recreate a SQL engine in your DB abstraction layer rather than letting the SQL engine do all the work.

NoSQL db should be used if you are doing something that they are designed for (eg: storing data that can’t have a schema).

Re:Startup Mistakes: Choice of Datastore by robinwassen on Thursday November 16, 2017 @10:36AM (#15713461)
RMDBs often has great schema migration tools which usually are no problem to use on small datasets.

NoSQL often rely on handling difference in the data model in the application layer, which can become messy and cumbersome if you switch a lot of requirements.

Re:Startup Mistakes: Choice of Datastore by geophile on Thursday November 16, 2017 @11:01AM (#15713701)
Well, they rely on doing pretty much everything in the application layer, which can become messy and cumbersome. Relational databases are the way they are for a reason. NoSQL is often selected, not because they address requirements better, but because developers are not even aware of the problems that relational databases solved years ago.
Re:Startup Mistakes: Choice of Datastore by davidgh on Thursday November 16, 2017 @10:33AM (#15713425)
If your goal is to build a sustainable business you must address two questions: Will I provide value people are willing to pay for? Will I be able to find those people profitably? Both answers must be yes.

You might be able to answer one of the questions without an MVP but very unlikely to answer both without. So you need to ship something.

It would be clearly stupid to not consider your technology choices at all in the early stages. But the best decisions are usually the ones that allow you to determine the answers to both questions as soon as possible. Because if you aren’t able to eventually answer yes to both questions you will not build a sustainable business.

If you are providing value, people are paying you and you can find customers, the technology sins from the past can be addressed. It might be really hard. But at least you’ll be working on something that matters.

Spending a massive amount of time on getting it “just right” will most likely teach you that you’ve developed the perfect solution to a problem no one cares about.

Speaking from experience of being involved with two acquisitions of companies I started (on the selling side) I can tell you that the “technology stack” discussion is a tiny footnote to the discussions on revenue, sales, customer lifetime value, etc.

Re:Startup Mistakes: Choice of Datastore by mamcx on Thursday November 16, 2017 @11:25AM (#15713895)
>Spending a massive amount of time on getting it “just right” will most likely teach you that you’ve developed the perfect solution to a problem no one cares about.

And this is the reason a RDBMS is totally better. You NOT need to "spend a massive amount of time". With this:

1- You get a totally proven and mature tech 2- With massive tolling support and documentation 3- With the capabilities to model even most NoSql structures. 4- With a query engine that will outperform most developers. And more flexible that most NoSql give 5- With enough scalability that you will ever need for your MVP and beyond 6- And with the ability to plug specialized storage like column stores, time stores and more without complicating your code duplicating efforts.

---

P.D: I have been part of several projects that drink the NoSql koolaid, and be part of the weeks-long coding efforts that could be summarized as:

"This could have been a one line sql code/or index ..."

My last one, rebuilding core aspects of a RDBMS and still building features... dedicating large part of dev time is really "Spending a massive amount of time on getting it “just right”"

Re:Startup Mistakes: Choice of Datastore by davidgh on Thursday November 16, 2017 @12:41PM (#15714681)
I was speaking less about the specific debate between NoSQL and RDBMS and more about being cautious to not over-think the technical aspects in general.

Speaking specifically about NoSQL: I’ve been able to launch entire MVPs using DynamoDB, some Lambda functions and HTML / JavaScript in about the same amount of time it would have taken me to plan out, build and refactor a proper schema. And I usually learned through the MVPs that my idea was stupid and no one cared about what I built, much less about my choice of database.

Re:Startup Mistakes: Choice of Datastore by vidarh on Thursday November 16, 2017 @11:38AM (#15714043)
Consider that for a startup money now is worth a lot more than money even 6-12 months from now, unless you're failing anyway.

Let's say the valuation increases 10-fold between the MVP and the next round. Any $1 in investment I can defer from the first round to the second round then costs shareholders a tenth as much dilution. So even if I end up spending far more money, I may come out ahead, as long as the choices won't hold us back until then.

This doesn't take into account that I might not even be able to raise enough for a more expensive solution, so I might not have a choice.

So a lot of the time the right choice is what costs the least amount right now even if you know it will cause costly re-engineering down the road if you're successful.

For a mature business the tradeoffs are different. If you know your growth rates will be modest, then getting it right from the start matters a lot more.

Re:Startup Mistakes: Choice of Datastore by BukhariH on Thursday November 16, 2017 @10:45AM (#15713571)
Also, later on if you decided that a RDBMS doesn't suit your needs it's significantly easier to migrate RDBMS -> NoSQL versus the other way around.
Re:Startup Mistakes: Choice of Datastore by tanglefuzz on Thursday November 16, 2017 @09:55AM (#15713035)
What's the use of fixing something in a cost effective way if your startup is going to fail anyway?

You need to validate and test first. Get customers ASAP. If that means writing shitty code and using a shitty database or not even a database at all, than that's great. You can always convert to a better system afterwards.

I do agree that the longer you wait, the harder it gets to rewrite code and convert to different systems, but the most important thing for any startup is to get their idea validated as soon as possible.

Re:Startup Mistakes: Choice of Datastore by StavrosK on Thursday November 16, 2017 @10:02AM (#15713113)
This is just the balance fallacy. Relational and document databases aren't equally likely to be the type you need. Most of the times you have relational data, so you want relational databases. Pick the datastore that's more likely to fit your data from the start.
Re:Startup Mistakes: Choice of Datastore by relyio on Thursday November 16, 2017 @10:01AM (#15713105)
The point OP is making is that in a startup, you don't do engineering for the sake of engineering. In the end, what matters is generating wealth.

Obviously, you want a sane legacy for the future but unless you already have hit that sweet product-market fit spot it is likely that you will spend a non-trivial amount of time optimizing (prematurely) for something that's in fact orthogonal to what you actually need.

The trick, as usual, is to balance engineering and business depending on which stage your at.

Re:Startup Mistakes: Choice of Datastore by vidarh on Thursday November 16, 2017 @11:45AM (#15714121)
This, I find, is one of the hardest things about hiring developers for a startup. Because on one hand you want people who knows how to do things right.

On the other hand, you want people who are prepared to accept that decisions that are insane in a stable business can be very much rational in a business where the cost of cash to fund large engineering changes is dropping at a crazy rate if you're successful, and doesn't matter if you're not (because you'll never get to it).

Of course it's good to avoid completely pointless mistakes. But it often matters far more to deliver.

Re:Startup Mistakes: Choice of Datastore by onion2k on Thursday November 16, 2017 @10:29AM (#15713380)
That something (if chose very wrong) will totally derail you progress and will cost a lot of fix it later.

A bad tech decision in a problem you can fix later, even if it's an expensive problem[1]. Having no customers isn't a problem you can fix later because your startup will fail and you'll have to close it. Consequently, at the start, you should work on getting paying customers over everything else, including prevaricating about what database to use. Anything is better than not making a decision.

All that said, "just use Postgres" as many people here are suggesting is actually pretty good advice.

[1] Unless it's so bad fixing it will sink your startup, in which case you have to live with it. That does happen but it has to be really bad.

Re:Startup Mistakes: Choice of Datastore by dang on Friday November 17, 2017 @12:43AM (#15719284)
Could you please not use uppercase for emphasis? It's basically yelling. Asking not to do that is one of the original HN guidelines: https://news.ycombinator.com/newsguidelines.html. Imagine this comment in all caps and I'm sure you'll understand why.
Re:Startup Mistakes: Choice of Datastore by notyourday on Thursday November 16, 2017 @10:11AM (#15713197)
It won't.