News for hackers. Stuff that matters.
Re:Startup Mistakes: Choice of Datastore by onion2k on Thursday November 16, 2017 @09:16AM (#15712718)
Just pick something and build an MVP with it. Then get on with the hard part of finding paying customers. You can fix the bad tech decisions later. Without customers it won't matter which database you used before your startup failed.
Re:Startup Mistakes: Choice of Datastore by jrobn on Thursday November 16, 2017 @09:44AM (#15712951)
I would suggest that something be PostgreSQL. With JSONB and and all the other goodies in version 10 they are too numerous to list there is no reason not to use it for 90% of apps. It WILL scale until you need something more specialized.

Postgres is far from a boring relational database. Don’t make the mistake of building a mvp on rickety stilts and then swap them out later for proper columns while trying to run.

Re:Startup Mistakes: Choice of Datastore by hvidgaard on Friday November 17, 2017 @04:11AM (#15720401)
Postgre will be good for 99.99% of all workloads. And when you need something else, you should have enough revenue to migrate the parts that make sense to a different datastore.
Re:Startup Mistakes: Choice of Datastore by BozeWolf on Thursday November 16, 2017 @09:45AM (#15712952)
Agree! Most of us techies think tech is more important than sales. We also loathe bad design decisions. But your customers (in most cases) don't care!

But I always wonder exactly "what" is easier with mongo or another NoSQL db. Just get a relation db like PostgreSQL, MySQL or SQLite for that matter. If your site starts to get performance problems go to the pub, celebrate a bit of success and then optimise!

Re:Startup Mistakes: Choice of Datastore by maephisto on Thursday November 16, 2017 @09:48AM (#15712977)
Yup, people stuck on optimising (or overthinking which datastore to use) before actually having a business are missing the point.
Re:Startup Mistakes: Choice of Datastore by BozeWolf on Thursday November 16, 2017 @09:53AM (#15713021)
Or they miss the point when overthinking the whole code for the program/website. Get it done. Start finding customers.

The worst code I worked with was at travel agency startup and they are REALLY successful now. It was REALLY REALLY bad and hard to maintain. The owner of the business just said: well, bugs happen. But when a customer (not too many of course) encounters one, they probably call the helpdesk. Sounds stupid, worked great, business wise.

4 years later, I am sure they still have problems with the crappy code, but the business is fine!

Re:Startup Mistakes: Choice of Datastore by maephisto on Thursday November 16, 2017 @10:04AM (#15713128)
Agree, the mantra of success is: GET SHIT DONE! Customers do not care what's your code coverage or how maintainable is your code. Or if it's Mongo or Postgres. They want features, they want it now, and if you can't deliver that because you are too busy refactoring that's bad.
Re:Startup Mistakes: Choice of Datastore by mannykannot on Thursday November 16, 2017 @10:00AM (#15713082)
People starting a business need to be able to consider more than one thing at a time.
Re:Startup Mistakes: Choice of Datastore by StavrosK on Thursday November 16, 2017 @09:19AM (#15712735)
I agree, but it's better if later you only have to fix ten bad decisions instead of twenty.
Re:Startup Mistakes: Choice of Datastore by mamcx on Thursday November 16, 2017 @09:38AM (#15712895)
Is even worse to fix 1, but big.

DataStore is 1 that is too big to pick wrong.

Re:Startup Mistakes: Choice of Datastore by jijji on Thursday November 16, 2017 @10:12AM (#15713215)
you pick "something" based on the requirements of the application or whatever solution you're building. if your requirements mention that you need to have multiple people accessing the database at the same time, RDBMS environments like MySQL, Postgres, Oracle would be appropriate. If it is a single application instance with no multi-user access, SQLite might be appropriate. However, if you're choosing SQLite, FileMaker, FoxPro, MS Access, etc and your requirements specify that you needed multi-user access, then you've made a large mistake.
Re:Startup Mistakes: Choice of Datastore by mamcx on Thursday November 16, 2017 @09:37AM (#15712890)
BIG ERROR.

Like, MONUMENTAL.

That something (if chose very wrong) will totally derail you progress and will cost a lot of fix it later.

Is incredible. Nobody remember that the most cost effective way to fix a problem is in the early stages?

And, yes, the best overall primary datastore, like 90% of the cases, is a RDBMS. Very few actually need to use something else.

Re:Startup Mistakes: Choice of Datastore by geophile on Thursday November 16, 2017 @09:55AM (#15713038)
Yes and no.

Yes, choose carefully at the beginning, and use relational. More specifically, use Postgres.

But choose the right things to worry about at the beginning. So worry about something that can accommodate changing requirements, i.e., a relational database. Don't worry about scalability. You will be very, very lucky to ever have that problem. Worry about it then.

Re:Startup Mistakes: Choice of Datastore by StavrosK on Thursday November 16, 2017 @09:59AM (#15713075)
Also, if you find that some data is better suited for some other datastore, just move that data off. For example, I start with Postgres but almost always end up storing session data in Redis, for a bit more speed.
Re:Startup Mistakes: Choice of Datastore by edneypitta on Thursday November 16, 2017 @10:06AM (#15713158)
>something that can accommodate changing requirements, i.e., a relational database

Wouldn't NoSQL be better suited for this scenario? Genuinely curious.

Re:Startup Mistakes: Choice of Datastore by vidarh on Thursday November 16, 2017 @11:27AM (#15713926)
Postgres is one of the best NoSQL datastores you'll find, until/unless you get to the point where scaling a single Postgres database becomes a problem.

But the point is 1) you're unlikely to ever get there. 2) if you're lucky enough to get there, if you start with your data in a structured database and gradually e.g. move things to JSON blobs in your Postgres DB, then move the things that really, seriously needs it off into a more easily shardable NoSQL datastore, it's a far easier direction to go in than the opposite.

Recovering structure when you've tossed it out the window is a massive pain.

And even a lot of the time when you end up needings things like e.g. ElasticSearch for search or something like that, it may still be better to keep the canonical data store in Postgres and stream changes to a secondary store to scale reads. Or add caching.

I'm not suggesting there are no cases where NoSQL datastores can't be the right choice from the beginning. But if you don't have a very compelling reason to, it's probably not.

Re:Startup Mistakes: Choice of Datastore by mamcx on Thursday November 16, 2017 @11:13AM (#15713799)
Exist the myth RDBMS are not flexible (because it have schemas). To the contrary, the relational model is very flexible, and allow to model everything you could want. Even model a NoSql :)

And the Relational model is fairly simple. You could learn anything you need in minutes and just remember in add a few index here and there. For the small amount of things you need to do, is incredible how many features a RDBMS give you for free (like transactions, some basic axioms, and a query engine).

----

I don't remember the original quote, but is alike:

"Novices worry for code, Experts focus on data structures".

Properly chose your data-structures, schemas and data-layout will have a huge net impact in your code. That is why a well modeled database will perform well and allow to easily code against it.

This is the hard part, and most novices like to "defer" it.

In my times, we start designing the app first on the DB layer, including the queries, reports, etc. Now most start with the front-end or back-end... Imaging the are focusing in the abstract logic (because some can't imagine the datastore is part of it!) and ignore or reject the concept of learn what are the datastore capabilities.

That is like ignoring the documentation about arrays, building a layer on top of it, rejecting the idea of use arrays as full, and then wondering why his code performs bad and re-creating, badly, what it already have.

---

For fast prototypes, sqlite (stored in RAM) could be good. In the early stages I erase the DB in each run of the code (after the initial DB design, tweaking it). I continue to do that as far as possible, and only worry about migrations and all that when start shipping to customers.

Also, not be afraid of build several copies of the tables - like experiments- (customerA, customerB, etc), use views and peek on your DB documentation.

Re:Startup Mistakes: Choice of Datastore by SQLite on Thursday November 16, 2017 @12:29PM (#15714576)
> I don't remember the original quote, but is alike: "Novices worry for code, Experts focus on data structures".

The origin is Linus Torvalds on the Git mailing list. Here is a copy from LWN: https://lwn.net/Articles/193245/

The quote is ironic considering that Git uses a bespoke and not particularly well-designed key/value database, which has resulted in notorious usability problems in Git.

Re:Startup Mistakes: Choice of Datastore by extrapickles on Thursday November 16, 2017 @10:19AM (#15713284)
In general a SQL db will be a better choice for CRUD apps. They get slow if you try to recreate a SQL engine in your DB abstraction layer rather than letting the SQL engine do all the work.

NoSQL db should be used if you are doing something that they are designed for (eg: storing data that can’t have a schema).

Re:Startup Mistakes: Choice of Datastore by robinwassen on Thursday November 16, 2017 @10:36AM (#15713461)
RMDBs often has great schema migration tools which usually are no problem to use on small datasets.

NoSQL often rely on handling difference in the data model in the application layer, which can become messy and cumbersome if you switch a lot of requirements.

Re:Startup Mistakes: Choice of Datastore by geophile on Thursday November 16, 2017 @11:01AM (#15713701)
Well, they rely on doing pretty much everything in the application layer, which can become messy and cumbersome. Relational databases are the way they are for a reason. NoSQL is often selected, not because they address requirements better, but because developers are not even aware of the problems that relational databases solved years ago.
Re:Startup Mistakes: Choice of Datastore by davidgh on Thursday November 16, 2017 @10:33AM (#15713425)
If your goal is to build a sustainable business you must address two questions: Will I provide value people are willing to pay for? Will I be able to find those people profitably? Both answers must be yes.

You might be able to answer one of the questions without an MVP but very unlikely to answer both without. So you need to ship something.

It would be clearly stupid to not consider your technology choices at all in the early stages. But the best decisions are usually the ones that allow you to determine the answers to both questions as soon as possible. Because if you aren’t able to eventually answer yes to both questions you will not build a sustainable business.

If you are providing value, people are paying you and you can find customers, the technology sins from the past can be addressed. It might be really hard. But at least you’ll be working on something that matters.

Spending a massive amount of time on getting it “just right” will most likely teach you that you’ve developed the perfect solution to a problem no one cares about.

Speaking from experience of being involved with two acquisitions of companies I started (on the selling side) I can tell you that the “technology stack” discussion is a tiny footnote to the discussions on revenue, sales, customer lifetime value, etc.

Re:Startup Mistakes: Choice of Datastore by mamcx on Thursday November 16, 2017 @11:25AM (#15713895)
>Spending a massive amount of time on getting it “just right” will most likely teach you that you’ve developed the perfect solution to a problem no one cares about.

And this is the reason a RDBMS is totally better. You NOT need to "spend a massive amount of time". With this:

1- You get a totally proven and mature tech 2- With massive tolling support and documentation 3- With the capabilities to model even most NoSql structures. 4- With a query engine that will outperform most developers. And more flexible that most NoSql give 5- With enough scalability that you will ever need for your MVP and beyond 6- And with the ability to plug specialized storage like column stores, time stores and more without complicating your code duplicating efforts.

---

P.D: I have been part of several projects that drink the NoSql koolaid, and be part of the weeks-long coding efforts that could be summarized as:

"This could have been a one line sql code/or index ..."

My last one, rebuilding core aspects of a RDBMS and still building features... dedicating large part of dev time is really "Spending a massive amount of time on getting it “just right”"

Re:Startup Mistakes: Choice of Datastore by davidgh on Thursday November 16, 2017 @12:41PM (#15714681)
I was speaking less about the specific debate between NoSQL and RDBMS and more about being cautious to not over-think the technical aspects in general.

Speaking specifically about NoSQL: I’ve been able to launch entire MVPs using DynamoDB, some Lambda functions and HTML / JavaScript in about the same amount of time it would have taken me to plan out, build and refactor a proper schema. And I usually learned through the MVPs that my idea was stupid and no one cared about what I built, much less about my choice of database.

Re:Startup Mistakes: Choice of Datastore by vidarh on Thursday November 16, 2017 @11:38AM (#15714043)
Consider that for a startup money now is worth a lot more than money even 6-12 months from now, unless you're failing anyway.

Let's say the valuation increases 10-fold between the MVP and the next round. Any $1 in investment I can defer from the first round to the second round then costs shareholders a tenth as much dilution. So even if I end up spending far more money, I may come out ahead, as long as the choices won't hold us back until then.

This doesn't take into account that I might not even be able to raise enough for a more expensive solution, so I might not have a choice.

So a lot of the time the right choice is what costs the least amount right now even if you know it will cause costly re-engineering down the road if you're successful.

For a mature business the tradeoffs are different. If you know your growth rates will be modest, then getting it right from the start matters a lot more.

Re:Startup Mistakes: Choice of Datastore by BukhariH on Thursday November 16, 2017 @10:45AM (#15713571)
Also, later on if you decided that a RDBMS doesn't suit your needs it's significantly easier to migrate RDBMS -> NoSQL versus the other way around.
Re:Startup Mistakes: Choice of Datastore by tanglefuzz on Thursday November 16, 2017 @09:55AM (#15713035)
What's the use of fixing something in a cost effective way if your startup is going to fail anyway?

You need to validate and test first. Get customers ASAP. If that means writing shitty code and using a shitty database or not even a database at all, than that's great. You can always convert to a better system afterwards.

I do agree that the longer you wait, the harder it gets to rewrite code and convert to different systems, but the most important thing for any startup is to get their idea validated as soon as possible.

Re:Startup Mistakes: Choice of Datastore by StavrosK on Thursday November 16, 2017 @10:02AM (#15713113)
This is just the balance fallacy. Relational and document databases aren't equally likely to be the type you need. Most of the times you have relational data, so you want relational databases. Pick the datastore that's more likely to fit your data from the start.
Re:Startup Mistakes: Choice of Datastore by relyio on Thursday November 16, 2017 @10:01AM (#15713105)
The point OP is making is that in a startup, you don't do engineering for the sake of engineering. In the end, what matters is generating wealth.

Obviously, you want a sane legacy for the future but unless you already have hit that sweet product-market fit spot it is likely that you will spend a non-trivial amount of time optimizing (prematurely) for something that's in fact orthogonal to what you actually need.

The trick, as usual, is to balance engineering and business depending on which stage your at.

Re:Startup Mistakes: Choice of Datastore by vidarh on Thursday November 16, 2017 @11:45AM (#15714121)
This, I find, is one of the hardest things about hiring developers for a startup. Because on one hand you want people who knows how to do things right.

On the other hand, you want people who are prepared to accept that decisions that are insane in a stable business can be very much rational in a business where the cost of cash to fund large engineering changes is dropping at a crazy rate if you're successful, and doesn't matter if you're not (because you'll never get to it).

Of course it's good to avoid completely pointless mistakes. But it often matters far more to deliver.

Re:Startup Mistakes: Choice of Datastore by onion2k on Thursday November 16, 2017 @10:29AM (#15713380)
That something (if chose very wrong) will totally derail you progress and will cost a lot of fix it later.

A bad tech decision in a problem you can fix later, even if it's an expensive problem[1]. Having no customers isn't a problem you can fix later because your startup will fail and you'll have to close it. Consequently, at the start, you should work on getting paying customers over everything else, including prevaricating about what database to use. Anything is better than not making a decision.

All that said, "just use Postgres" as many people here are suggesting is actually pretty good advice.

[1] Unless it's so bad fixing it will sink your startup, in which case you have to live with it. That does happen but it has to be really bad.

Re:Startup Mistakes: Choice of Datastore by dang on Friday November 17, 2017 @12:43AM (#15719284)
Could you please not use uppercase for emphasis? It's basically yelling. Asking not to do that is one of the original HN guidelines: https://news.ycombinator.com/newsguidelines.html. Imagine this comment in all caps and I'm sure you'll understand why.
Re:Startup Mistakes: Choice of Datastore by notyourday on Thursday November 16, 2017 @10:11AM (#15713197)
It won't.
Re:Startup Mistakes: Choice of Datastore by emodendroket on Thursday November 16, 2017 @10:02AM (#15713112)
I don't disagree, but you'd probably rather pick "something" tried, true, and appropriate for most use cases, and that's RDBMS in this case.
Re:Startup Mistakes: Choice of Datastore by hashkb on Thursday November 16, 2017 @09:42AM (#15712930)
...as long as that something is Postgres.
Re:Startup Mistakes: Choice of Datastore by izolate on Thursday November 16, 2017 @09:27AM (#15712802)
Exactly. This article typifies premature optimisation. If MongoDB lets you prototype faster, then use it until you have enough paying customers to warrant a refactor.
Re:Startup Mistakes: Choice of Datastore by StavrosK on Thursday November 16, 2017 @09:33AM (#15712843)
I think you're greatly underestimating both the effort of changing a datastore from under a live application and the free time developers in a startup have.
Re:Startup Mistakes: Choice of Datastore by pandama on Thursday November 16, 2017 @09:46AM (#15712967)
Oh, wait. So you should actually think about what you're trying to build and what would work best for it? I just read this hilarious blog post that said the opposite. This is all too hard, I'll just use Postgres(R).
Re:Startup Mistakes: Choice of Datastore by StavrosK on Thursday November 16, 2017 @09:51AM (#15712999)
By "Said the oppposite" do you mean this part?:

> Think before you pick a database. If you insist on not thinking, pick PostgreSQL. Trust me.

Re:Startup Mistakes: Choice of Datastore by mannykannot on Thursday November 16, 2017 @10:11AM (#15713203)
Knuth's aphorism on premature optimization is perhaps the most misunderstood point in software development. It applies to code-level algorithmic optimization, not architectural decisions, where a bad choice can lead down a very long dead-end. You can recover if you recognize this in time, but you are better of if you do not have to. Obligatory rewriting is not what you want to be doing just as competitors take notice of your initial success.
Re:Startup Mistakes: Choice of Datastore by notacoward on Thursday November 16, 2017 @10:31AM (#15713401)
> If MongoDB lets you prototype faster

Does it? Even a prototype involves some iteration, which can be derailed by the kind of missing-schema problems that the OP mentions. Throw in a few operational problems, and your prototype isn't really any faster than something with better features and reliability. What's better for that first few lines of code on your laptop might not even get you to a prototype.