I absolutely agree. This is what makes technologies like graphql so compelling to me. Your data layer shouldn’t mimic the structure the presentation layer, and the presentation layer shouldn’t be tightly coupled to the data layer. And more importantly, data will typically be used by a multitude of apps, all with different purposes. There’s nothing worse than starting a new project that relies on data that was structured to facilitate an old obsolete interface.

Linus Torvalds said "Bad programmers worry about the code. Good programmers worry about data structures and their relationships.".

Here's a really interesting discussion about this quote and the idea behind the above: https://softwareengineering.stackexchange.com/questions/1631...

As a backend dev myself, I totally agree with it, but I think when we're developing full blown apps we should have the end user in mind and focus on their experience using our app more than on adopting data-driven UI design.

I generally do like the data-first approach when I have a good idea of what we're building. One place I’ve found it useful to start when you don't have the views planned ahead of time is to mock up the main data management views with handwritten sample JSON data needed to render the functionality desired. Then make the DB schema to provide that data.

If you want to get fancy, you can use Mirage JS [1] to emulate network requests with mock data.

[1] https://miragejs.com/

That's also my way.

First design the data. This will tell you how this part of the solution integrates with what everybody else is doing, and will make very obvious some problems that otherwise you may only discover after the system is mature into usage.

Then design the UX and users processes. Programmers get to design only half of those, but both have to be set at the same time.

Then you can do everything else.

And yes, letting an ORM dictate migrations guides people into a completely wrong sequence.

What goes in the database? The whole universe? What do you leave out? What do you denormalize and index? These questions are answered by the frontend.

You store the entire universe or whatever reporting requirement demand. You define the database structure based on the requirements of your business logic.

The frontend requirements can help define what api's are required but if you treat your database as dumping tables you limit what your data can do.

You add can add and remove indexes at any point. If you made a bad decision or no decision you would add those when more information is available.

There's no silver bullet I guess but I think the trap that one could fall into here is to think of "the" frontend. There's a bunch of additional consumers of the data model even for a single web-app. Auditing, security, reporting (prior to moving to a separate reporting database), internal tools, possible future mobile or desktop applications, etc. I have found the safest approach is to enforce invariants in the database rather than assume the application code will do the right thing. But as you highlight the schema can be refined by the needs of the front-end, especially where indexing is concerned.

I agree on the schema being the source of canonical truth. I also agree on ORMs - we use jdbi mainly. But I find it hard to reason about everything up front. There are generally too many unknown unknowns that emerge when the feature is more fully fledged out.

I tend to agree, and usually go straight for the schema when I start on a project, because the data structure is the model of the real world problem.

With regard to migrations though, I tend to disagree. Code is very easy to manage in version control, so the migration files are merely the data structure represented in code. I love migrations and they've vastly changed the way I work.

You’re both right.

The data model is the thing that stays more invariant in an app, and you can start with that:

https://github.com/Qbix/architecture/wiki/Internet-2.0

But a centralized database is not the future. Client-first web apps are:

https://qbix.com/blog/2020/01/02/the-case-for-building-clien...

So your data will be stored client-side and synced, with Dat or MaidSAFE or IPFS etc.

> For me it all starts with the database, always.

I agree, though I generally reduce this question to, "what is the shape of my data?".

I was just thinking something like this as well. The best term for it is 'schema' but that moniker seems so, ill-fitting. For lack of a better term the 'schema' isn't really designed, it's discovered.

Thats how I know I'm on the right path. I start to organize the data and new things that appear fit naturally into what I've already discovered.

This is the epitome of organic design because you are simply the conduit between the truth and a schema definition. If you force yourself to not be clever and just interpret what you've found, your database model will be solid.

100% agree, though a good ORM handling migrations can be nice as it documents how the schema has changed. Just be sure there is no schema logic that exists only in the ORM and not at the database layer itself.

> Of course you need to know roughly what the data will be used for ...

Can you talk about how you figure this part out? As you imply, this seems like it is your actual first step, not designing the database.

Sure, I guess by first step I was referring to the first thing when I sit down to start coding.

There's all the stuff that comes before actually doing that, generally, though not always, the feature is driven by some front-end requirement and may have accompanying product specification and possibly even UX designs. But I try to take those and think about what the domain concept being addressed is, rather than the shape of the data the front-end requires. Almost always the front-end and requirements will change through iteration so I find that purely mapping the FE concepts directly to the data-structure is less robust than taking a step back and thinking about the actual data without consideration for the proposed front-end design. Though the front-end can still inform aspects of the database at this stage it should only provide minor optimizations to the schema through indexes or normalization choices.

I suppose in part it's taking a domain-driven design approach and making the first step encoding the domain in the database. It has been my experience that many headaches in applications I encounter could have been avoided if the constraints had been enforced against the data at the persistence level rather than assuming the application level code will handle it correctly.

That makes sense. Thanks for the response. It seems like some people in this comment section are coming from your perspective, where the feature has been somewhat defined already, while others are talking about the entire process from the initial idea or problem to working software.

I definitely agree that when possible it is very useful to enforce important constraints in the database.

You would (should?) have a well defined schema with MongoDB. I know some people see this as one of the values of it, but personally it's not why I use it.

If I understand it correctly this was a newer feature, from what I remember when it was first released it was schemaless?

A schema is just a definition and a set of constraints; in a "schemaless" database, you implement the schema in your application.

the linux command UI (for actions/services) and spreadsheet UI (for data) are the best interfaces that have lasted for over 40 years.

Anything else is ephermal.

History is littered with millions of enterprise software that can't beat the simplicity of a spreadsheet or the flexibility and efficiency of a unix command line