News for hackers. Stuff that matters.
Re:Ask HN: What do I need to learn how to write a scalable app? by juangacovas on Tuesday June 02, 2020 @09:31AM (#23390789)
Where exactly is your bottleneck right now? Database queries? I mean, where those >300ms go, foreach request or just some of them?
Re:Ask HN: What do I need to learn how to write a scalable app? by stonecharioteer on Tuesday June 02, 2020 @09:40AM (#23390872)
Definitely the database. How would I begin speeding that up?
Re:Ask HN: What do I need to learn how to write a scalable app? by juangacovas on Tuesday June 02, 2020 @11:05AM (#23391784)
Two million rows in tables are not a problem even for old databases (I'm thinking of mysql 5.1)... Are you confident on your indexes, normalization, etc?
Re:Ask HN: What do I need to learn how to write a scalable app? by stonecharioteer on Tuesday June 02, 2020 @01:42PM (#23393738)
I definitely will look at the indices. I have several joins. Perhaps if I explain what I'm trying to build.

I am building a git analytics dashboard. I want to help my organization analyse developer productivity from bitbucket. So I am pulling in all the git commit logs, the diffs for all projects my org within the company has. We have 1500 repos across 20 projects. I am building dashboards atop of this. The data will grow daily. I would like to showcase project health in terms of commits, pull requests and whether there are duplications in work (surprisingly we have this. Actual repos being copies from here to there btw, insane shit). In doing this I realized I am struggling to scale it. The UI Takes 2 minutes to load. It is abysmal. The org is impressed but I am not, and it is for my own learning.

Re:Ask HN: What do I need to learn how to write a scalable app? by LarryMade2 on Wednesday June 03, 2020 @10:45AM (#23403882)
Optimise your queries as well, complex queries with multiple many-to-many joins will add up quick to tons of rows to filter. Look at separating queries; to keep the overall in a single query, the db may have to churn through tens of thousands of rows. Where if you do a first select than a second based on the result of the first, you can bring it down to just hundreds or tens - with a quicker response than just one query.
Re:Ask HN: What do I need to learn how to write a scalable app? by grizzles on Wednesday June 03, 2020 @07:19AM (#23402144)
That's the problem. Try denormalizing your data. Use indexes more. If you can, pre fetch & cache your data.