News for hackers. Stuff that matters.
Re:MondayDB: A new in-house data engine from monday.com by TekMol on Monday October 09, 2023 @06:48AM (#37818998)

    Until a while ago, when a user landed on their
    board, we threw all the board data right into
    the client (usually a web browser running on a
    desktop computer).

    the client is limited in its resources. Depending
    on the client device and board structure, it
    started to struggle and finally, crashed after a
    few thousand items (“table rows” in monday.com
    terminology). If we really pushed it, we could
    handle up to 20k items. Beyond that, it was game
    over.
This does not make sense. Why would a browser not be able to handle more than 20k table rows?

I have not tested it, but I would think juggling arrays with a few million entries should not make a browser sweat.

Re:MondayDB: A new in-house data engine from monday.com by kardianos on Monday October 09, 2023 @07:35AM (#37819283)
That is exactly what I came here to say as well. In my datatable custom query layer, I can pull down 1 million rows and view them instantly displaying them in a continuous list on the client, lazy load related table rows, and neither the web client nor the traditional web server on a traditional SQL Server, where I dynamically build the queries and can filter on any column, break a sweat. And the server is on 10 year old hardware that still runs spinning disks and the clients are often older lower powered computers. And that's with me not optimizing performance, other then thinking about it lightly up front. EDIT: Yes, the tables are sometimes medium wide, 20-50 columns buffered on the client on those rows.
Re:MondayDB: A new in-house data engine from monday.com by DrScientist on Monday October 09, 2023 @09:15AM (#37820149)
I think because their whole original logic was client side - to generate a filtered view would require loading all the data in the client first and then applying a filter.
Re:MondayDB: A new in-house data engine from monday.com by kardianos on Monday October 09, 2023 @04:38PM (#37825204)
Or just re-query the server with the new filters applied.
Re:MondayDB: A new in-house data engine from monday.com by robust-cactus on Monday October 09, 2023 @07:34AM (#37819272)
Works fine until you try to do it on a mobile device
Re:MondayDB: A new in-house data engine from monday.com by wahnfrieden on Monday October 09, 2023 @08:58AM (#37820001)
mobile will crash the page simply off memory use, esp older phones
Re:MondayDB: A new in-house data engine from monday.com by tmikaeld on Monday October 09, 2023 @06:55AM (#37819026)
You're right, a modern browser can handle it without issues using virtual scroll [0].

[0] https://shlomiassaf.github.io/ngrid/demos/virtual-scroll-per...

Re:MondayDB: A new in-house data engine from monday.com by eviluncle on Monday October 09, 2023 @07:12AM (#37819130)
Virtualzing (or windowing, as it is sometimes called) is definitely one way to tackle this problem. However, it's not a silver bullet. The more complex the UI/rendering is, the more optimizations you will need. The demo you link shows a very simple rendering with no complex user interactions, mouseover events, etc.
Re:MondayDB: A new in-house data engine from monday.com by white_dragon88 on Tuesday October 10, 2023 @05:18AM (#37829997)
It’s a solved problem. Things like aggrid allow you to use virtual scroll with custom components for your preferred framework, some simple efforts towards optimising the css and dom complexity would be more than enough.
Re:MondayDB: A new in-house data engine from monday.com by danjc on Monday October 09, 2023 @07:53AM (#37819438)
Agreed. Obviously you shouldn't render all of that, just keep it in memory and render the rows that are scrolled into view.