Tech stack:
Everything runs on AWS
Frontend: TypeScript + some Rust compiled to WASM, our own virtualized custom canvas magic
Backend: Rust, Apache Arrow
The biggest tech challenges... where to start.
1. As with any editor application, Undo/Redo is devilishly hard
2. A spreadsheet is modeled as a Directed Acyclic Graph where cells depend on other cells down to some root cells. There are a TON of fascinating graph theory algorithms you can bring to bear here. Some of the hardest problems I have ever worked on. My whiteboard looks like an insane person's with crazy arrows between cells and stuff.
3. Sometimes there really is no magic bullet and you just have to run benchmark, get flamegraph, chip away 3%, repeat, do that 100 times and you are very fast. Gotta grind.
I interviewed at a spreadsheet company (for a frontend role) and they asked, "how would you go about determining what cells need a border when a user clicks an individual cell, clicks a cell and selects multiple cells, clicks a cell next to an already selected cell." Fascinating problem and we talked about solutions for a little bit.
Noticed that you can't unselect a cell once it's selected? I'm on a Mac with Chrome (latest, no updates available).
Repo steps: 1. Select a few cells (⌘ + click) or an individual cell 2. Try unselecting (⌘ + click) those same cells clicked in #1 3. Cell is not unselected
Great interview question. Tons of nuance to drawing borders on adjacent cells, how to handle varying thickness, etc. Once you start looking closely, you notice the pixel differences between how this gets handled by various spreadsheeting tools.
Thanks for the report! This one's already on my list actually (selection negation & unique selection deduping) -- look for a fix soon.
One pivotal feature that is difficult to map onto 3P tools is our data table UI, which is a separate scrollable grid that floats on top of the main sheet. That, combined with the complexity of formula selection, inserting buttons into cells (header dropdowns, filter, sort), led us to decide that rolling our own solution for full control was the right choice.
I am the creator of DataGridXL (https://datagridxl.com), an Excel-like data grid component and it uses native scrolling. However, the document/sheet height/width is indeed limited by max div dimensions. Does your spreadsheet have a max?
For storage and orchestration we use S3 and Dynamo.
Yes exactly, we use CRDTs for multiplayer stuff.
Our frontend guy is brilliant and built a lot of awesome stuff at Airtable, but I can't speak to the craziness that is high-complexity high-performance frontend. But it's absolutely necessary to a good product too!
Can you share some of the books that can help with this? Which algorithms were used, how etc.
Thank in advance, I am using Row Zero since yesterday and loving it thoroughly.
For a project that will teach you literally everything there is to know about backend development:
1. Write a simple PUT/GET/DELETE REST API network layer, this should be a few hundred lines of code max depending on how much you choose to lean on libraries vs write it yourself
2. Then, write a simple in-memory blob store behind the PUT/GET/DELETE REST API. Just use a very simple hashmap of keys to data. You now have an in-memory S3 mock!
3. Now, rather than in-memory hashmap, start saving the files to disk. Write code that allows you to start and stop the process and recover all the data from disk. Now you have a persistent S3 mock.
4. Now, start finding the limits of this application. Write a program that calls your API with different usage patterns to load test it. Find the limits.
5. Now start optimizing. You can go as far as you want here. This optimization process was my whole career at AWS S3, and most of it at Row Zero.
6. If you want to really learn some advanced data structures and algorithms, stop using the filesystem directly, and write your own storage mechanism. Allocate 1 giant 16GB (or however large) file and store all of your blob data inside that 16GB "partition". You have to do all the serialization and retrieval etc yourself.
Your filesystem is already doing this for you when you store as files (look up how the ext4 filesystem works, inodes, pages, etc). But your filesystem is probably optimized for consumer use, not this blob-storage system use.
So you can implement your own much faster version if you want. If you want to really learn datastructures and algorithms, try to code a simple Log Structured Merge Tree. This will teach hashes, trees, bloom filters, serialization, deserialization, etc, all with high performance in mind.
What made you choose this tech stack? I'm especially wondering about Rust, was that something you had familiarity with before so just went with that? Did you evaluate a few different options and decided that would best fit this use-case?
(I have very minimal familiarity with Rust, I'm just wondering here.)