mikepk

Given yesterday, May the 4th. Ended up with a lightly unhinged Star Wars/data infrastructure rant about centralized warehouses, and dark data.

I think it was funnier in my head than on the page (and maybe I've thought about data architectures for too long).

Disclosure - used AI for brainstorming and some of the phrases (and the death star image).

Made me think, is there an opportunity to build a management layer for this? Handle subscribing and cancelling automatically when you want to watch certain things? Would probably be blocked pretty fast but amusing to think about.

Data lakes promised schema-on-read flexibility but became swamps. The industry responded by turning them into databases again, defeating the original purpose. We built Matterbeam differently: immutable storage with inferred intermediate schemas that translate rapidly into any structure you need. When materializations aren't right, replay them. The original vision, actually operationalized.

We gave our first employees restricted stock agreements instead of ISO options, with the exact same legal terms and vesting schedule I got as founder. 2-6% equity grants, six-year vesting, hiring bonuses to cover exercise costs and taxes. Values have to cost something, or they're just words on a website.

Founder here - happy to answer any questions!

Founder and CEO if you want to ask me any questions! :)

This is a little triggering :) Reminds me of all the promise back in 2005 when I built my first startup Grazr. It was: - a widget that was a mini RSS reader that let visitors to your site read the RSS feeds you subscribed to on your site - a way to share your collection of RSS feeds dynamically - a way to copy / remix those collections - a way to subscribe to those lists dynamically (if they had a dynamic blogroll or whatever) - a processing and filtering engine to allow merging collections of feeds together into a single stream - Javascript on the server (in 2005 :) ) run using embedded script tags in the OPML / XML blogrolls to create even more dynamic blogs

The net effect was you could make your own news feeds / timelines and use code to control how they were filtered / combined / etc... It was crazy powerful (for 2005) and I still miss it _today_ since it had the dynamism of the news feed, some of the social aspect, and total control since there was no algorithm other than your and the people's who's list you subscribed to curation and any code you ran against it.

Not a lot left from those long ago days but I did find one slightly-cringy video: https://www.youtube.com/watch?v=45DSrU23sPI

:)

It’s not ftl but there are ideas and projects to do interstellar travel in a lifetime. https://en.wikipedia.org/wiki/Breakthrough_Starshot

So many memories. That was a big part of my first startup in 2005. Grazr was the “next level of rss”. Unfortunately the first level never quite took off.

https://techcrunch.com/2006/09/18/grazr-10-blasts-off-into-t...

I've toyed with OSS'ing it or releasing the code. It would be a non-trivial amount of time and energy to package it in a way that people could host it. Gamescape was a hobby thing so I used some different architectural patterns I'd been wanting to experiment and play with that aren't totally standard.

Very nice! :) I also had a similar itch a while back and built https://gamescape.app (during the pandemic). Different focus -- all about shared maps and tokens rather than a "full" VTT. Plan is for it to be free forever. Can't work on it anymore since I started a startup that eats 100% of my time, but totally usable (my group uses it every week :) ).

I posted to get ideas and maybe help other startups. We dipped a toe into posting on Reddit. Targeted community, sharing a white paper, asking if people wanted to share their pain with us (we're still trying to learn). Got modded to oblivion. I thought we ticked the right boxes, interesting info (IMHO), being transparent about who we are.

I also wrote a small free application for drawing (quick) maps, real-time collab, moving tokens around etc. It has a small but happy user base :)

GameScape (https://gamescape.app)

Basically I wanted something to approximate pen, grid paper and physical tokens during the pandemic when I was playing with some old high school friends remotely. Roll20 was just too clunky.

I’ve got a new startup now (unrelated) so I haven’t had much time to work on it. If anyone wants to work on TTRPG tools (I’ve considered OSS this, or maybe even paying for dev) let me know!

We need to re-think how to make data _useful_. The fact that the value hasn't materialized after decades of attempts, billions of dollars, and lots of tools and technology points to the fact that our core assumptions and patterns are wrong.

This post doesn't go far enough. It challenges the assumption that everyone's data is "big data" or that every company's data will eventually grow to be big data. I agree that "big data" was the wrong model. We also need to challenge that all data should be stored in one place (warehouse, lake, lakehouse). We need to challenge that one tool can be used for every data need. We need to challenge how we build systems both from a technology and people standpoint. We need to embrace that the problems and needs of companies _are always changing_.

We are living with conceptual inertia. Many of our patterns are an evolution from the 70's and 80's and the first relational databases. It's time to rethink how we "do data" from first principles.

I'm tempted to write a blog post... I bristle a little when microservices are described as "best practice". Monolith vs microservice is really about _people_ and _organizations_. Monoliths make sense in some contexts and microservices in others, but the deciding factor is really the size of the team and number of people working on different functional contexts.

The best analog I can come up with is monoliths in larger organizations are like a manifestation of Amdahl's law. The overhead of communication and synchronization reduces your development throughput. Each additional person does not add one persons worth of throughput when you cross a critical individual count threshold (mythical man month and all that).

I'm not describing this clearly so I should probably actually commit to writing out my thoughts on this in a post describing my experience with this.

Wow memories. My first startup, grazr.com created a feed processing system that could achieve similar (imho superior) results to pipes and worked in a different way (https://mikepk.com/2007/03/grazrscript-v12-beta/). The idea was to use actual code (rather than graphical drag-drop) to write feed processing and transformation applications.

We used server side javascript (before Node, I like to think we were ahead of our time, our own fork of Spidermonkey with network and other server code!). We used OPML as a container format with <grazr:script> tags in them analogous to html <script> tags. The "pipes" in our system were much more like unix pipes, where the output of one script could be the input to another. In the running script you could access the feed with a customized "Feed Dom" with items and do manipulations.

With grazr's real-time freshness, feed normalization and database it worked amazingly well. We never were able to get people to understand it (it was super nerdy). I built some pretty amazing apps with it, unfortunately none of it exists anymore :).

Yahoo pipes seemed like the sign of the end for Grazr though. Since we had a widget and feed display tech. a _lot_ of people wired their pipes to grazr's widgets for display and publishing. We saw an initial "burst" on pipes announce and then saw the yahoo pipes usage drop pretty fast and never really recover. As a proxy, it looked like there wasn't enough "there" there to continue to pursue our programming environment.

I still miss it though, it was fun to work on.

The one that really blows my mind is Vannevar Bush -- July 1945 - "As we may think" - https://www.theatlantic.com/magazine/archive/1945/07/as-we-m... If you can look past the way he describes technologies of the day, there's an amazing amount of foresight in how access to information, its modalities and interconnectedness would become critically important.

"I admit that mathematical science is a good thing. But excessive devotion to it is a bad thing. If we evolved a race of Isaac Newtons, that would not be progress. For the price Newton had to pay for being a supreme intellect was that he was incapable of friendship, love, fatherhood, and many other desirable things. As a man he was a failure; as a monster he was superb." -- Aldus Huxley