melozo

Question to math people - are you actually able to intuitively understand and derive meaning from some of the formulae? How did you get to that point?

As an example - "The optimal linear memory store" talks about whitening the key by undoing key covariance through transposing and multiplying the "K" matrix, then inverting it, then multiplying it with K again. There must be some beautiful mathematical relationship there, but I'm looking at this completely confused as to how such a thing could effectively scale the key-space such that covariant keys are widened and avoid/reduce overlap.

I took linear algebra over a decade ago in college but obviously don't remember it. I'm just surprised that such semantically interesting outcomes can come of these things, but reading this for me as someone without that deep and second-nature understanding makes the math impenetrable for me. Even if I understand what gets multiplied by what and the theoretical basis for each operation, being able to read this and make sense of the math alone is impossible for me.

All of this to try and learn something new

I think you’re underrating the sheer amount of iteration that occurs. SaaS has requirements and then implementation- it’s a product that has a target it hits. Yes, churn happens, but the point is you’re working to a specific goal.

Games and any creative work isn’t like that. There’s constant iteration as new ideas come up or developers learn in play testing that certain parts of the game work better than others, and so they realign things continually and keep refining. I’m not a game dev btw, but blogs from devs show a continual cycle of refinement and feature creep.

Huh? Open source is a quality of the software, not specific to the hardware used to run the model. The demand is that model weights are openly available for anyone to run and fine tune without restriction. Has nothing to do with the hardware it runs on.

I worked at Amazon and I do think that more than anything we were overhired with little meaningful work. A lot of compliance goal chasing. Not that it’s performative to the top brass, but the work was very little and not usually very technical.

Yeah I’m struggling to understand why the same header field would be used for git options in the first place. Why ever allow users to modify that specific header?

Ironically Square isn’t good at anything in particular. Square fails to have a good point of sale (Toast wins) and fails for online store fronts (Shopify wins). Square is in a position where they spread themselves thin and aren’t at the top of anything in particular.

I’m not sure how controversial this is - but 100% code coverage is almost always a waste of time, paid both immediately and long term, for certain languages. Go, for example, requires explicit error handling, but the way errors are handled are usually plain and homogenous. Adding unit testing everywhere creates a phenomenal amount of test code that can become 3x the size of the source, and certain changes (like interface changes) can require updates to all tests, especially if mocking is used.

Obviously with AI maybe those issues I have go away. But I really don’t like letting the AI modify tests without meticulously manually reviewing those changes, because in my experience the AI cares more about getting the tests passing than it does about ensuring semantic correctness. For as long as tests are manually maintained I will continue keeping them as few as necessary while maintaining what I view as an acceptable amount of coverage.

You’re right about that. I guess what I mean is, how long will people be enthusiastic about AWS and its ability to innovate. But AWS undeniably has some really strong product offerings - it’s just that their pace of innovation has slowed. Their managed solutions for open source applications is generally good, but some of their bespoke alternatives are lacking over the last few years (ecs kinesis code* tools) - it wasn’t always like that (sqs ddb s3 ec2).

Sure, but it’s not reasonable that internal collaboration platforms built for ticketing engineers about outages doesn’t work during the outage. That would be something worth making multi-region at a minimum.

Even internal Amazon tooling is impacted greatly - including the internal ticketing platform which is making collaboration impossible during the outage. Amazon is incapable of building multi-region services internally. The Amazon retail site seems available, but I’m curious if it’s even using native AWS or is still on the old internal compute platform. Makes me wonder how much juice this company has left.

I worked at <large tech company everyone hates> and we had this exact architecture. The senior engineer (who went to Stanford and was proud of it) designed this system. I had long arguments with him about how this system wouldn't scale when we launched it, and used real production data to prove that it would happen. No one listened to me, it launched and imploded within a few weeks, and soon after I transferred off the team. The worst part is, that senior engineer was eventually promoted and that system was passed off to an entirely new team to battle with. That entire system was terribly designed, and I think you can guess why.