enjeyw

The author hints at this but it seems like one issues is that while JEPA is good at distinguishing between unpredictable noise and predictable features, the model has no way of assigning importance to different predictable features.

So for a system where it’s very difficult to exactly reach the desired end state, the model needs to choose between (for example):

- reaching a relatively achievable scene where 95% of the features in the latent are correct, which includes stuff like visible enemies, Mario’s position on the screen etc

- reaching a far more difficult to access scene where there’s a bunch of differences in the actual level visuals, but theres a match on the latent for the tiny set of pixels in HUD that indicate you’ve hit the victory condition

We obviously know that it’s not good enough to reach an early scene that looks similar to the victory condition but isn’t. The model doesn’t.

In a sense, this is what the linear probe helps with - it allows us to re-weight the latent and say “actually, while the latent encodes many things about the world, the thing we really care about is the X position.”

I’d be curious what happened if rather than planning actions on cross entropy of a final scene, the model just tried to find the actions that maximize the predicted X value of the probe.

I think that’s a false dichotomy.

My Lazer Genesis Helmet is a MIPs and it’s the lightest helmet Lazer made at the time.

Much more breathable than my previous helmets too.

The story of the navigator in the photo is also worth a read [1]. Very reminiscent of Joseph Heller’s work.

1. https://www.rbogash.com/B-52/Carls_Letter.html

Yeah I did wonder myself if that tweet was an admission of guilt.

If I were a lawyer responsible for defending Trump in the Hague, I'd argue that the tweet was actually an abbreviated way of saying "If Iran does not comply, we will destroy all military assets, including but not limited to their ICBMs, Bridges, and Power Stations, such that we have total military dominance."

Now very obviously (to me at least) this was not the intent of the message, but I don't know whether you could prove that in a hypothetical war crimes trial.

What I find tricky to reason about here is that whether destroying infrastructure comes down to "whether the military advantage outweighs the impact to civilians", and as far as I can tell, there's no robust way to assess this.

Indeed, this seems to be what supporters of Trump are leaning on, as you can make the argument that _any_ bridge, or _any_ powerplant could hypothetically be used by the military, and that this conflict is sufficiently important for the livelihood of people in America/"The West" that doing anything that even slightly helps tips the odds is justifiable.

Ok I’ll bite…

37 miles?!? Why??

One of the big problems with Attention Mechanisms is that the Query needs to look over every single key, which for long contexts becomes very expensive.

A little side project I've been working on is to train a model that sits on top of the LLM, looks at each key and determines whether it's needed after a certain lifespan, and evicts it if possible (after the lifespan is expired). Still working on it, but my first pass test has a reduction of 90% of the keys!

https://github.com/enjeyw/smartkv

Oh I didn’t know that! Thanks for the clarification.

Well markets are evaluated on a number of different metrics depending on what you’re trying to determine.

If you want to go be pedantic about it and select one metric, markets are evaluated on their Brier Score or some other Proper Scoring Rule, not accuracy.

However, I prefer calibration as a high level way to explain prediction market performance to people, as it’s more intuitive.

It absolutely has been!

In general prediction markets can’t be “correct” or “incorrect” - for instance if a prediction market says there’s a 60% chance of an event occurring, and it doesn’t occur, was the market right or wrong? Well it’s hard to say - certainly the market said the event was more likely to occur than not, but only just, and who knows? Maybe the event _only just_ occurred, and very nearly didn’t!

So generally we say a prediction market is “correct” if it is “well calibrated”, which is to say that if we took all the events that the market said had a 60% chance of occurring, then approximately 60% percent of these events occurred (with the same holding true for all other percentages).

On this note, an interesting phenomenon that used to occur was “favorite-longshot bias”, where markets would consistently overestimate the likelihood of longshot events occurring - so events that the market predicted would occur 10% of the time would only occur 5% of the time. What’s fascinating is that once people realized that this bias exited, they began to exploit it by making bets against longshots, which had the effect of moving the market and removing the biases, making the markets well calibrated. It’s a pretty neat example of the efficient market hypothesis in action!

I used to share a somewhat similar sentiment.

I know one anecdote is not data, but his investment in BYD all the way back in 2008 does counter that viewpoint somewhat - his investment success in the BYD case isn’t from other investors following him in, it’s from him identifying BYD as a successful company far before any other major investors did.

Overly specific LLM research into KV cache eviction.

The vast majority of tokens in a sequence will be irrelevant to an attention mechanism outside of a very small window. Right now however we tend to either keep all cache values forever, or dump them all once they hit a certain age.

My theory is that you can train model to look at the key vectors and from that information alone work out how long to keep a the token in the cache for. Results so far look promising and it’s easy to add after the fact without retraining the core model itself.

I made a tool for this! It's an essay writing platform that tracks the edits and keystrokes rather than the final output, so its AI detection accuracy is _much_ higher than other tools: https://collie.ink/

I've been exploring this concept in LLMs for the last week or so, to see if I can RL train one into being inherently curious.

I haven't got any beyond my own working notes and some basic plots, but I've unceremoniously dumped them into a document here incase anyone else finds them interesting. If so I'd _love_ to chat with you. enjeyw @ google's email provder.

https://thealephengine.substack.com/p/67e3786f-8e84-41bd-888...

Yeah, I've often found that getting everyone pulling in the same direction is much more important than what the direction actually is.

If lots of smart people have thought about something and still disagree on the correct approach, pick one and move one.

I mean I kind of get it - overgeneralising (and projecting my own feelings), but I think HN favours introducing and discussing foundational concepts over things that are closer to memorising/wrote-learning. I think AI Math vs Leetcode broadly fits into that category.

My bad; Missed the sarcasm!

Yeah on re-read I think you’re right. Though who knows in this day and age!

I have no reasonable theory as to how Trump/RFK will be able to reveal credible information about Autism that wasn’t already available from public research papers.

About 10 years ago I became more aware that reducing my consumption of meat was good for the world. The was good for Beyond Meat’s prospects.

About 5 years ago I became more aware that reducing my consumption of ultra processed food was good for me. This was very bad for Beyond Meat’s prospects.

I suspect this experience generalizes.

I made a text editor that detects whether something has been AI generated by analyzing the keystrokes and edit history: www.collie.ink

Haven’t really told anyone about it so no one is using it, but I think it’s a good idea!

Lol! Technically they're not wrong about it being AI generated, just not at all in the way they meant.

Yeah this is something I'm a little worried about - right now it's not extremely difficult to just take an AI generated essay and then just tweak the essay until it passes.

My first pass approximation is to make the assessment of whether the essay is AI generated or not accessible only to teachers. I may need to also rate-limit the checks, so people can't brute force it to gather data on what passess.

Yeah it's a good call out. I think it's a (more) winnable battle though.

For both a keystroke based AI detector, and software designed to mimic human keystroke patterns, performance will be determined by the size of the dataset they have of genuine human keystroke patterns. The detector has an inherent leg-up in this, because it's constantly collecting more data through the use of the tool, whereas the mimic software doesn't have any built in loop to collect those inputs.

I'm building Collie (https://collie.ink/).

It's a tool to help teachers detect student assignments that have been written by AI. Unlike other solutions out there, it's an entire web-based text editor that analyses not just the final assignment, but all the keystrokes used during the writing process.

My theory is that analysing the final text only is a futile struggle - billions are being pumped into making LLM text look more human, so trying to make an assessment off final text alone is guess work at best.

I'm curious what folks think! Especially teachers, devs, and anyone navigating this space...

Side note to dang:

Please don’t delete this thread. Yes it’s getting pretty heated, but it’s by far the most rational discussion of this topic I’ve seen in a while. Plus I’ve learnt a few things, which tends to be a positive signal for quality

It feels to me that even ignoring copyright law, Kastel has a limited claim to the credit here.

Art director Alex Gotfryd came up with the concept of the Shark and the Swimmer, while Paul Bacon did the original drawing.

At this point what’s to distinguish Kastel’s painting of a shark and a swimmer from anyone else making a painting of a shark and a swimmer?