encrux

Welch Labs has a beautiful visualization in their YouTube video about Vision Language Action Models, where the attention of the prompt can be traced to the exact origin pixels in the image from one of those matrices.

This is impossible, or it’s going to be soon. LinkedIn just introduced a „this is AI-slop“-button and it remains to be seen if it’s used for good, or used to train better AI-slop.

It’s a losing battle in my view.

That’s fair, and I don’t mind this. All I‘m proposing is the option for platforms to use this method on their own.

If FB or IG don’t want to use it, fair. I‘m hopeful that there’s gonna be competition if there’s a system with little friction.

In my opinion: the only way forward is zero-knowledge-proof authenticated social media.

We can’t have legitimate debate if we have to assume a few bad actors are cloning their voice by the thousands, poisoning debate.

If we can pin one account to a real person, we won’t get rid of LLM-content and misinformation, but at least we can hold them accountable.

I don’t see any other way past this other than regulation:

We need authenticated online spaces where your account is linked to a real, government issued ID. This way, moderating content becomes feasible again. Bans will hold value, since you can’t just create new accounts all the time. I predict people will gravitate towards authenticated markets (and social media).

I for one am sick and tired of second guessing every text I‘m reading, at least give me someone to hold accountable.

Back when GPT-2 was released, I tried figuring out how to fine tune it. I found a google notebooks template, scraped a bunch of data from r/ChangeMyMind and asked it to change my mind on different topics.

I was dumbfounded that it actually tried doing that. Obviously GPT-2 wasn’t great at it, but the writing was on the wall quite literally.

Unfortunately, I was too broke to invest in stocks, but I did pivot my career quite a bit.

For everyone in the EU: Copying and pasting sensitive data (like customer data) into AI tools is a violation of the GDPR, and potentially the AI act, which will be enforced soon.

These violations come with hefty fines.

A friend of mine forked the repo and tried it with BPE (Byte-Pair Encodings), and it did noticeably improve performance.

Late reply I guess, but "Claude builds a C compiler" was the first thing I thought about when I saw that sensationalized write-up of Mythos.

It sounds awesome, but once you look deeper, there's no meat on it. I'd love to be blown away, but even 12 days later there's just... nothing.

Anything that beats alexa-level intelligence on an edge-device is what I'd call useful as well, which shouldn't be too hard.

It's mind-boggling how bad current voice assistants sometimes are when you prompt them some fairly easy questions.

Not really on topic anymore, but…

I keep wondering when this discussion comes up… If I take an apple and paint it like an orange, it’s clearly not an orange. But how much would I have to change the apple for people to accept that it’s an orange?

This discussion keeps coming up in all aspects of society, like (artificial) diamonds and other, more polarizing topics.

It’s weird and it’s a weird discussion to have, since everyone seems to choose their own thresholds arbitrarily.

For YouTube, this already exists and I‘m using it. The extension is caller DeArrow and aims to reduce sensationalism via crowdsourcing, though I wouldn’t be surprised if top contributors are bots using LLMs.

AI in it‘s current phase, definitely. However, we‘ve been seeing the transformer architecture plateauing in the last couple of years. There are still improvements, but open source models are catching up.

I feel like at this point it’s an inevitability that given enough time, capable models will be cheap enough for everyone.

In a Sense, cats did actually. Not through direct contact, but by getting rid of mice and rats that infiltrate food storage.

I don’t miss multi week debugging sessions.

Having a tool that instantly searches through the first 50 pages of google and comes up with a reasonable solution is just speeding up what I would have done manually anyways.

Would I have learned more about (and around) the system I‘m building? Absolutely. I just prefer making my system work over anything else, so I don’t mind losing that.

I think this is actually the correct way to move forward.

We should be able to verify facts about people on the internet without compromising personal data. Giving platforms the ability to select specific demographics will, in my view, make the web a better place. It doesn’t just let us age restrict certain platforms, but can also make them more authentic. I think it’s really important to be able to know some things to be true about users, simply to avoid foreign election interference via trolling, preventing scams and so much more.

With this, enforcement would also be increasingly easy: Platforms just have to prove that they’re using this method, e.g. via audit.

Which, ironically, is written in rust

Very much depends on what you want to do.

The fact that a language model can „reason“ (in the LLM-slang meaning of the term) about 3D space is an interesting property.

If you give a text description of a scene and ask a robot to perform a peg in hole task, modern models are able to solve them fairly easily based on movement primitives. I implemented this on a UR robot arm back in 2023

The next logical step is, instead of having the model output text (code representing movement primitives), outputting tokens in action space. This is what models like pi0 are doing.

> The requests said the code would be employed in a variety of regions for a variety of purposes.

This is irrelevant if the only changing variable is the country. From a ML-perspective adding any unrelated country name shouldn’t matter at all.

Of course there is a chance they observed an inherent artifact, but that should be easily verified if you try this same exact experiment on other models.

Nothing about this was quick. 2015 was the first time we had an increase in authoritarianism in the public debate.

Project 2025 was announced in 2023.

> We should eliminate anonymity online.

for certain platforms. IMO platforms should be able to decide for themselves whether they want the option to have people verify themselves via ID or not.

It's the government's job to provide this service

Officials are usually elected because the people trust them (yes von der Leyen indirectly elected, but that's besides the point here). In geopolitical decisions for example, the people can't and shouldn't be able to know everything.

Possible, sure. In reality it's unlikely though.

Unless you still believe in the american dream, I'm pretty sure we can agree that the increase in housing prices makes it exceedingly difficult for young people to buy a house without a significant inheritance.

Surely at some point the market will just regulate itself and amazon will have to improve working conditions to keep operations running... Right? Isn't that how capitalism is supposed to work?

This seems surprisingly reasonable for my personal experience, so I'm gonna use it as a general rule now /s

I will argue that you don't even need real world application. What I do find really important is motivation of a problem.

Before even talking about math, teachers just need to explain where the problem came from, who was thinking about it and why (if it's known specifically, like cardano's 3rd degree polynomials, even better).

If there's one thing that gets me (and by my own observation, the high school students I tutored) motivated, it's this by far.