snats

did a pretty similar thing last month for the text rendering library last month.

trained and made a viz for the model and then made it displace text.

should probably do a proper write-up:https://x.com/i/status/2038367016969724259

you can also have removed layers of models and keep the same score in benchmarks [1].

i feel that sometimes a lot of the layers might just be redundant and are not fully needed once a model is trained.

[1] https://snats.xyz/pages/articles/pruningg.html

personal website: snats.xyz weblog: weblog.snats.xyz

I also did a couple of experiments with pruning LLMs[1] using genetic algorithms and you can just keep removing a surprising amount of layers in big models before they start to have a stroke.

[1]https://snats.xyz/pages/articles/pruningg.html

i get it. i want one of those. the problem is that most cellphones are not actual cellphones, they are entertainment machines. they are a pocket tv / social media feed place. most usage for my normal friends is for that.

I am working on the https://moviemovie.club/about, it's a tiny website about film review.

It works like a run club, where you have to make a review first to see other people's reviews.

I am currently implementing watchlists, comments and a mural to make it feel a bit less lonely. Right now I like the UI but it feels to lonely.

Not an insider but imo the work on diffusion language models like LLaDA is really exciting. It's pretty obvious that LLMs are good but they are pretty slow. And in a world where people want agents you want a lot of the time something that might not be that smart but is capable of going really fast + searches fast. You only need to solve search in a specific domain for most agents. You don't need to solve the entire knowledge of human history in a single set of weights

It's pretty funny to test in-distribution for AI models. But they fail horribly once you push them a bit[1].

I recently made LLMs play Minesweeper and ALL LLMs that I tested had a pretty bad win to loose ratio. Like the only model that won more than 3 times was R1 (mind you there were 50 games).

[1] https://snats.xyz/pages/articles/minesweeper_bench.html

yup, if i went to do a PhD interpretability is the only interesting subject for academia IMO right now

It's more of a distilled model, not a fair 1:1 comparison

I use .XYZ because it was pretty cheap when I bought it

I post about ML/DL and SWE at https://bsky.app/profile/snats.xyz

This paper is not comparing against MOLMO or Qwen, so I would take it with a grain of salt

I made some image generation with CLIP and Evolutionary algorithms not so long ago[1] and the results had a bit more life than I was expecting. It is still a contested area of research and you have some cool stuff like CLIPDraw[2] where they use gradient decent to approximate a vector to the embeddings of CLIP.

[1] https://snats.xyz/pages/articles/optimizing_images.html [2] https://arxiv.org/pdf/2106.14843