News for hackers. Stuff that matters.

xaskasdf

Show HN: The Simpsons Hit and Run Revamped

https://github.com/xaskasdf/The-Simpsons-Hit-and-Run

Hey folks, was browsing around for some videogame references and found a leaked Simpsons Hit n Run game source, so i took it and fixed some pretty bad issues with running the game on Windows 11 with a widescreen, including some broken audio stuff, memory leaks and security holes and forced letterbox video res.Also added some experiments that works, but they clearly need some love: DXR (Global...

( Read More... | no comments | 3 points | on HN )

Show HN: A LinkedIn Browser Gate Blocker

https://github.com/xaskasdf/linkedin-defense

Hi, I read the news here about LinkedIn accessing and collecting unauthorized data from users computers https://browsergate.eu/ and I'm kinda worried about digital identification, besides, Microsoft has a extensive questionable behaviour towards their users and their privacy, also because of their government involvement. Said this, here are a bunch of extensions that blocks and spams the...

( Read More... | 1 comment | on HN )
It didn't had any quality loss, since the PSNT as quantization it's mainly to convert the model over the console constraints (you can convert any model you want, even when i trained a model for this hw); it's q8 quantization, so quality loss is negligible for these sizes. For the speed, I will fix the Tok/sec count since now drops 0 always for showing measures

PS: Thank you! And forgot to mention PSNT also supports bitnet models, they work like crap tho

Show HN: Can I run a model language on a 26-year-old console?

https://github.com/xaskasdf/ps2-llm

Short answer: yes.The Emotion Engine has 32 MB of RAM total, so the trick is streaming weights from CD-ROM one matrix at a time during the forward pass — only activations, KV cache and embeddings live in RAM. This means models bigger than the RAM can still run, they just read more from disc.Had to build a custom quantized format (PSNT), hack endianness, write a tokenizer pipeline, and most of...

( Read More... | 14 comments | 4 points | on HN )
Actually since they have direct GDS it should perform really well on professional gpus
yeah, the mobo upgrade is something I gotta do anyway, so I'll cover that up more or less, the optane is something I didn't thought about
I wondered the same, but the rendering seems right, the output was almost instant. I'll recheck the token counter; anyway as you say, fast isn't practical. Actually I had to develop my own tiny model https://huggingface.co/xaskasdf/brandon-tiny-10m-instruct to fit something "usable", and it's basically a liar or disinformation machine haha
Ya know, here on the local market there are a bunch of optanes hanging around, I'll try to manage one to check if there's any improvement
Actually is purely bandwidth-bound, the major bottleneck of the whole process, for me in this case, is the B450 mobo I got that's only capable of pcie3 and 1x8 in the pcie lanes for gpu instead of 1x16; so I'm capped until I get an X570 maybe. I should get around twice or triple the tok speed with that upgrade alone
I thought about that, but idk if they allow me to modify the linux kernel and nvidia cuda kernel at all
Actually I can't go full tdp with a 650w PSU, I got to upgrade it asap
I updated the documentation to provide more info for the patching process, I added the patches themselves too and provided some risk info about the patches
I did it, but with different quantization compressions, It ran into quality issues, I will try to rerun with the same quants if that fixes the issue, but the most that looks unused, its being used by rotating layers that are being swapped by the cpu from the ram itself, that manages to keep layers warm, ready to use while inferencing and discarding already used ones
Did you even read anything? hahaha
Actually I'm thinking about buyin an AMD BC-250 that's bassically a ps5 with pcie factor format; and it's linux capable by default, maybe next month
This was the experiment itself https://github.com/xaskasdf/ps2-llm

The idea was basically to run a llm on a ps2, then I ran into some problems as the 32mb ram cap with 4mb vram cap; so I had to figure out a way to stream layers on the forward pass. Given that ps2 manages to give instructions directly to the vram that's capable of 32bit addresses, it gave an insane amount of tok/s, then I wondered if I could do the same on my puter

I got an m3, I will test it on metal and check how it goes
Actually this idea was fueled by those since I went to check if there was anything near to what I wanted to achieve, pretty useful tho
yeah, actually, I'm bottlenecked af since my mobo got pcie3 only :(
That would be nice to see. Actually I was thinking about getting another 3090 and a mobo upgrade since I'm bottlenecked by pcie3 to tryna run glm 4.7 or 5 at q4_k_m, it should be possible.
yeah, actually I wanted to see if this was possible at all. I managed to get around 3000 tokens/s on a ps2 with classic transformers, since the emotion engine is capable of 32 bit addresses, but it has like 32gb of ram. So I ran into the question of why was that fast and I couldn't get that speed even with small models, and the deal is that the instructions went right of the memory to the gpu and that's the main difference that does when a regular computer does inference: it has to request the instructions to the cpu every time. As I mentioned too, on professional cards you can avoid these problems naturally, since they got instructions precisely for this, but sadly I don't have 30k bucks to spare on a gpu :(

Show HN: Llama 3.1 70B on a single RTX 3090 via NVMe-to-GPU bypassing the CPU

https://github.com/xaskasdf/ntransformer

Hi everyone, I'm kinda involved in some retrogaming and with some experiments I ran into the following question: "It would be possible to run transformer models bypassing the cpu/ram, connecting the gpu to the nvme?"This is the result of that question itself and some weekend vibecoding (it has the linked library repository in the readme as well), it seems to work, even on consumer...

( Read More... | 105 comments | 2 points | on HN )