xpct

Thank you, finally someone comes in with the receipts. I'm going mad over the breathless hype in this thread.

Why? There's no 'make me smart' neuron.

By no means I'm saying this isn't impressive, but I think this is another case of LLMs being good at translating semantic meaning between different representations.

Obvious ones are spoken languages and code, but they can also natively interpret some simpler ciphers, hashes or simpler encodings, partly (I think) because it was in their training data.

As others have postulated, if they've been ingesting raw weights, naturally they learn how to translate semantic meaning to matrices.

> Even if it's currently not good for my ego

If you don't mind answering, how come? Is it specifically that you dislike how abusable your code was, or something else?

People are creating projects by paying money and not time, things that they wouldn't actually invest time into.

I suspect that this fad will die out when LLM capabilities are no longer as exciting.

I feel like we're arguing over semantics. What op quoted, referring to what I imagine were things like: confidently running wrong commands, answering something without looking it up, getting stuck on long scripts and not self-correcting all absolutely still happen, so the statement isn't incorrect.

I would also wager that OOP knows LLM limits quite well, and a more interesting discussion would be for why someone perceives LLMs as dumb as opposed to speculating whether they're aware of the frontier or not.

Why does that matter? The argument still stands, they are still dumb. We learned some ways to bypass the dumbness and automate some major workflows, and we still see them fail at random things.

> They clearly aren't writing for others to read their work.

I don't know, the prose and material is fine in my opinion. I do find the uncapitalized style a bit weird, but it's not something that stops me from reading it.

Think about it this way: if new open source models come restricted where they refuse to run tasks related to abliteration, then even if public utility programs exist, it will already add a lot of friction to how many people will be capable of doing it. Now, if unverified models become outlawed and will move to more obscure hosting sites, that will add friction as well.

Especially as people grow more dependent on agents doing all of their computer work.

> With a slow transmission rate (t = 10ms), A transmission is maintained at a distance of 700 cm away. With medium transmission rate (t = 5ms), A transmitted is maintained at a distance of 450 cm away and BER of 3%-4%. With a fast transmission rate (t = 1ms), a transmission is maintained at a distance of 300 cm away and BER of 2%-4%.

Coming from a non-EE background, this is the most interesting part to me. 10ms is an eternity in programming perf work, but it's already very constrained in how clear the signal is for reading the transmission. I just find it interesting that we want to do nothing to create a signal. Perhaps this makes it harder to exploit?

Well, not sure what I can add here, because this is exactly the type of semantic classification that I expect LLMs to do well, and can't really say why your agents failed to do it.

I've done similar work simply by giving a model a database of text, and asking it to do a unit of work for every passage. "Checking for spam" would be the model checking surrounding messages and user's history, likely an opinionated verdict, but would work. Difficulty comes in when those units of work become complex or have to work in tandem, which isn't the case here.

I think they're criticizing the system, not the people who participate in the system.

To the dismay of frontier labs, swapping models is actually very basic. What takes time is only acclimating to the capability changes since they won't all change linearly.

The Algorithm is what YouTube sells. This is only a static collection of videos with YouTube embeds.

Obviously they were jesting but it shouldn't be more than a few paragraphs.

This is a very niche forum that contains mostly textual data. I believe the full dataset is under 20GBs.

I'm not sure I follow, what is it exactly that your bots can't do? This is a relatively simple project implementation wise, since it only has to serve static content. LLMs are also OK at manually sifting through text passages to classify them semantically, if you ask them to.

Thank you for sharing. I saw some outrage discussions on social media concerning this, and it's unfortunate how far people are willing to spin this to feed their narrative.

Is the actor trained on world model's mistakes as well? If so, it should learn to force hallucinations that force it into an easier state.

+1. Would be fun to give Claude tasks and have it do them in the background, sometimes running into it occasionally. Judging by the article they struggle to do stuff so it'd probably feel balanced as it wouldn't accomplish much.

Tangential, but RTS is already a simple variant of AI + human. Attack clicking uses a dumb target selection and routing strategy on purpose so you have to micro. Now, if troops moved out of threat range and used smart targeting there'd be a lot less of 'game' in it. I believe some RTS players disliked the SC2 engine because the pathing was 'too intelligent' and they argued that it made surrounds and some other mechanics too easy.

Endgame of AI + human gameplay is delegating everything to the AI. I'm not sure how you would have to design a game where you only delegate parts of macro/micro, and still make the game interesting.

I played around with Nemotron 3 recently. It's OK, but makes an error every few sentences, at least on the data I worked with (standard vocab). I'm sure we'll get there, but I was pleasantly surprised by how I can recognize words better than it can.

If that is so, would it be entirely to decrease search space for layout algorithms? I believe there isn't anything in the fabrication process that couldn't model diagonals.

I could see that square packing is more simple, but by shifting elements per row I don't see why it couldn't propagate at odd angles.

Ah, this prompted a few questions from me. Perhaps someone with more hardware background could chime in.

They don't really show it here, but supposedly a Cerebras' chip is sub-divided into a 2D mesh of what they call PEs (cores). So data flow generally flows in two axis, and maintains consistent timings to its 4 neighboring cores. They show the layers flowing from left-to-right being streamed by PEs, but I wonder how true that ends up being: maybe the communications are easier to stream via diagonals, or some other shape that has better topology for data flow.

Then, even from die shots of CPUs and GPUs we often see that cores are tiled as squares, and those don't have the same cross-core connection and streaming requirement. Now, I assume that some parts of what we call a core can be moved around more freely while maintaining whatever power/heating/timing requirements we have, so perhaps it's easier to 'force' a square by moving those loose parts around. But why do we confine those to squares? Is super-core layout just something not worth investing time into, like how interconnects are timed, non-convex tilings, and distances to I/O?

And for streaming architectures like Cerebras, why aren't other types of tilings considered, something like a hexagon grid? This could have 3 inputs / 3 outputs, which could allow more complex data flow (?). Similarly, we could have 8 connections with longer diagonals in a square tiling, etc.

I guess, what are the tradeoffs in place such that so many chip designs seem to reuse square tilings both locally and globally?

I feel completely the opposite. Mind, I play games a fair bit, but I generally don't feel that excited about most, and I'd say a lot of them have very repetitive tropes. I'm generally in the market for 30h+ spent on a game, so that filters out a lot of the more trendy stuff.

I agree.. partly, and my problem is with enjoyment. I did some hand coding today and immediately got in the zone, something I'm still not able to do with LLMs. It most definitely took me probably ~2x longer than it would with an LLM, but it was much more enjoyable owning every decision behind the process, and having a full view of how the system worked.

I think this could be achievable with LLMs if they had a much higher token rate. That way you'd have less downtime and could work on the problem more aggressively. How it's designed to work now, offloading cognitively, doesn't feel very good, even when you 'ship' more.

I proudly stand in the 'interesting as long as I don't understand it' camp!

Hmm. I wonder if that can say anything about an individual's brain. That is, what type if people like the rhythmicity?

Anti-human sentiment has been rising in some online spaces in the last year. Soon, LLMs will start mimicking that sentiment, the same way they copied our 2000s morality and political opinions.

What does that even mean? They would have tried the models, noticed their limitations, and decide it's not AGI. Same as today.