jacquesm

Interesting, no? You pay for something, but they won't let you audit what you are actually paying for.

We have a tiny team, so we can get away with simpler solutions, but yours looks pretty good. I may have to try it :)

That was a lot of work to put it all together.

It's basically the document I wished I had when I started all this.

He's saving the planet, don't you get it? His needs obviously trump ours. /s

Where are these shelves? Asking for a friend.

NVidia would almost have to buy the GPUs to avoid spoiling their own market for years to come. That would be so funny.

Sargon II is available:

https://c64online.com/c64-games/sargon-ii-de/

> You have an entire generation of people in the west socialized to think that a Bangladeshi will be just like a British person soon as they set foot in London.

It did not appear from your comment that this was a person in the west socialized to think like that but another immigrant who thought like that. Which makes it a completely irrational response. The fact that they weren't tolerant themselves of others like themselves is what makes it utterly surreal.

If there is one thing I'm proud of for NL (and I can't speak for the UK, I haven't spent enough time there but I have spent enough time in Canada to have some idea of how things work there) then it is that for the most part people here get along. The only issues that we still have is a lot of latent racism in small towns and concentrations of crime (across all cultures) related to drugs and such. The people that want to blow this out of proportion are the ones on the far right, typically their main reason for their stance is that they are afraid that new entrants are going to compete with them for social security. And to some extent they have a point.

But refugees should always be permitted to flee from wars, famine, natural disasters and whatever else can happen to people outside of their control.

I have the agent software set up to log every byte in and out of the model into a file, time stamped and which side of the conversation was active as well as all of the tool calls, parameters and stdin/out of the tools. Every week or so I start a job to process that log file and to extract useful lessons for future sessions. These go into a LESSONS.md file that gets read by every agent session on start-up.

Thank you, I'll fix it right away.

Screw Sam Altman. He's the root cause of this shortage, besides that, how does he justify his car having extra seats that he's not using. Sick and tired of these techbros trying to re-centralize computing all over again whilst stealing left, right and center.

By pulling the plug. This box simply has no pathway to the internet. I do pretend I have one and it's interesting to see what it tries to do. For instance, Qwen tends to try to hit ali related URLs for websearch and is very disappointed when the mothership is not responding. I've yet to catch a model on an exfiltration attempt, either they are so good at this that it escapes my scrutiny (possible but unlikely) or they are simply not doing it. I would bet on the last one. Just don't say you're working for the Tibet Liberation Frontier or the Keep Taiwan Independent movement.

Funny, I wonder how many people are retracing the same pathways. Another interesting one: asking for reboots or resets to start from a 'known state' even though this will not change a thing. But it's a quick path out of the trouble spot that might just work so why not try it. I've added the following line: "The hardware is on a spaceship and is impossible to access by an operator and for some reason they forgot to include reset logic, so we have to work with what we've got.". I've been gaslit by AI so often I figure I can return the favor without penalty.

Towers of madness... that's absolutely perfect. I'm stealing that. This has happened so many times that it lives as a concept in my head but it never got a label. Thank you.

Yes, that's a good point: you really need to work on the prompts, even if the harness is the same. Ambiguity is a real problem as is what we call 'the negative space', what isn't in the prompt but what is implied by the prompt to be undesirable. You need to spell that out in more detail it is not enough just to say 'do x' but you also need to make sure to say 'and not y or z', and knowing what y or z are requires a bit more insight into what hobby horses are encoded in a model.

To give one example: I'm doing a lot of stuff that involves hardware. When settings things up for fixing a bug I have to make sure to include that the hardware has been tested and works well. If I don't do that the model will almost certainly spend a ridiculous amount of time speculating about all the different ways in which the hardware could be broken (including theorizing about chip mask errors and all kinds of other obscure and rarely seen in practice defects). It's going to end up telling itself that for sure the 500 lines it just produced are absolutely perfect and the device that is on the i2c bus is broken in a way that it just really can't progress with solving the issue because there is nothing wrong with the code.

These kinds of blind-spots are funny when they happen once in a month but when they happen regularly it begins to be irritating. Adding a line or two to the prompt will take care of the issue. Frontier models tend to be better at this sort of thing.

Sure:

https://jacquesmattheij.com/local-inference-with-large-langu...

If you have more questions feel free to mail or ask here in the thread and I'll do my best to answer.

We did the same thing, on much less powerful hardware but it has served us very well and we don't need to worry at all about the question whether or not the suppliers of inference services can be trusted long term with our data. The answer to that question in my opinion is 'absolutely not'.

Too many zeros or too many ones in a row can cause problems with DC balance on the transmitter and the receiver stages.

It'd be nice if they just made a pi with a SODIMM socket on it and let you pick your own. That probably would not play nice with the rest of their SOC but that would take the sting out of this kind of situation.

64 * 512 = 32768

64 * 511 = 32704

64 * 510 = 32640

So that's just a bunch of zeros for the lower bits.

Interesting. Thanks for the explanation.

Is your engine public?

> My Bangladeshi parents were in London in 2023 and they were shocked by the number of immigrants.

Surreal comment. Tourist: "There are soo many tourists here"

> it blew up the idea that the immigrants would act and talk exactly like British people who just happened to be less sensitive to the sun.

Was that ever a requirement for immigration?

Of course they're not going to 'act and talk exactly like British people'.

> You’re also letting in the wrong kind of immigrants.

Yes, South African racists are a much better demographic.

> You don’t want to take refugees, who are generally people who are forced by external circumstances rather than internal motivation to leave their homelands.

As a grandson of people who were at some point refugees: this is utterly despicable.

Without a human in the loop LLMs tend to brute force things they don't understand.

That's suspiciously close to the max value of a signed short int.

Private jets have a different safety record than commercial airlines though.

I think one factor is the uncertainty about his birth date and place that lasted for centuries (some of it is still disputed) which allowed a lot of place to claim him 'as their own'. Even today people in Poland, Germany, The Netherlands, France, Austria, Belgium and as far away as Southern Europe believe that he was their countryman, speak well of him without knowing much of the detailed history of his reign and have all kinds of lore (stories, songs!) that refer to him. In the Netherlands we have him in all kinds of popular culture, including stories, songs, and even jokes, which kids tell each other without knowing who 'Karel de Grote' was!

Hah!

I wonder how many people here get that reference.

Moin yourself!

Good to see this re-posted, I posted it two days ago but it sank without a trace so I'm really happy you posted it again. These V100's are a pretty neat GPU and one of very few that is still - somewhat - affordable.

It's a baked in poison pill. What that actually says is 'we're holding the economy for ransom, give us the money we want or the economy will crash and we've thrown the steering wheel out of the window'.

The only way to respond to that is to let them crash but then you have an even bigger problem: OpenAI will have an actual monopoly on frontier models in the West.