spott

OpenAI hasn’t had any problems getting impressive amounts of funding. So why ipo?

But I also doubt Nvidia is on the hook if OpenAI just no longer wants the compute. I bet they are only on the hook if OpenAI cannot pay for it (is insolvent in some way).

I also have to bring up that OpenAI has already spat out an inference chip that beats Nvidia on flops per watt. So they could potentially not need the compute while other ai companies do.

The reason Nvidia is comfortable making these deals is because if OpenAI can’t use the compute, someone else can.

Granted OpenAI going insolvent likely means a drop in the value of compute…

Claude or OpenAI.

I’m not sure what the value of perplexity is these days.

Anyways, I found it here: https://github.com/Modos-Labs/Glider

It would also wreck Japan.

3.1 percent. Just for context.

If I go to a bank, and I get a loan, then the terms of that loan allow the bank to do that.

If I sell a bond, then I set the terms, and I say that you can’t do that. You buy the bond understanding the terms.

China owns treasury bonds, they don’t have the ability to “bind up the borrower”. They can sell their bonds at market prices, but they can’t say “this debt is worthless, pay us now”. That’s not how bonds work.

That debt is all treasury securities bought on the open market.

They can’t demand the US pays them back early any more than you can.

https://github.com/spott/pi-task-compaction

I’m still playing with it, but it essentially has the model define a region, and mark it with begin_task, and end_task. End_task also requires a fairly hefty summery of the entire region.

It then pulls that entire region and replaces it with the summary. If the model wants it can actually look into the output that was pruned from the conversation, but in practice the summaries have been good enough it hasn’t needed to.

I’ve been using it with 5.6 sol, and even really long sessions have like 6% context used at the end.

Any organization is eventually controlled by people more interested in the organization than its mission. Or something to that effect.

(Accidentally attached this to the wrong post, meant for the post above this)

3 4 +

for example, would return 7.

I think the argument that pre-training gives a lib-left model is probably correct, so Grok, pre-post-training has a lib-left bend, then they try and post-train it.

I imagine it is hard to post-train a political bend, as it is wide ranging and touches so much. If they didn't give the resources to do it well, we could get something like this. I could also see people half-assing the training.

If the system prompt is being ignored half the time then the model is worse than I thought, cause that is pretty bad.

``` curl -fsSL https://x.ai/cli/install.sh | \ llm -s "Review this shell script and tell me what it does" ```

That isn't unauditable trust...

Token economics also are weird. If you design a fancy new frontend that for example uses a cheap model to parse a PDF into text that is fed into an expensive model, you will probably spend more money because you are on API payscale rather than the "max plan" payscale.

This is exceedingly unlikely, as training will only push one of them up for any individual sample. There are likely some pathological situations that could end up with that situation, maybe, but it is pretty unlikely in a general case.

Gmail isn't outstanding, search isn't outstanding, maps isn't outstanding.

They are all pretty par for the course. Google used to be outstanding... but I'm not sure of a single product they have that is outstanding (def: significantly better than the competition) anymore. On the other hand I rarely use any google products these days, so maybe I'm not the one to be judging.

I'm curious how they pre-trained it... I feel like it must have had audio/image output that they chopped off.

I wonder how hard it would be to add it back on.

FAIR did this 2 years ago now: https://arxiv.org/abs/2405.09818

I've been waiting for something like this to be released since then.

The annoying thing is that chameleon was multi-modal out based on the same principles, but this model is just inputs... (I'm curious how they did pre-training without having multi-modal outputs as well. I wonder if they just chopped them off rather than support image output).

Sshing as a regular user and then sudo to root works 95% of the time…

Before, you measured diffusion rate of oxygen and inferred oxygen concentration from that (the concentration outside the chamber is always greater than the concentration inside). Dirty membranes etc all changed the rate of diffusion, which caused issues.

After you measure oxygen concentration directly (the concentration inside and outside the chamber are always the same).