dacox
However, they are aggressively deprecating them (OpenAI is as well), and replacing with newer models. These newer models are all reasoning models, and importantly, only bear the flash name. They are not fast. And they are very expensive!
i tried to install with brew, but it told me my cli tools were "too out of date". Never seen that before! and also just upgraded.
Will try again tomorrow
I think I found a broken problem (or it’s worded strangely?) and I’m unable to progress beyond it
“Pick 2 that beat villain on board”
“QQ552”
In submitting Queen and 2 to make a full house but it just says
“Incorrect. Two pair on board. Win with a full house or ace kicker”
> Although it’s typically not viewed as a partisan board, the Biden administration had installed the entire committee.
After some degree of googling the history of ACIP I had not found any explanation and thought maybe someone here(who is actually American and maybe follows this kind of thing more closely?) would just know
Looks like there are actually some comments now that are more clarifying.
> Are you just asking questions to smokescreen for this executive power grab?
I’m just trying to understand the background. I get that this is a sensitive topic, but I’d ask that we keep things civil and give people the benefit of the doubt when they’re asking honest questions.
Apparently ACIP is very much not new. I am curious to the specifics of the prevous mass appointment, however.
Sometimes dozens of these studies get wrapped up and analyzed together, and we headlines that THING IS BAD with a hazard ratio of like 1.05 (we figured out smoking was bad with a hazard ratio that was like 3! - you need a really good signal when you are analyzing such low quality data)
We are starting to see models with longer and longer generation limits (gpt-4o-mini having 16k, the o1 models going up to 64k), as well as longer and longer context limits (often 128k, google offering a million).
I find it very unlikely they are actually training with inputs or outputs near these maximums.
If you want to convince yourself, do the attention calculation math for these sequence lengths.
I suspect the 4096 limits have been set as a "reasonable" limit for a myriad of reasons.
For example, consider a prompt sent to Llama 3.1 405B that uses 128k input tokens.
The KV cache will be 123GB. No matter how many GPUs you shard the model across, you are not fitting that KV cache in GPU memory (a H100 has 80GB)
The first phase is referred to as "prefill", where the input is processed to create the KV Cache.
After that phase, the "decode" phase is called auto-regressively. Each decode yields one new token.
This post on [Inference Memory Requirements](https://huggingface.co/blog/llama31#inference-memory-require...) is quite good.
These two phases have pretty different performance characteristics - prefill can really maximize GPU memory. For long contexts, its can be nigh impossible to do it all in a single pass - frameworks like vLLM use a technique called "chunked prefill".
The decode phase is compute intensive, but tends not to maximize GPU memory.
If you are serving these models, you really want to be able to have larger batch sizes during inference, which can only really come with scale - for a smaller app, you won't want to make the user wait that long.
So, long contexts only have to be processed _once_ per inference, which is basically a scheduling problem.
But the number of decode passes scales linearly with the output length. If it was unlimited, you could get some requests just _always_ present in an inference batch, reducing throughput for everyone.
This was a bit less true of older models like davinci - you really could basically just send them data (not sure how they were fine tuned under the hood). However, these models were less powerful in general.
For `gpt-3.5-turbo`, which definitely uses LoRA or something very similar, I have not witnesses this behaviour.
Another way to look at this is that when a LoRA is added to a sufficiently large foundation model like GPT-3, it doesn't really lose its "GPT-ness"
Because of this, there are "B.Seng" programs at some Canadian universities, as well as the standard "B.Sc" computer science program.
The degree was very new when I attended uni, so went for Comp sci intead as it seemed more "real". The B.Seng kids seemed to focus a lot more on industry things (classes on object oriented programming), which everyone picked up when doing internships anyways. They also had virtually no room for electives, whereas the CS calendar was stacked with very interesting electives which imo were vastly more useful in my career.
In practice, no one gives a hoot which degree you have, and we tend to just use the term SWeng regardless.
It honestly kinda feels like a bunch of crotchety old civil engineers trying to regulate an industry they're not a part of. I have _never_ seen a job require this degree.
After a year long foray into the world of PLC, I felt like I was programming in the dark ages.
I'm assuming its a bit better at very big plants/operations, but still.