froobius

Isn't being "underweight" generally shown to reduce the speed of aging

Counterpoint: vertical tabs are way better

these are good / accurate, but what's the error-rate? assuming these have been cherry picked, and he made more predictions

I think it's fun. You'll also get the IKEA effect helping you to appreciate the book more

https://en.wikipedia.org/wiki/IKEA_effect

> context a session moves grows with roughly the square of its length

Readme doesn't mention that LLM providers do context caching — your costs aren't quadratic unless you frequently invalidate the cache.

I'm starting to wonder if any of this is better than local newspapers, where there's an editor with oversight over all content...

You can run a lot of these on e.g. M1 max 64gb

Erm that's higher than the healthy BMI range, so not "fairly light". >25 is considered overweight.

On the other hand, taking backwards compatibility so seriously is a big part of the massive success of Python

It's very clear that the consumer is getting a worse experience than what is technically possible. There is no good phone-slash-laptop, purely because it's less profitable than locking down the devices and selling them separately.

I couldn't see any citations or references in that video or its description. It presents it as him solving the problem himself, but I'm sure other people have written about solving the Game of Life in reverse with SAT solvers prior to this...

Edit: here's a paper on it from 2006, https://link.springer.com/chapter/10.1007/978-3-540-76928-6_...

And the news just now is that the chair of the OBR has resigned because of this [1]

[1] https://www.bbc.co.uk/news/live/cly147rky81t

Hmm it can capture more than just single words though, e.g. meaningful phrases or paragraphs that could be written in many ways.

> There isn't an idea that "IQ is some innate fixed quality"

Actually yes there is; I have come across many people who believe this, specifically saying that IQ is fixed.

> that intelligence is an innate fixed quality

I would also disagree with this — intelligence can be increased, (e.g. through education, training, and practice), and also decreased, (e.g. by lifestyle / environment).

See also: "Major IQ differences in identical twins linked to schooling, challenging decades of research" [1] [2]

I.e. the idea that IQ is some innate fixed quality has evidence against it. It seems obvious that this is the case, given that people get their children tutors so they can do better at IQ tests to get into schools...

[1] https://www.psypost.org/major-iq-differences-in-identical-tw...

[2] https://www.sciencedirect.com/science/article/pii/S000169182...

> Any leakiness in BackProp is addressed by researchers who introduce new optimizers

> As a developer, you just pick the best one and find good hparams for it

It would be more correct to say: "As a developer, (not researcher), whose main goal is to get a good model working — just pick a proven architecture, hyperparameters, and training loop for it."

Because just picking the best optimizer isn't enough. Some of the issues in the article come from the model design, e.g. sigmoids, relu, RNNs. And some of the issues need to be addressed in the training loop, e.g. gradient clipping isn't enabled by default in most DL frameworks.

And it should be noted that the article is addressing people on the academic / research side, who would benefit from a deeper understanding.

YouTube has so much questionable content on it that gets millions of views... Parents who've found ways to monetize their kids. Dangerous / unpleasant pranks being pulled on members the public. Conspiracy theories. Fake game shows where the winners being given money are actually friends of the host. Or where the host pretends contestants are doing something dangerous, but actually it's CGI, (misleading young viewers into thinking the dangerous stuff is real / fun). Morons making content that's attractive to kids who don't know better. Etc.

While there is some quality content on there, the amount of terrible content getting vast amount of views is pretty high.

I guess one question is whether TV is much better.. I would say on average it probably is less bad, although there have also been / are questionable unethical tv shows. But at least with TV shows there's more likely to be a few more layers of questioning / analysing / looking at the ethics, with responsible people involved.

Is software quality generally lacking at the moment?

- latest android update broke their limit charging to 80% feature, various people complain but they don't fix it

- windows 11 crashing a lot, the fix for lots of people is to disable windows realtime threat protection

- macos electron bug, making it super slow

I do suspect it's due to a shift in software culture, where working and testing carefully is devalued, and "just ship it" and "move fast and break things" is glamorised.

Also: "we take a fast iterative approach"

Now translates to: "we release poorly tested software and wait for users to complain, instead of testing it ourselves"

Stuff like this feels like some company has managed to monetize an open source object detection model like YOLO [1], creating something that could be cobbled together relatively easily, and then sold it as advance AI capabilities. (You'd hope they've have at least fine-tuned it / have a good training dataset.)

We've got a model out there now that we've just seen has put someone's life at risk... Does anyone apart from that company actually know how accurate it is? What it's been trained on? Its false positive rate? If we are going to start rolling out stuff like this, should it not be mandatory for stats / figures to be published? For us to know more about the model, and what it was trained on?

[1] https://arxiv.org/abs/1506.02640

I agree, the post above you is patently wrong / hasn't read the paper they are dismissing. I also got multiple downvotes for disagreeing, with no actual rebuttal.

I think the main thing to clarify is:

It's not a uniform distribution after the first measurement, t_obs. That enables us to update the distribution, and it becomes a decaying one.

I think you mistakenly believe the distribution is still uniform after that measurement.

The best guess, that it will last for as long as it already survived for, is actually the "median" of that distribution. The median isn't the highest point on the probability curve, but the point where half the area under the curve is before it, and half the area under the curve is after it.

And the above equation is consistent with that.

> Model state is present only in so-far-generated text

Wrong. There's "model state", (I assume you mean hidden layers), not just in the generated text, but also in the initial prompt given to the model. I.e. the model can start its planning from the moment it's given the instruction, without even having predicted a token yet. That's actually what they show in the paper above...

> It is only after the model has found itself in a poetry generating context and has also selected the first line-ending word, that a rhyme scheme "emerges" as a variable

This is an assertion based on flawed reasoning.

(Also, these ideas should really be backed up by evidence and experimentation before asserting them so definitively.)

Ah no, it is the Copernican principle, in mathematical form.

Yes that's pretty much it. There will be a decaying probability curve, because given you could fail at any time, you are less likely to survive for N units of time than for just 1 unit of time, etc.

(Just to expand on that, it's true not just the for the first token. There's a lot of computation, including potentially planning ahead, before each token outputted.)

That's why saying "it's just predicting the next word", is a misguided take.

That's a paradox that comes from getting ideas mixed up.

The most likely time to fail is always "right now", i.e. this is the part of the curve with the greatest height.

However, the average expected future lifetime increases as a thing ages, because survival is evidence of robustness.

Both of these statements are true and are derived from:

P(survival) = t_obs / (t_obs + t_more)

There is no contradiction.

That's inverted, it would be:

If: P(1 more minute | 1 minute so far) = x

Then: P(1 more minute | 2 minutes so far) > x

The curve is:

P(survival) = t_obs / (t_obs + t_more)

(t_obs is time observed to have survived, t_more how long to survive)

Case 1 (x): It has lasted 1 minute (t_obs=1). The probability of it lasting 1 more minute is: 1 / (1 + 1) = 1/2 = 50%

Case 2: It has lasted 2 minutes (t_obs=2). The probability of it lasting 1 more minute is: 2 / (2 + 1) = 2/3 ≈ 67%

I.e. the curve is a decaying curve, but the shape / height of it changes based on t_obs.

That gets to the whole point of this, which is that the length of time something has survived is useful / provides some information on how long it is likely to survive.

> while this Copernican principle sounds very deep and insightful, it is actually just a pretty trite mathematical observation

It's important to flag that the principle is not trite, and it is useful.

There's been a misunderstanding of the distribution after the measurement of "time taken so far", (illuminated in the other thread), which has lead to this incorrect conclusion.

To bring the core clarification from the other thread here:

The distribution is uniform before you get the measurement of time taken already. But once you get that measurement, it's no longer uniform. There's a decaying curve whose shape is defined by the time taken so far. Such that the estimate `time_left=time_so_far` is useful.

Ah, that's not correct... That explains why you think it's "trite", (which it isn't).

The distribution is uniform before you get the measurement of time taken already. But once you get that measurement, it's no longer uniform. There's a decaying curve whose shape is defined by the time taken so far. Such that the statement above is correct, and the estimate `time_left=time_so_far` is useful.

You've added some useful context, but I think you're downplaying it's use. It's non-obvious, and in many cases better than just saying "we don't know". For example, if some company's server has been down for an hour, and you don't know anything more, it would be reasonable to say to your boss: "I'll look into it, but without knowing more about it, stastically we have a 50% chance of it being back up in an hour".

> The better thing to do is to get some even-specific knowledge, rather than trying to reason from a priori logic

True, and all the posts above have acknowledged this.