feanaro

This is unrelated to the above comment, but I figured I'd ask while we're here. You state at the end of your blog post on this topic that you'd now probably recommend the NIST curves over Curve25519. Why is that?

I was responding to a comment from a person claiming this is not an example where the "intention/substance of the work might exist somewhere buried within a wall of impenetrable LLM text". I am saying this is in fact the case, given human experts almost universally agree that the papers are badly written, with inscrutable, hard to follow logic.

I had initially understood you as saying that this inscrutable logic was merely a consequence of superhuman intelligence, which is what I deemed unlikely. I don't see experts agreeing with that position either.

I am not unhappy about this result, but I am somewhat unhappy about it being dumbed down, decontextualised and understood without any nuance.

I mean, this is possible, but it seems quite unlikely if you take a good impartial look. It's nothing I personally dislike, in fact I enjoy LLMs when they are understood as what they are, which is a tool.

What I do not don are these rose-tinted, thought-terminating glasses which make people assume that this just must be an impenetrable product of superhuman intelligence.

No, it's just a word for which LLMs have an inordinately large bias, so it's often found in blog posts written by one.

How is it not true here? Plenty of papers in this release are hard to follow, use non-standard terminology, make hard to understand leaps, and so forth.

Even if it were not, it is a wise aim, unlike ceding understanding to a robot.

Of course, sorry if that came across as more combative than necessary. I was trying to point out two things I guess:

1. "Your experience is not universal from my perspective, and I offer my experience as evidence"

2. The possibility that this adaptation you are observing is neither a stable equilibrium given future developments, nor is necessarily an indication that the people using AI have bought into it fully. I use AI heavily while remaining a sceptic.

What exactly is appalling? Are you referring to something in the sentence you quoted?

It's possible and I dare say even likely that you are missing nuance and observing bias from your bubble. It's possible to avoid being an AI "opponent" while still being a sceptic. It's possible to use AI reluctantly and with tact instead of going all in.

The whole thing is far from over when we haven't experienced the worst of the technology yet: deskilling, concentration of power, extreme imbalance of wealth, military and surveillance uses, etc.

I think this matters very little. These were adaptations to initial advances of small scope which we are rapidly leaving behind. If there is truly no near and practical ceiling to the abilities of AI in mathematics, not even in terms of efficiency, then I think it is likely there is similarly no ceiling in other cognitive endeavours.

In that case, everything stands to be disrupted and those who wield AI will have massive amounts of power in the short term. But this is not a stable situation, and therefore those who wield AI will have to cede control of it or otherwise enter into a great war with those that do not.

I hope for all our sakes that there is a practical ceiling.

Are you advocating for inaction? If so, what do you wish to achieve?

If you consider the data on which an LLM was trained on to be points on a very highly multidimensional object, the claim is that the LLM can interpolate a convex hull spanned by those points, therefore recovering a subset of consequences attainable from those points. Obviously this hull includes completely novel points that were not present in the initial data set, so the output of the LLM goes beyond its initial training. And yet, there are clearly points outside a convex hull spanned by any finite number of points, such that we can imagine not all possible outputs are attainable using this method.

The claim is furthermore that truly original thinking, the infamous leaps in understanding and creativity, happen by attaining points outside such a convex hull.

It's hard to rigorously verify or disprove this claim. Hopefully this helps build an intuition of why the claim is not as shallow and obviously wrong as it may seem initially.

Full text: https://hal.science/hal-05643897

Surveillance is decidedly and completely unlike locks.

> This is the million dollar we'll see answered in our lifetime. Software engineering exists to automate work, are we arrogant to think we are not destined to the same fate? Is this truly a job befitting of a human over an agent?

There is a fine distinction here that I believe is often glossed over, so the two things it's delineating get muddled together. One of those two things is coding—the rote, mechanical encoding of meaning into computer instructions. It can be argued the LLM is fit to take this out of hands hands almost entirely, and it's almost indisputable the LLM is better at this in at least a certain sizeable subset of coding tasks.

But the other thing is the choosing, determination, specification of the intended meaning itself. This I think is squarely the job of the human, because letting this fall through to the AI means it is no longer the human that is making the decisions. This then becomes not merely automating work but ceding control. This, ultimately, is a bad thing.

So if we accept the premise that the specification of the intended meaning is the job of the human, the question is how you do that. Today many of us do it somewhat half-assedly, by writing lots of natural language text at the LLM and hoping it sticks. It is our hope that the text, given that there is a lot of it, will drive the stochastic machine in a sufficiently correct direction. This works to a degree—meaning we've ceded some control but not the majority of it—not least because we still read (most) of the code but cannot work in the limit, if code reading ceases.

A more proper way to specify the intended meaning is to specify (or "model") your system formally in a system that is mechanically verifiable. Then the final artefact produced by the LLM can be validated by verifying that it aligns with the specification. However this type of high-assurance specification looks a lot like a certain type of programming. In my opinion, writing this kind of specification is the future of human software engineering.

I do not accept the approach of simply rolling the dice and hoping the machine knows better than us, though I'm sure that church is also going to have its acolytes.

What is also lower is your understanding of the change. So yes, if you are now essentially only doing the final mile of paper pushing for the LLM, then the mental burden is lower but so is the assurance of what has just transpired.

Whether this mode of working is going to be long-term viable is going to depend on how important it is for you to be aware of what has happened for the system in question, how viable the economics are for the LLM usage at this level of assurance and how much ownership you exert over the LLM used or another similarly powered one (because otherwise the LLM can be taken away from you, leaving you at the mercy of a third party with goals that do not align with your own).

> How do you type an em-dash? Is it OS specific?

On Linux X11 at least, you can enable the Compose key and then press `<Compose>---` which results in — and `<Compose>--.` which gives you –

> The almond thing is false, but I'd argue that "misleading" might be defensible if you were to accompany it with "the majority of almonds are grown in California, but not all of them".

The "majority" in this case meaning about 51%, according to Wikipedia[1]? How could 51% ever be considered to be close to "all", such that "misleading" would be a valid answer?

Am I missing something?

[1]: https://en.wikipedia.org/wiki/Almond#Production

> I mean, "the pieces were already there" is true of everything? Einstein was synthesizing existing math and existing data is your point right?

If it's true of everything, then surely having an LLM work iteratively on the pieces, along with being provided additional physical data, will lead to the discovery of everything?

If the answer is "no", then surely something is still missing.

> And the "anyone who read all the right papers" thing - nobody actually reads all the papers. That's the bottleneck. LLMs don't have it. They will continue to not have it. Humans will continue to not be able to read faster than LLMs.

I agree with this. This is a definitive advantage of LLMs.

No, by saying this, I am not downplaying Einstein's sizeable achievements nor trying to imply everyone was wrong about him. His was an impressive breadth of knowledge and mathematical prowess and there's no denying this.

However, what I'm saying is not mere nitpicking either. It is precisely because of my belief in Einstein's extraordinary abilities that I find it unconvincing that an LLM being able to recombine the extant written physics-related building blocks of 1900, with its practically infinite reading speed, necessarily demonstrates comparable capabilities to Einstein.

The essence of the question is this: would Einstein, having been granted eternal youth and a neverending source of data on physical phenomena, be able to innovate forever? Would an LLM?

My position is that even if an LLM is able to synthesise special relativity given 1900 knowledge, this doesn't necessarily mean that a positive answer to the first question implies a positive answer to the second.

This alone still wouldn't be a clear demonstration that AGI is around the corner. It's quite possible a LLM could've done Einstein's job, if Einstein's job was truly just synthesising already available information into a coherent new whole. (I couldn't say, I don't know enough of the physics landscape of the day to claim either way.)

It's still unclear whether this process could be merely continued, seeded only with new physical data, in order to keep progressing beyond that point, "forever", or at least for as long as we imagine humans will continue to go on making scientific progress.

No, that's a completely different concept, because we have faultless machines which perfectly and deterministically translate high-level code into byte-level machine code. This is another case of (nearly) perfect abstraction.

On the other hand, the whole deal of the LLM is that it does so stochastically and unpredictably.

This is essentially a group theoretical result about permutation groups. Would be nice to see a treatment from this angle.

And yet the categorical concepts in Hask are undoubtedly practically useful, more so than an arbitrary sample of concepts, and compose extraordinarily well. Does that have nothing to do with those concepts deriving from (even more general concepts of) category theory?

I don't think anything good ever came from Ylva Johansson. Mentions of her name on something should make one automatically treat that thing with suspicion.

How do you discover the principles in the first place? You can discover them once and then apply them in all applicable places precisely because you generalised them.

You have a point that the result may very well be more easily explained in concrete terms to practitioners of a given field in which you applied it, though.

What do you mean by the Wireguard option for mitmproxy?

EDIT: Oh, look at this https://mitmproxy.org/posts/wireguard-mode/. TIL.

A server is someone else's device. Your phone is your own device. So no, doing the scan on your own device and making your device your potential adversary is not better than doing it on the server. You can always choose not to use the server.

So there is an upper limit, which is the real price?

What are you even on about, mate? A hacker's multi tool with infinite potential for exploration is an idea "too malicious" to consider?