ordersofmag
ordersofmag on Hacker News
I read it in a cabin on a lake, far from any internet access, and without a computer. I still found it quite useful.
I think it's more accurate to say that the fundamental differences (between LLM's and humans) are abundantly clear and what we haven't really got a handle on are the similarities. Why is it that an LLM's can so convincingly mimic some things that humans do? Of course the fact that we don't really understand how humans do it in the first place is a real barrier. Perhaps looking at the intersection will shed some light on how humans work.
There is no such thing as a GPS radio in a phone in the sense of something that broadcasts a signal. GPS signals are broadcast by satellites and any device on earth can choose to listen for the signal and triangulate their position by receiving the signal from multiple satellites. GPS doesn't send out any signal from your phone to work. In the same way that your eyeballs don't need to emit light for you to see (unless you're a superhero). Phones broadcast signal to connect to cell networks and wifi networks.
Now if you expect your phone to show your position on a map (using the GPS coordinates it calculated from the satellite signal) then it likely needs to retrieve map data from the internet (and in doing so it might be broadcasting over cell or wifi frequencies). But that's a different story.
Much confusion is caused by people saying "I tracked X with GPS" with the implication that GPS allows tracking of remote objects. It doesn't. It allows the remote object to know its own location. For that information to get to someone else (the spy in the chasing vehicle) the remote object needs to be broadcasting that location information through some other means (which has nothing to do with GPS).
True for the pre-training data scraped from diverse sources. Less so for the later stage data for RL which is by all account more of a differentiator. In most cases the labs themselves produced the data so they are the copyright holders (or they are borrowing it from other labs via distillation). A lot of it is synthetic data, and since you can't copyright AI output, it becomes less about copyright and more about trade secrets.
Historically large data centers used evaporative cooling. So the water disappeared in sense that it's no longer available in a useful form in it's original locations. That's changing with growing use of closed-loop systems.
True if you're using a closed-loop system. But if you're using evaporative cooling (which is the main cause for concern) then the place where the water 'comes back into the environment' may be quite far away (at least in a useful sense).
This is entirely dependent on where you are geographically in terms of rainfall and weather. Both water and energy are abundant. The trick is getting them where you need them. So the where is what matters.
Please compare 1 million gallons to the 1 quadrillion gallons in the aquifer, or the 31 trillion that falls from sky each year in Nebraska and get back to me.
I think you confused "can't" with "shouldn't".
How to say you've never met a physicist....
If you're confused about who to contact to follow up on the ideas of a particular paper then write to the first author, or whichever author is listed as the contact person. This is not hard and is an orthogonal consideration to the length of the author list.
Which itself not necessarily a sign of fraud: https://www.science.org/content/article/physics-paper-sets-r...
These are all silly non-problems (speaking as someone who works with lots of professors around how to put AI systems to effective use)
-the thing that motivates you doesn't have to be the same thing that teaches you (e.g. a book can be a useful learning tool even if it isn't human)
-you can absolutely choose to interact with an LLM in ways that generate productive struggle. If you naively ask it for answers it won't be much use in this regard. But that's just a poor choice of how to use the tool. Just like hitting yourself on the head with a book doesn't mean the problem is with the book as learning tool.
-You don't need structured content 'on long scales' to learn. You learn a little bit at a time. Long textbooks are a side effect of the economics of moving paper around. Not a prerequisite for learning. An LLM will happily mimic the table of contents of a textbook or an outline for course. And you can use future prompts to generate the bits you need as you need them. One-shoting a textbook is a silly use of the tool. Plus, we're not short on textbooks. We're short on ways students can get customized help: answers to the specific ill-posed questions they have in their heads. LLM's are quite good at this. You can have textbooks AND LLM's (and professors)
-LLM's are happy to disagree with you. Just ask them to and they oblige. Ask them to respond as if they were a tutor and they will mimic the assuredness if appropriate.
Spoken as someone who has never seen a weather forecast. Computers have long produced results that were, if interpreted naively, incorrect. These systems require humans to take into account the conditional, probabilistic nature of the output and the inherent limitations of the model the computer was executing, when they interpret the output. LLM's are no different. They give you next token prediction probabilities. That is all. There is no particular reason to think that comes (or even could come) with a promise that those tokens will express truths about the world and only truths. The fact that greed-driven hype, and our human inclination to anthropomorphize the token generator inclines us to imagine things should be different doesn't make it so. No model output is truth. Never has been.
I'm genuinely unclear on what you mean here. Do you mean:
1. Most mathematicians are currently largely paid as part of their academic job (as a faculty member) and will continue to do that work (including the research part) even if extra-mural grant funding goes away. That seems entirely plausible to me.
Or do you mean?
2. Real mathematicians will continue to devote a significant part of their life/energy to mathematics research as a hobby, even if no one is paying them (in any way) to do it? That seems more of a reach and makes me wonder if many currently folks current employed as mathematicians aren't 'Real mathematicians' from your point of view.
If what you're trying to build is a perfect match for the full set of abstractions React provides then it's gonna be cleaner to use React than have AI re-invent it. But most cases I run into don't fit that description.
The LLM does not determine the next token. It generate odds for all of the tokens it knows as to their likelihood of being 'next'. It's up to the harness running the LLM (and in most cases the a temperature setting) to actually decide on a particular next token. I think it's more accurate to call the thing the LLM actually generates (an ensemble of probabilities) a 'prediction'. It might be accurate to say the harness decides on the next token based on the prediction from the LLM. The role of the LLM is much more akin to predicting your opponents move than deciding your own.
Sounds like a very narrow view on what constitutes science. There are many fields of science where there is existing data against which new ideas can be tested without additional 'real-world' measurements. Newton's theory of gravitation relied entirely on pre-existing astronomical data for which there was no existing unifying theory. He made progress by putting forward a theory which explained that data. Now you can argue that it's not really science unless you include the original data collection and subsequent real-world measurement validation steps. But I'd be comfortable saying that Newton was indeed a scientists and did make progress in science despite only doing what some might say is the 'middle' part of the process. There are plenty of modern analogs where work like this sits out there waiting to be done using existing data.
Yes. But you don't need to monitor glucose levels to do this. Because it's well known you can productively use as much as you could manage to consume (without vomiting). So all the device will ever tell you (if you're in the middle of significant endurance event) is that you need to raise your blood sugar. It'd be no more actionable than a writing 'your blood sugar is low' on your hand with a sharpie.
If 'democratic values' are everyone having an equal voice in how the systems of a country are arranged. And if those people, through a democratic process, decide that the systems of their country shouldn't support the exploitative elements of capitalism, then it would definitely be democracy in action to make that come true. Anything counter to that would be, by definition, anti-democratic.
When you say "socialism conflicts directly with democratic values" I genuinely have no idea what "democratic values" you're talking about.
So is a zip file of source code. There is no 'compilation process'. Model training isn't a fixed end point. You start with random weights. You train. The weights get better. You do this iteratively. At some point you say 'good enough' and release. People doing further training would start with those weights and further iterate. Demanding the original training data and training process would be equivalent to demanding a completely replay-able sequence of editing steps starting with a blank text file and allowing you to replay ever edit that led the original author to the released source code.
Now there are reasons you might want to know about the training data when you wouldn't care about the authoring process used by a traditional open-source process. And these get at the reason LLM's are different than traditional software and so maybe our existing definitions of what 'open-source' means aren't a good match for LLM's. Of course there is software associated with LLM's (beyond the weight) -- defining the structure of the particular neural net those weights fit into. In every open-weights model that I'm aware of that software is open source (though trivial).
99% of AI users these days aren't using a 'raw' LLM. They are interacting with a harness that includes tools to do search of the live (or recently crawled) web. And so the output users actually see could absolutely include correct citation of sources. So the 'architecturally impossible' bit may be technically correct but it is not practically relevant. Now whether those harnesses do a good job of orchestrating LLM output to get accurate citations (and whether they are transparent about their process) is another thing entirely. But if you're contemplating 'what LLM's can do' and aren't taking into account the harness and tooling ecosystem they are embedded in then you're missing the point.
In my previous life I did similar computational physics work with similar environments. I ran into a similar set of problems and found that with a little work one can cobble together tools to automate all the steps that are a hassle. And then the hassle goes away. And this was pre-Claude Code. And pre-Google. And actually pre-ssh. Good old telnet for me. Some shell script or a little AppleScript. Granted no 2FA in those day. But the same learned helplessness with people sitting in front of a device who's entire purpose is to automate things and when presented with a task that would be better automated they fail to use the tools at hand to solve the problem. Kids these days (and back in those days too for that matter).
He is possessing it. That is the goal. To have more and to continue to acquire more. The mountain is there and he feels a need to climb it. But if he were to reach the top he would not be happy. The greed admits no conclusion. It must be fed continually. The joy is in the moment of 'winning', which is transient. There is no end other than death.
Some people are like this. Ever major religion and thought movement throughout history recognizes people like this. They need to have more for themselves even (and in many cases especially) if it means less for others. In pre-history when we lived in small groups and we really knew each other the group could identify and isolate and disempower them. Populations grew and so did their ability to lie and get away with it. To deceive and amass power. To exploit the systems that we built to help the collective to their own ends.
Nope. The main cost of the old climate.gov was the salaries of the folks writing the articles and pulling together the resources. They were not getting paid exorbitantly and are quite interested in still getting paid. Source: I am one of those people.
The site wasn't (isn't) about the data. It's about articles that contextualize the data. The money raised has allowed them to stand up a new site with all the old articles (which truth be told were all still nominally accessible via the internet archive) and will help fund them to create new ones. So it will stay relevant by paying the people who in the past worked for NOAA to creation the content, to now create the content paid for by donation.
I think the point is that there has been a push to move away from this data continuing to be available from these sources.
The grass is there to keep the dirt in place.
If you want to get pedantic we can't simulate
anything with
complete accuracy in the absence of a theory that encompasses all the known forces. Which we don't have. (Damn you gravity. Can't you just get along with the others)
To a useful level of accuracy we can certainly simulate water. And we can do the same for a single proton for some definitions of useful (but not other definitions).