djokkataja
> Importantly it was admitted that internal Anthropic models had been used in their proof of Euler blowup; I therefore felt I could not consider Levent to be an independent academic.
If an Anthropic employee is doing independent research, but with models that aren't available to the public (because they're internal models), then . . . idk. It's not clear to me why that should necessarily require a refusal to cooperate between OpenAI and Anthropic employees who are excited about solving a problem like this.
For me, the bigger question here is what "internal models" means to these employees, especially in the context of the OpenAI employees repeatedly avoiding directly answering whether their model had been trained on Tristan's and Levent's ongoing work on the problem. It had always seemed like a loophole that AI companies might be tempted to exploit: yeah, they can say that they won't train on your data, but if an AI company doesn't care about ethics, they might go ahead and train a model for internal use only on everyone's data anyway, just to have as much data as possible and potentially gain an advantage in what the company can internally do. They could never publicly release any versions of a model like that, of course. And of course this is speculation.
Compared to what??
Actually, someone even commented in that thread about how it was similar to biological mechanisms: https://news.ycombinator.com/item?id=47390619
> A quick note: All of the data visualizations in this roundup are free to use as common content. You are welcome to share and republish them.
Plus if you want to make a robot which can do everything a human can do, without exception, you'll start running into weird edge cases if you make it with a fixed wheeled base. Like suppose you want it to climb a ladder or turn sideways to slip into a narrow space and retrieve something. Or maybe you want it to be able to climb in and out of vehicles designed for human occupants. Would you want to have to buy a special vehicle to bring your robot somewhere with you?
And judging by the name of the company and that they likely aspire towards more than what they've made so far, I'd guess that a long-term goal could be to also make something for people who realize the weakness of their flesh and crave the strength and certainty of steel.
For better or for worse, the equivalent race right now is in AI.
Also, on the graph you linked to, you can click on the line in the graph of population versus year to the right, and if you start back in the 1950s and work your way along to the present, you can see the big lump of the baby boomers getting older. That's why people say the United States has an aging population.
This seems almost comically lemming-like. Fear of missing out . . . on dying?
> Getting frozen and brought back in the distant future, even if we assume you'll be in great health, means all your friends and family are gone.
(Assuming none of your friends and family were also frozen.) But more importantly, I wouldn't suggest that people should aspire to death because of losing all their friends and family. If starting over (in what's likely an unfamiliar society and culture) is all one's got, that's still something.
Mainly I find it grotesque that it might be a way to slip past death, but only for rich people. But I can't bring myself to suggest that it would necessarily be a better universe if they took the normal death route instead.
Penrose tries to respond to this by saying that various things may affect the functioning of a brain and keep it from reliably perceiving such truths, but when brains are working properly, they can perceive the truth of things. Most people would recognize that there's a difference between an idealized version of what humans do and what humans actually do, but for Penrose, this is not an issue, because for him, this truth that humans perceive is an idealized Platonic level of reality which human mathematicians access via non-computational means:
> 6.4 Sometimes there may be errors, but the errors are correctable. What is important is the fact is that there is an impersonal (ideal) standard against which the errors can be measured. Human mathematicians have capabilities for perceiving this standard and they can normally tell, given enough time and perseverance, whether their arguments are indeed correct. How is it, if they themselves are mere computational entities, that they seem to have access to these non-computational ideal concepts? Indeed, the ultimate criterion as to mathematical correctness is measured in relation to this ideal. And it is an ideal that seems to require use of their conscious minds in order for them to relate to it.
> 6.5 However, some AI proponents seem to argue against the very existence of such an ideal . . .
Source:
https://journalpsyche.org/files/0xaa2c.pdf
Penrose is not the first person to try to use Gödel’s incompleteness theorems for this purpose, and as with the people who attempted this before him, the general consensus is that this approach doesn't work:
https://plato.stanford.edu/entries/goedel-incompleteness/#Gd...
Also I think in the US at least, religion plays into it. God over humanity is conceptually the ultimate hierarchy model that Christians buy into. So it seems like Christians often don't have as much issue with embracing the idea of one person being over another person as a reflection of that model. This seems a little bit funny given how anarchic Jesus was, but he never said, "Yo, check it out, I am literally an anarchist and you should be too," and so they seem to have missed that.
> Only on the outside, where they can be appreciated as a whole, will the impression of intelligence emerge. A human brain, too, does not exhibit the intelligence under a neurobiologist's microscope that it does participating in a lively conversation.
We only have fuzzy definitions of "intelligence", not any essential, unambiguous things we can point to at a minute level, like a specific arrangement of certain atoms.
Put another way, we've used the term "intelligent" to refer to people (or not) because we found it useful to describe a complex bundle of traits in a simple way. But now that we're training LLMs to do things that used to be assumed to be exclusively the capacity of humans, the term is getting stretched and twisted and losing some of its usefulness.
Maybe it would be more useful to subdivide the term a bit by referring to "human intelligence" versus "LLM intelligence". And when some new developments in AI seem like they're different from "LLM intelligence", we can call them by whatever distinguishes them, like "Q* intelligence", for example.
This is true ...
> If it becomes impossible to rent properties profitably, then eventually there would just be fewer rentals available.
But this is a leap that doesn't apply to the Georgist scheme. A key feature of the scheme is that taxes on buildings (possibly among other taxes) are removed in favor of the land value tax. It's true that if the tax on some piece of land increases sufficiently, it will eventually become impossible to profitably charge rents on existing buildings on that land--but improving existing buildings or building some superior building (for example, a new building with more units than a pre-existing building) allows landlords to continue profiting.
If the current landlords don't want to improve buildings there, they can sell the land to someone who will. And it's clear that someone will, because if there's enough demand for the land that the land value tax makes it unprofitable to sit on it without improving it, someone is going to want to do something more with it.
The reason we don't already have a Georgist-style economy is quite obvious, of course: the homo sapiens landlordicus is a very soft, squishy, vulnerable subspecies. If they had incentives to improve their properties and thereby possibly even work as hard as the rest of the human species, they might well go extinct. And no one wants that.
> We don't.
I don't mean humans as they presently are, but humans as somewhat-novel GAI entities after upgrading (and humans as an ongoing source: likely there will be some humans that aren't interested in changing themselves in such ways, but who might change their mind over time and/or have children who see things differently).
> (I am speaking in economic terms. We can deeply intensely highly "value" many things, but if we don't actually invest economic value in protecting or nuturing those things, our appreciation has little or no impact.)
I don't think this is true. For example, suppose you deeply value a bunch of things that you don't think have much economic significance, but economic significance is the most important thing to your perspective / philosophy, so you don't take care of the things that you deeply value. This does have a significant impact: you become sad, you become depressed, you become less economically productive. So then you say, oh, I'll just take care of those other things I care about, and then I can be economically productive! So you make a few changes but then find yourself feeling down again not long afterwards because economic value is the cornerstone of how you think about things. And then you realize that from an economic perspective, it's more beneficial economically to stick with a perspective that embraces your values as you discern them (and how those continue developing over time) rather than sticking with a perspective that puts economic value first. And then you are taking your first steps to freedom from the curse of your Bachelor's in Economics. ;)
Ethical GAIs also seem likely to me ... hopefully.
The additional value that humans present would thus be a diversity factor. Much as it might be economically beneficial to cut down the entire Amazon, there's also significant economic benefits in keeping it intact and finding out what kinds of crazy biological stuff comes out of there. And over the long run, we get way more economic benefit from finding crazy things in there than in chopping it all down. There are all kinds of ways to explore uncertain dimensions, but looking at whatever the universe has already managed remains a consistent source of value.
> But transforming ourselves into GAI to keep up (while maintaining continuity of memories, etc.) is going to be a much more expensive proposition than simply making more GAI hardware from scratch.
It's basically a fixed cost because of how slowly the human population grows (if it even continues to grow).
Synthetic GAI entities will be able to use vastly more resources than humans presently do, because they'll be able to create more of themselves as quickly as resources become available, and they won't be stuck operating at a fixed clock speed. It's also probably going to be much easier for them to make use of off-world resources. So the total population of GAI entities could plausibly explode compared with the total human population.
It comes down to timing: if there's technology available for humans to become less ... biological, and it seems very desirable to lots of people, but it's really expensive and there aren't that many GAI entities yet, that might be less pleasant. But if that kind of tech depends sufficiently on AI developments that there's some significant lag time between a GAI population explosion and that kind of human-upgrading tech showing up, the cost of making it available to whatever humans want it might be a drop in the bucket from a total economic perspective.
To be clear, I don't see biological humans "keeping up" over the long term. If we take AGI and BCIs and various Ship of Theseus questions seriously, there could be some significant blurring of lines between "human" and "AI." And if we combine that with GAIs originating from humans, the concept of "keeping up" seems to become less meaningful. Who's keeping up with who?
Lastly, I'm suspicious that GAI entities won't be purely economically motivated, because I don't see any reason that they'll be "purely" anything at all. There is no magical "essence d'intelligence"; instead there's staggering layers of complexity. And every plausible AI safety approach I've seen so far involves training AIs to be inclined towards beneficial acts and away from harmful acts because there's no way to program conceptually pure "motivations" into them.
Which is why no child has ever survived until adulthood.
And TBH I find this an appropriate metaphor because the promise of AGI is that every adult human will be infantile by comparison--but there are also potential technologies that could allow humans to ... "upgrade" themselves and become a mature ... whatever. Sadly brain-computer-interfaces seem like a technology that works best when enabled by AI (to help with interpreting brain signals), so it seems quite unlikely that any biological humans are going to keep up with AI over the medium term.
Consider the relative charisma of the people around him, though.
1. I wouldn't necessarily assume FTL. But even if we assume "arbitrarily technologically advanced," there's no reason to think that any technology will ever have an absolute 0% chance of failure.
2. It only has to be sufficiently peaceful to not wipe itself out.
3. I don't think AI doing things changes either of the previous two points.
What I find harder to believe is that humans would be able to get their hands on a crashed alien craft and keep it.
I also found this amusing. But you are loosely correct, AFAIK. GPT-4 cannot reliably explain itself in any context: say the total number of possible distinct states of GPT-4 is N; then the total number of possible distinct states of GPT-4 PLUS any context in which GPT-4 is active must be at least N + 1. So there are at least two distinct states in this scenario that GPT-4 can encounter that will necessarily appear indistinguishable to GPT-4. It doesn't matter how big the network is; it'll still encounter this limit.
And it's actually much worse than that limit because a network that's actually useful for anything has to be trained on things besides predicting itself. Notably, this is GPT-4 trying to predict GPT-2 and struggling:
> We found over 1,000 neurons with explanations that scored at least 0.8, meaning that according to GPT-4 they account for most of the neuron’s top-activating behavior. Most of these well-explained neurons are not very interesting. However, we also found many interesting neurons that GPT-4 didn't understand. We hope as explanations improve we may be able to rapidly uncover interesting qualitative understanding of model computations.
1,000 neurons out of 307,200--and even for the highest-scoring neurons, these are still partial explanations.
> Since the beginning of the 20th century, intelligence has been conceptualized as a qualitatively unique faculty (or faculties) with a relatively fixed quantity that individuals possess and that can be tested by conventional intelligence tests. Despite the logical errors of reification and circular reasoning involved in this essentialistic conceptualization, this view of intelligence has persisted until the present, with psychologists still debating how many and what types of intelligence there are. This paper argues that a concept of intelligence as anything more than a label for various behaviors in their contexts is a myth and that a truly scientific understanding of the behaviors said to reflect intelligence can come only from a functional analysis of those behaviors in the contexts in which they are observed. A functional approach can lead to more productive methods for measuring and teaching intelligent behavior.
https://www.researchgate.net/publication/266418013_The_myth_...
> Download the tinyLM.hpp file from the tinyLM GitHub repository and include it in your project.
"tinyLM GitHub repository" links to https://github.com/sksg/tinyLM/blob/master/tinyLM.hpp, but that's a 404 page. And digging around in the actual repository doesn't show a tinyLM.hpp file anywhere--it looks like the project never actually developed beyond a gitignore and a readme.
It almost makes me wonder if whoever does their site design is trying to make it seem like OpenAI is making HAL.
Granted, that's still really nebulous, but I like the idea of it better than endless cosmic boredom.
https://www.techspot.com/news/97572-mount-blade-ii-mod-uses-...