nfc

Sci-fi plot:

We change as a society the definition of intelligence to "what humans can do better than AI".

We end up in a world where AI is only worse than humans in things where we are worse chimpanzees.

It seems like an extremely coarse classification. Category 3 contains languages with very different degrees of difficulty, while Bulgarian and Russian are both Slavic they are nothing alike in terms of difficulty since Bulgarian is the most analytic of Slavic languages (has the less inflection). That makes it extremely easy to learn compared to Russian.

Thank you for your reply, I may have misinterpreted what Weatherby was saying and I admit I did not spend enough time reading it. I've re-skimmed it and think you may be right.

With respect to the use of LLMs for my original comment. I think however that this is a useful use for them. It started a conversation on an article that had not comments on it and helped at least one person (me but hopefully others too) to get a better understanding of what was said (thanks to your comment). But it's not a hill I'm willing to die on, specially after already having been wrong once in this thread :)

Had a conversation with an LLM (not even a thinking one) about this article. As I am in mobile and have not much time I asked it to summarize some of my views for a comment on HN, this is the result, hope this does not go against HN guidelines :)

Weatherby argues that LLMs challenge traditional ideas of language tied to human cognition, and he promotes a structuralist/dialectical view where meaning arises from systemic relations and oppositions—not just usage or intent. But he misrepresents the “statistical” view by treating it as naive co-occurrence tracking. Modern LLMs (e.g., transformers) don’t just count— they build complex, high-dimensional structures where meaning emerges from position in a learned space. So ironically, these statistical systems end up modeling something very close to the structuralist “totality” he champions. The divide isn't as clear as he suggests.

Something I ponder in the context of AI alignment is how we approach agents with potentially multiple objectives. Much of the discussion seems focused on ensuring an AI pursues a single goal. Which seems to be a great idea if we are trying to simplify the problem but I'm not sure how realistic it is when considering complex intelligences.

For example human motivation often involves juggling several goals simultaneously. I might care about both my own happiness and my family's happiness. The way I navigate this isn't by picking one goal and maximizing it at the expense of the other; instead, I try to balance my efforts and find acceptable trade-offs.

I think this 'balancing act' between potentially competing objectives may be a really crucial aspect of complex agency, but I haven't seen it discussed as much in alignment circles. Maybe someone could point me to some discussions about this :)

I agree with your point about the validation bottleneck becoming dominant over raw compute and simple model scaling. However, I wonder if we're underestimating the potential headroom for sheer efficiency breakthroughs at our levels of intelligence.

Von Neumann for example was incredibly brilliant, yet his brain presumably ran on roughly the same power budget as anyone else's. I mean, did he have to eat mountains of food to fuel those thoughts? ;)

So it looks like massive gains in intelligence or capability might not require proportionally massive increases in fundamental inputs at least at the highest levels of intelligence a human can reach, and if that's true for the human brain why not for other architecture of intelligence.

P.S. It's funny, I was talking about something along the lines of what you said with a friend just a few minutes before reading your comment so when I saw it I felt that I had to comment :)

The Turing test seems to be a product of an era where the nature and capabilities of artificial intelligence were still in the realms of the unknown. Because of that it was difficult to conceive a specific test that could measure its abilities. So the test ended up focusing on human intelligence—the most advanced form of intelligence known at that time—as the benchmark for AI.

To illustrate, imagine if an extraterrestrial race created a Turing-style test, with their intelligence serving as the gold standard. Unless their cognitive processes closely mirrored ours, it's doubtful that humans would pass such an examination

Just thought about this scenario, there are probably more likely ones.

If some AI had access to the missile launch system, the best course of action for it would probably not be to launch immediately. This is because nowadays it is very unlikely that it would be able to repair itself so launching immediately would ensure its own destruction (and probably auto-destruction is not its goal)

If it was discovered it could just threaten humans with launch if they do not help it reach the state at which it would be able to repair itself (at which point humans would no longer be necessary)

I enjoyed the article but have a very minor nitpick. I didn't understand why the author added this sentence.

"However, the timescales involved in these calculation are so unreasonably large and abstract that one could wonder if these makes any sense at all."

Apart from the fact that we could wonder about anything and everything I think the author does not state what evidence do we have to suspect that large enough timescales would change the laws of physics.

It could be the case of course, and it would be great to talk about them if they exist but without further justification I feel that this sentence is an unjustified opinion in what is otherwise a very nice article that helps better understand enthropy.

“If we define understanding as human understanding, then AI systems are very far off,”

This took me into the following line of thought. If we wanted AGI we probably should give this neural networks an overarching goal, the same way our intelligence evolved in the presence of overarching goals (survival, reproduction...). It's these less narrow goals that allowed us to evolve our "general intelligence". It's possible that if we are trying to construct AGI through the accumulation of narrow goals we are taking the harder route.

At the same time I think we should not pursue AGI the way I'm suggesting is best, too many unknown risks (paperclip problem...)

Of course all this begs the question of what is AGI, how we define a good overarching goal to prompt AGI and many more...

I'm trying to help friends in Ukraine as much as possible and advice on secure communications would be a way to do it. I know messaging apps security have been discussed in HN before but I wanted to ask the community about it in the context of the conflict of Ukraine.

The end goal for me is to give the best advice for my friends but I think it can also lead to the type of discussion HN is focused on.

I was thinking something similar recently, and I also believe it's an idea worth exploring. Something else to add to this conversation. There's an obvious difference between two cases in which a trusted person trusts a url:

1) Single contributor website (blog, personal page...): It seems that we could spread the trust the whole website in the algorithm (at least more than for the next case)

2) Multi contributor website (forum, newspaper): It seems the trust should be given at an URL level

Something worth delving into if we are designing this trust based search engine in real-time here at HN ;)

Then you just stop trusting them for your search results, it's not like you are unfriending them.

Couldn't we run experiments to discover whether they really have this role?

Like for example choosing an island with mosquitoes (and far enough from other lands as to not be rapidly repopulated) eliminating them there and seeing what the effect is in the ecosystem?

I wonder if something like this has ever been tried.

So we should allow wolves population to increase until the proportion of flying to normal cars is?

I'm gonna advocate Tenerife (my island) since it would be amazing to have more HNers around :)

https://weatherspark.com/y/147639/Average-Weather-at-Tenerif...

Doesn't seem to get much better according to weatherspark :)

This just made me think how much the influence on civilizations that act based on some "Dark Forest" type of hypothesis could change based on what these civilizations (believe to) know about physics

Hi dang, thanks for the amazing work you do moderating HN.

Reading your words "The weariness that arises at the thought of caring about commenter opinions" got me thinking about how the unique experience (even if perhaps not specially positive) that a moderator has to go through can influence his view of the world. I hope this equilibrium that you seem to have reached makes you happy, I can imagine how moderating can be an ungrateful job.

Yes, this is different from dark matter. This is baryonic "matter" that is predicted by our theories but that had not been previously detected.

However the title is misleading, this is not the discovery of this matter even if the press release makes it look so. Previous detections of this matter had been discussed on HN before https://news.ycombinator.com/item?id=17953600

What this is is an advance on our understanding of this previously undetected matter. You can read more about it here https://www.aanda.org/articles/aa/full_html/2020/11/aa38521-..., this work has advanced our knowledge of the temperature and density of the cosmic filaments where this previously undetected matter is.

I'm not an astrophysicist but I did my PhD about the stacking technique used in this paper (but applied to other problem). It was a long time ago though, I could be forgetting something important.

I also think there's a lot to be tried in the space of "how to build technology companies". I see many options available to technology companies that are not realistic for traditional companies. I think that we need to experiment with legal frameworks for technology companies the same way we experimented with licenses for software for this potential to be fulfilled.

This is the only source I've found to easily compare those https://euromomo.eu/graphs-and-maps

That's great, it's certainly something fascinating to explore, I'm happy you are working in this kind of ideas :). I'll have a look when I'll have a bit more time.

Edit: Remove the question about contact info since you already answered in your comment in another thread :)

Hi mimixco, I just wrote a comment https://news.ycombinator.com/item?id=23124707 about some thoughts I had about this kind of ideas, what do you think?, I'll try to read more about what you are doing as soon as I find some free time :)

I've given some thought to this and a related idea.

Let's say that for a subset of scientific papers you have the possibility of specifying both the premises and results in a way that can be composed.

Let's say for example that your result is that the rate of expansion of the universe is N. Other papers could cite this result let's say through a URL of the result. Other papers could then use this URL as premise for their results and we could create an automatic system that would notify all this papers if the result has changed after new data for any of the elements of the chain. Scientists could be notified that they should revise their own papers to see if their conclusions change with the new data so other papers depending on them can be notified. A paper could be even marked stale if after the change of one of the premises the authors have not confirmed that the conclusions are still valid, this staleness would propagate down the chain

A very simplified structure of the data would be sth like this:

{ premises: ['urlOfResult1', 'urlOfResult2',...], conclusions: [RateOfExpansion: > N] }

This is obviously terribly simplified, I suppose it'd take me a lot of time to clearly explain in more detail how such a system could work and I thought about it a long time ago.

It could be interesting to apply this to other fields, for example public policy: In this case let's say that we have created this law because of this piece of data. The data changes, we could be notified that perhaps the law could benefit from a new look.

{ premises: ['urlOfResultLeadIsNotDangerous'], conclusions: ['PeopleCanUseLeadInPipesAsMuchAsTheyWant'] }

Such a system could be made even more generic. Probably people have already worked in this kind of systems but I never took the time to investigate it. If someone knows of examples of this kind of systems applied I'd be very happy to know more

That's exactly the point I was trying to make

> they generally don't evolve to become more deadly

That's a very important point and one that makes me agree with you in that it'll probably won't happen. This is because in most cases a virus reduces its R0 when its deadliness increases.

What I'm trying to point out is that "unlikely" is not impossible and since the stakes are high we should at least think seriously how likely the other possibility is.

For example we could think that for a virus like SARS-CoV-2 with asymptomatic contagion it's possible to think that a more deadly strain could maintain an R0 above 1 (because people pass it before becoming very ill). I talked more about this in https://news.ycombinator.com/item?id=23025899 to which I received a very thoughtful answer.

Again that does not mean that it will happen just that it could, and I believe we should encourage an informed discussion about this possibility

This all depends on so many factors and there are still so many things that we don't know that I'm skeptical of people claiming they know how unlikely it is to happen. What is the degree of confidence of those virologists on their predictions?, what does highly unlikely mean in probabilities?. If you can share your sources I'd be happy to read them

From my understanding. A decrease of the virulence is the most likely outcome, however it's also possible (just not as likely) that a future strain could be both more deadly and have R0 > 1 (with R0 obviously depending of how we behave). We probably will not have a worse wave in the future but we should think about how to prepare for it.