News for hackers. Stuff that matters.
Re:500'000€ Prize for Compressing Human Knowledge by TheRealPomax on Wednesday September 13, 2023 @06:52PM (#37502727)

  Q: Why do you restrict to a single CPU core and exclude GPUs?
  A: The primary intention is to limit compute and memory to some generally available amount in a transparent, easy, fair, and measurable way. 100 hours on one i7 core with 10GB RAM seems to get sufficiently close to this ideal
 
Sorry, who are these people that don't have a GPU? Even laptops have GPUs. Why would you spend 100 hours on an "i7" (which generation? 4790K or six times faster 13700k?) CPU when you can achieve orders of magnitude better performance on a consumer GPU that literally everyone has access to?
Re:500'000€ Prize for Compressing Human Knowledge by lucb1e on Wednesday September 13, 2023 @06:55PM (#37502756)
Note that the competition close to 20 years old

...though I also had a GPU in 2006, so idk. Then again, you need to define something as reference hardware and it doesn't really matter what it is. Better compression should win out over less-good compression no matter if you run both on a 100-core system or a 1-core system, I think?

Re:500'000€ Prize for Compressing Human Knowledge by TheRealPomax on Wednesday September 13, 2023 @06:58PM (#37502782)
In the category "then update your FAQ, you've have many, many years to do so" =D

(not to change the rules, but to explain why they rules haven't changed, and to clarify the super vague phrasing even back when the 3dfx Voodoo was the pinnacle of graphics. Level playing fields are a worthwhile pursuit)

Re:500'000€ Prize for Compressing Human Knowledge by Zandikar on Wednesday September 13, 2023 @07:17PM (#37502919)
> Sorry, who are these people that don't have a GPU

Literally millions of people. Especially circa early 2000's when this started.

> GPU that literally everyone has access to?

They don't. (Money is a barrier to access).

> Why would you spend 100 hours on an "i7"

Because it's what they had/have.

This a competition to move understanding forward, not to see who has the biggest budget.

Re:500'000€ Prize for Compressing Human Knowledge by nomel on Wednesday September 13, 2023 @09:08PM (#37503666)
The level of privilege that some people are completely blind to is just incredible.
Re:500'000€ Prize for Compressing Human Knowledge by modeless on Thursday September 14, 2023 @12:16AM (#37504909)
Sure, in the early 2000s. But today literally every laptop or desktop computer or phone with a touchscreen has a GPU. Even most feature phones have GPUs, I'd actually be interested to see if you could find a currently manufactured one that doesn't. And they have all been programmable for a very long time now. And almost without exception they have more FLOPS than all of the CPUs in the same machine put together, yes, even the integrated GPUs.
Re:500'000€ Prize for Compressing Human Knowledge by TheRealPomax on Thursday September 14, 2023 @10:14AM (#37509387)
In which case: https://news.ycombinator.com/item?id=37502782

And 20 years changed A LOT of things, turns out the folks who can't afford a deskop or laptop don't need to, they have a call phone. And even cell phones have GPUs that are more powerful than the CPUs of 20 years ago.

Re:500'000€ Prize for Compressing Human Knowledge by caseyavila on Wednesday September 13, 2023 @06:57PM (#37502768)
I do think it's interesting that recent submissions use nearly the entire 50 hours. I wonder how much better people could do if faster hardware was allowed.
Re:500'000€ Prize for Compressing Human Knowledge by AnotherGoodName on Wednesday September 13, 2023 @07:09PM (#37502868)
Absolutely they could. All the entries so far can do even better with more RAM. That PAQ entries have command line flags to do this. That's already known in fact.

What this is looking for is fundamental improvements, not "i brute forced a known way to win this competition".

Re:500'000€ Prize for Compressing Human Knowledge by londons_explore on Thursday September 14, 2023 @02:31AM (#37505678)
An element of compression is usually a search problem - 'lets try all these ways to encode the data, and see which is smallest'.

Therefore to maximize compression, you tweak parameters to search as hard as possible, and therefore use all the time.

Re:500'000€ Prize for Compressing Human Knowledge by nier on Wednesday September 13, 2023 @07:02PM (#37502799)
Maybe commodity servers with integrated and probably never used graphics capabilities that no one would actually describe as a GPU?
Re:500'000€ Prize for Compressing Human Knowledge by ozzmotik on Wednesday September 13, 2023 @07:08PM (#37502863)
anecdata: for my entire life, I have neber, until about 3 weeks ago, owned a laptop with any type of gpu other than an integrated graphics card built into the motherboard. for most of my life in fact I have never really had any modern or really serviceable hardware, as I've basically inherited all of my compute devices secondhand never had the money to just go out and buy a decent rig (or to even put one together for the matter, perhaps tbst will elucidate the economic sector which I have basic y always occupied. well, not always, as now that I'm homeless, I somehow went even LOWER! )

though it truly does excite me that I finally have something with some type of GeForce branded x graphics device inside of it, now I can finally run local machine learning tasks and not have to cloud it out

Re:500'000€ Prize for Compressing Human Knowledge by anon____ on Wednesday September 13, 2023 @08:33PM (#37503438)
The first winner DIDN'T EVEN HAVE A COMPUTER!

    Q: What if I can (significantly) beat the current record?

    A: In this case, submit your code and win the award and/or copyright your code and/or patent your ideas. You should be able to monetize your invention beyond the HKCP. This happened to the first winner, a Russian/Ukrainian who always had to cycle 8km to a friend to test his code because he did not even have a suitable computer, and who now has a lucrative job at QTR in Canada.
http://prize.hutter1.net/hfaq.htm#fame
Re:500'000€ Prize for Compressing Human Knowledge by tyingq on Wednesday September 13, 2023 @08:56PM (#37503592)
>lucrative job at QTR in Canada

Google failed me on this one. What company/entity is QTR?

Re:500'000€ Prize for Compressing Human Knowledge by anon____ on Wednesday September 13, 2023 @09:16PM (#37503732)
Quantum Technology Recruiting Inc. (https://ca.linkedin.com/company/quantum-technology-recruitin...)

Our guy, Alex Rhatushnyak, is listed as employee.

BTW, this is the first result on DuckDuckGo. (https://duckduckgo.com/?q=qtr+company+canada)

Is it time to switch? ;)

Re:500'000€ Prize for Compressing Human Knowledge by stronglikedan on Wednesday September 13, 2023 @08:55PM (#37503588)
My two years old high end productivity Thinkpad has an Intel Iris XE display adapter, which I would hesitate to call a GPU.
Re:500'000€ Prize for Compressing Human Knowledge by concurrentsquar on Thursday September 14, 2023 @12:00AM (#37504831)
This competition was from 2006.

The consensus among AI experts (really just everyone) during the 1990s/2000s was that:

- AGI could be achieved by giant GOFAI (usually expert systems/knowledge bases) projects (like OpenCog and Cyc).

- ... or that AGI development is limited by lack of knowledge about key insights (mostly symbolic/rational) into intelligence, not computation/data. (IE AGI was viewed similarly to proving that NP = P or other very-hard math/computer science/psychology/philosophy problems).

- ... or that AGI will be achieved through brain scanning/connectomics.

- ... or that AGI is impossible.

Nobody (except for LeCun and Schmidhuber) paid much attention to neural networks until AlexNet (2012) showed that they could be ran and trained at fast speed and beat the symbolic competition. In the 2000s, only a real, actual psychic would be able to tell you that LLMs would be a valuable path for AGI research.

Here is a list of various expert (and "expert") perspectives on AGI during the 1990s/2000s (notice how nobody is talking about neural networks, and they are definitely not talking about anything remotely close to a LLM or transformer):

> Copycat is a computer program designed to be able to discover insightful analogies, and to do so in a psychologically realistic way. Copycat's architecture is neither symbolic nor connectionist, no was it intended to be a hybrid other two (although some might see it that way); ... [describes a very symbolic system to our modern day eyes, though it was not really symbolic to 1990s AI researchers]

- Douglas Hofstadter and Melanie Mitchell, Fluid Concepts and Creative Analogies (Chapter 5), 1995

> Interviewer: Are you an advocate of furthering AI research?

> Dennett: I think that it’s been a wonderful field and has a great future, and some of the directions are less interesting to me and less important theoretically, I think, than others. I don’t think it needs a champion. There’s plenty of drive to pursue this research in different ways.

> Dennett (cont): What I don’t think it’s going to happen and I don’t think it’s important to try to make it happen; I don’t think we’re going to have a really conscious humanoid agents anytime in the foreseeable future. And I think there’s not only no good reason to try to make such agents, but there’s some pretty good reasons not to try. Now, that might seem to contradict the fact that I work on a Cog project [sic] with MIT, which was of course is an attempt to create a humanoid agent, cogent, cog, and to implement the multiple drafts model of consciousness; my model of consciousness on it.

> Dennett (later): [Cog is intended as a] proof of concept [for AGI]. You want to see what works but then you don’t have to actually do the whole thing.

- Daniel Dennett, Daniel Dennett Investigates Artificial Intelligence, Big Think, 2009

> [Context: Marvin Minsky had a speech where he talked about how expert systems don't work, because they do not have any common sense (and the only solution seems to be to create a giant AGI project (without automatic data gathering)).]

> Only one researcher has committed himself to the colossal task of building a comprehensive common-sense reasoning system, according to Minsky. Douglas Lenat, through his Cyc project, has directed the line-by-line entry of more than 1 million rules into a commonsense knowledge base.

- Mark Baard, AI Founder Blasts Modern Research, Wired, 2003

> Section 1 discusses the conceptual foundations of general intelligence as a discipline, orienting it within the Integrated Causal Model of Tooby and Cosmides; Section 2 constitutes the bulk of the paper and discusses the functional decomposition of general intelligence into a complex supersystem of interdependent internally specialized processes, and structures the description using five successive levels of functional organization: Code, sensory modalities, concepts, thoughts, and deliberation. Section 3 ... [yada yada, this is old, wrong stuff]

- Eliezer Yudkowsky, Levels of Organization in General Intelligence, 2007, Machine Intelligence Research Institute

I could list more examples, but I have spent way too long on this post. What I will say is that Hutter probably had the most correct idea of how modern semi-general AI would work (from the 2000s). He figured out that compression is a extremely important component of intelligence > 10 years before everybody was doing LLMs. That is impressive.

I should probably write a blog post over this.

Re:500'000€ Prize for Compressing Human Knowledge by TheRealPomax on Thursday September 14, 2023 @10:13AM (#37509366)
Trying to figure out who you're replying to though, none of this has anything to do with compression algorithms.
Re:500'000€ Prize for Compressing Human Knowledge by concurrentsquar on Thursday September 14, 2023 @01:36PM (#37512149)
> Being able to compress well is closely related to intelligence as explained below. While intelligence is a slippery concept, file sizes are hard numbers. Wikipedia is an extensive snapshot of Human Knowledge. If you can compress the first 1GB of Wikipedia better than your predecessors, your (de)compressor likely has to be smart(er). The intention of this prize is to encourage development of intelligent compressors/programs as a path to AGI.

- Marcus Hutter, http://prize.hutter1.net/

Re:500'000€ Prize for Compressing Human Knowledge by TheRealPomax on Friday September 15, 2023 @01:35PM (#37526517)
okay, but why post that as a reply to a complaint about how GPUs can't be used and how an "i7" is not one specific CPU? You almost certainly wanted to just post that as its own comment instead of a reply to anyone.