News for hackers. Stuff that matters.

stealthcat

Re:A Multimodal Dataset with One Trillion Tokens on Thursday July 25, 2024 @11:10AM (#41069598)
Marketed as “multimodal” but actually texts and images.

Multimodal dataset should be multimedia: text, audio, images, video, and optionally more like sensor readings and robot actions.

Re: on Monday July 01, 2024 @10:27AM (#40846151)
This worldly life is no more than play and amusement. But the Hereafter is indeed the real life, if only they knew. [1]

Know that this worldly life is no more than play, amusement, luxury, mutual boasting, and competition in wealth and children. This is like rain that causes plants to grow, to the delight of the planters. But later the plants dry up and you see them wither, then they are reduced to chaff. And in the Hereafter there will be either severe punishment or forgiveness and pleasure of Allah, whereas the life of this world is no more than the delusion of enjoyment. [2]

[1] https://quran.com/29:64

[2] https://quran.com/57:20

Re:X.ai: $6B Series B on Monday May 27, 2024 @02:03AM (#40488029)
Unitree is better than Optimus.

Qwen et al is better than Grok.

BYD is better than Tesla.

Did you have some small toy experiments to prove this?
Re:Training great LLMs from ground zero on Wednesday March 06, 2024 @06:50PM (#39623350)
Should list most of the technical debt accumulated so far and rank them. At this stage, lots of corners have been cut.
Are there decent wind models and motor models if you want to model drones
Re:The VAE Used for Stable Diffusion Is Flawed on Thursday February 01, 2024 @09:53AM (#39216604)
Imagine all those millions of $$$ GPU of cloud credits for training, only to overlook this bug
Re:Ask HN: Taking a hiatus to learn more ML? on Thursday February 01, 2024 @09:35AM (#39216378)
Consider applying PhD program to some ML labs in Europe other than UK and France, they pay livable salaries like junior engineers. Better if in Swiss since the salaries are much higher.

European ML labs generally have less prestige than US/China ML labs but they pay much more humane wages even for people with family.

Re:Ask HN: Managing Myopia as a Developer on Thursday February 01, 2024 @04:14AM (#39214067)
Eat lutein supplements and get as much sunlight and barefoot grounding
Kotlin Multiplatform.
Re:Passing nothing is surprisingly difficult on Wednesday January 17, 2024 @06:42AM (#39026443)
In ML the problem is passing 0D scalar tensor as 1D 1-element tensor.
Re:Thinking in an Array Language on Saturday January 13, 2024 @07:12PM (#38986087)
How do you trust numbers to “quantify extremely smart people”?

Interviewers know to hire/no-hire within first 10 minutes of talking to candidates.

Similarly, the best LLMs nowadays are probed only with some prompts.

Just last time I see LLM midwit meme on quantifying best scores on all benchmarks vs idiots/geniuses who just prompt something for some while.

Do not try to put numbers on everything.

More like GodotDesktop
Re:Fastest Autograd in the West on Thursday January 04, 2024 @10:26PM (#38875359)
Fun exercise… but for such use cases, people use Casadi and DrJit

https://github.com/casadi/casadi https://github.com/mitsuba-renderer/drjit

DrJit is made by same author of pybind11 and nanobind.

Ask HN: Traning LLM directly on file bytes

Multi-modal LLM like PaLM, GPT4, MiniGPTv2 relies on data encoder (image, speech models) to map data to token embedding space.Is there any attempt to directly train on file bytes? Make the only vocab of LLM as base-2, base-8 or hexadecimal, then do next token prediction on this.I know some attempts have been done like MEGABYTE and Charformer but some may have is not directly learning from bytes...

( Read More... | no comments | 2 points | on HN )
Re:Ask/Tell HN: Our Problem with Identity on Monday December 18, 2023 @03:16AM (#38680237)
Isn’t this more of symptom of modern western values?

Try a book by Muhammad Asad (Leopold Weiss), like the Road to Mecca.

Every year during hajj in Mecca, there’s profound view of sea of humans wearing white from vast different backgrounds, wealth, health, ethnicities, continents. You can find Japanese, Africans, Australians, Russians, Indonesians, etc, variety of skin colors and builds, in one place.

Put the world in your hand, not in your heart.

Kotlin multiplatform
Re:Ask HN: What book has been most influential in your life? on Thursday October 05, 2023 @08:03AM (#37777519)
Quran
Re:Ask HN: Where do I find good code to read? on Tuesday August 29, 2023 @05:06AM (#37305067)
Good code? Code reading is more like understanding different cultures. Read big tech codebases meta and google, to startups codebases like openai and comma.ai, you will see.
Re:Why Tcl? on Tuesday June 20, 2023 @09:18AM (#36403526)
Because you work in semiconductor industry.
Software engineering is hard problem. How is Julia in multiple authors codebase? A language need to consider ergonomics of multiple users working on same thing. Reading >> Writing.

C++/Python still hold ground in this regards sadly. One side for low-level control, another side for glue code.

Related: https://kidger.site/thoughts/jax-vs-julia/

Re:Ask HN: How to have a long career in tech? on Thursday February 16, 2023 @09:45AM (#34819265)
Not OP, but in my experience, two methods actually work for me

1. Use SRS (spaced repetition system) like Anki to learn a difficult-to-read language. Slowly but consistently ~1 hour a day learn mandarin or japanese. Personally I managed to get JLPT N3 this way in ~2 years.

2. Specific to muslims: Memorize the Quran with the Itqan method [0]. Literally intense drilling by reciting 100+ times for a page of Quran a day by looking the page then 100+ times not looking. Can spend 2-4 hours easily for a page.

[0] https://hifdh.weebly.com/mauritanian.html

Personally Itqan method is way harder than SRS but retention is faster and stronger even without revision for months.

Depends on subreddit. Birds of feather flock together.
Re:Ask HN: Burnt out from big tech. What's next? on Friday February 10, 2023 @08:31AM (#34739377)
Join HVM and Kindelia’s Discord. https://github.com/HigherOrderCO/HVM

Read Interaction Nets paper

Re:Ask HN: Can you crowdfund the compute for GPT? on Friday January 13, 2023 @10:14AM (#34368833)
Last time I saw a project aiming to be like a fast decentralized REPL but with blockchain.

https://github.com/Kindelia/Kindelia-Chain

Re:The Mathematical Hacker on Saturday December 24, 2022 @08:53AM (#34116782)
Seems like the so called Lisp hackers are Common Lisp users.

Before Matlab was cool, Lisp is VERY maths-heavy and engineering-heavy. Symbolic equation solvers, robot controls, neural nets, etc.

Even Julia which is very Matlab-like has roots to Lisp.

Re:Ask HN: How can I rediscover my passion about computers? on Friday December 09, 2022 @01:14AM (#33918144)
Find that “itch” that you really want to scratch.

Maybe building self-hosting language/compiler, or a toy linux-like kernel, or a physics engine, games then publish to itch.io, a small numpy-like library that can do autograd, whatever.

Once you start it really is hard to stop.

Older »