BTC literally hit all time high this month, fyi.
House prices are at all time highs too. That doesn't mean the housing bubble never happened.
Unless you're going to claim that previous large drops in crypto were perhaps bubbles, but this time it's real...
But if that's not the claim, then I'm saying that the current value makes it's clear that it's not the end of a bubble.
Anything "brain-like" that fits into one single paper is bullshit.
Don't berate the authors for the HN submitter's carelessness.
That's standard in computational neuroscience. Our standard should simply be whether they are imitating an actual structure or technique in the brain. They usually mention that one. If they don't, it's probably a nonsense comparison to get more views or funding.
Just to illustrate the absurdity of your point: I could claim, using your standard, that a fresh pile of cow dung is brain-like because it imitates the warmth and moistness of a brain.
The brain-inspired papers have done realistic models of specific neurons, spiking, Hebbian learning, learning rates tied to neuron measurements, matched firing patterns, done temporal synchronization, hippocampus-like memory, and prediction-based synthesis for self-training.
Brain-like or brain-inspired appears to mean using techniques similar to the brain. They study the brain, develop models that match its machinery, implement them, and compare observed outputs of both. That, which is computational neuroscience, deserves to be called brain-like since it duplicates hypothesized, brain techniques with brain-like results.
Others take the principles or behavior of the above, figure out practical designs, and implement them. They have some attributes of the brain-like models or similar behavior but don't duplicate it. They could be called brain-inspired but we need to be careful. Folks could game the label by making things that have nothing to do with brain-like models or went very far away.
I prefer the be specific about what is brain-like or brain-inspired. Otherwise, just mention the technique (eg spiking NN) to let us focus on what's actually being done.
AI systems are software, so if you want to build something brain like, you need to understand what the brain is actually like. And we don’t.
> BDH is designed for interpretability. Activation vectors of BDH are sparse and positive.
This looks like the main tradeoff of this idea. Sparse and positive activations makes me think the architecture has lower capacity than standard transformers. While having an architecture be more easily interpretable is a good thing, this seems to be a significant cost to the performance and capacity when transformers use superposition to represent features in the activations spanning a larger space. Also I suspect sparse autoencoders already make transformers just as interpretable as BDH.