News for hackers. Stuff that matters.
Re:The Santa Fe Time Series Prediction and Analysis Competition by AndrewKemendo on Monday August 07, 2023 @12:38PM (#37036852)
I seriously think the long term future of computing is massive scale time series correlation with analog hardware that doesn't do discrete processing with I/O and on/off transitors

I vomited my theory down years ago - though I probably need to refine it.

https://kemendo.com/research/streaminference.html

Transformers IMO are basically pointing towards time series predictions as being the root of epistemological grounding for "intelligence"

We probably need a new type of computer to realize this (nb. A previous good friend did bring up the point that digital sampling at a high enough rate could approximate biological "continuous" sampling - but I think it's flawed theory)

Re:The Santa Fe Time Series Prediction and Analysis Competition by mindcrime on Monday August 07, 2023 @02:41PM (#37038739)
I seriously think the long term future of computing is massive scale time series correlation with analog hardware that doesn't do discrete processing with I/O and on/off transitors

It's funny you say that. Over the past few months I've ordered copies of almost every book on analog computing that Amazon has, planning to do a deep dive into the "current" state of that, with an eye towards figuring out if there's some "there, there" to explore. I say "current" in scare quotes because there aren't a lot of recent books on the topic, given that analog computing mostly died out decades ago. There does seem to be a bit of a limited resurgence of interest recently though.

Transformers IMO are basically pointing towards time series predictions as being the root of epistemological grounding for "intelligence"

I've also been spending some time studying neuroscience and the literature on "spiking neural networks" based on a vague hunch that that and the analog stuff might tie together into something interesting.

All of that said, I do think that it makes sense to think about hybrid analog/digital systems since digital has so many advantages, especially in terms of flexibility and programmability. The question then, is something like "what's the best architecture that lets you best take advantage of both analog and digital styles of computation"? I won't pretend to say I know the answer to that, but that's a path I'm trying to explore a bit.

Re:The Santa Fe Time Series Prediction and Analysis Competition by AndrewKemendo on Monday August 07, 2023 @03:12PM (#37039356)
View the brain as a n-dimensional hyperspace receiving constant stochastic sequential input voltages from n-number of sensors.

Given that, lets consider your visual system, comprised of:

An analog collimator (lens) sending focused analog signals (light)...

...to light sensitive transducers (photoreceptors) ...

...Using the Visual phototransduction cascade [1] to turn light into voltage.

That signal is then transmitted through the optic nerve, pre-processed for reaction and early processing, and then hits the visual cortex where it flops around and other sensory systems combine their voltages trying to make meaning of things in increasingly abstract ways.

That is basically the same process used for all other senses as well, but just using different sensor methodologies

However the actual data uses the same medium of chemical-voltage transport throughout the PNS and CNS.

So using this construct, you need a hardware system that can remap pathways between sensors without rigid architecture and via processes that are emergent based on environmental changes.

I propose then that a liquid electro-chemical substrate that is materially additive (accretes more permanent power flows) depending on how much power (P = Volts * Amps) is flowing between nodes. That substrate then fills a cube covered in anodes and cathodes that represent individual sensor inputs/outputs.

It's basically recreating a human nervous system with a gel like substrate as the "brain" that can "rewire" itself infinitely as sensors are changed or otherwise.

I have a drawing of this but haven't uploaded it anywhere. If you're interested I'll scan it in

[1] https://en.wikipedia.org/wiki/Visual_phototransduction

Re:The Santa Fe Time Series Prediction and Analysis Competition by mindcrime on Monday August 07, 2023 @03:55PM (#37040077)
So using this construct, you need a hardware system that can remap pathways between sensors without rigid architecture and via processes that are emergent based on environmental changes.

Yep. Only in my case, my day-dreaming about this to date had me considering using digital components (digital switches, multiplexers, ADC/DAC's, etc) to build the configurable routing part. Not to say that I'm convinced that is the right approach, it's just some stuff that was bubbling around in my head.

I have a drawing of this but haven't uploaded it anywhere.

That would be great. And if you'd like to move this discussion to a different forum (aside from comments on an HN thread) feel free to ping me directly @ [email protected]. Or keeping it here is fine. Either way's good with me.

Re:The Santa Fe Time Series Prediction and Analysis Competition by AndrewKemendo on Monday August 07, 2023 @06:05PM (#37041587)
Drafting you an email now, thanks!

Also for anyone following along these are the images

[1] https://kemendo.com/Images/computer1.JPG

[2] https://kemendo.com/Images/computer2.JPG

Re:The Santa Fe Time Series Prediction and Analysis Competition by gregw2 on Monday August 07, 2023 @09:09PM (#37043172)
Re: "analog computing", I presume you have seen the following recent Wired survey/review of recent interest in it, but if not, see: https://www.wired.com/story/unbelievable-zombie-comeback-ana...

My perception is there has been a steady stream of analog circuit work in recent decades (not quite pure "analog computing", but related to it) due to the integration of analog signaling into cell phone chips that otherwise are digital (see https://en.wikipedia.org/wiki/Mixed-signal_integrated_circui... ); not sure if any of that is relevant to your explorations.

Re:The Santa Fe Time Series Prediction and Analysis Competition by AndrewKemendo on Tuesday August 08, 2023 @02:58PM (#37053847)
Thanks for those links
Re:The Santa Fe Time Series Prediction and Analysis Competition by mindcrime on Wednesday August 09, 2023 @02:23PM (#37066618)
You just reminded me, I've been meaning to look into getting some of those Cypress PSoC[1] chips to experiment with. I should go ahead and do that.

[1]: https://en.wikipedia.org/wiki/Cypress_PSoC

Re:The Santa Fe Time Series Prediction and Analysis Competition by tnecniv on Monday August 07, 2023 @09:54PM (#37043503)
It’s a fairly esoteric topic, but reservoir computing can be done in an entirely analog manner (analog reservoirs + a weighted linear sum via op amps; I don’t know how you dynamically update the sum weights in an analog fashion but I assume it’s possible) and some people work on it.
Re:The Santa Fe Time Series Prediction and Analysis Competition by AndrewKemendo on Monday August 07, 2023 @11:59PM (#37044356)
I actually hadn't heard of reservoir computing before so thanks for the pointer.