hdjdbdirbrbtv

I am playing with open weights models at home and yeah they are like that ... I use Claude 3.7 @ work and yeah it is a lot better ... Sometimes it will flub things but it also can write large amounts of code ... Mostly how I want (the pareto principle comes into play for the parts I don't want though).

So for me, the future will tend towards this ... Currently the tech is early days, we have no way to steer thought.. We have no way to align it to our thought processes... But eventually we will get to I want x pls make and it will be able to do it well.

As a professional rust dev, I will say this, I don't notice. Generally because I am doing partial builds mostly. And when I am not the m1 max I have is fast enough to compile the project.

Really there are much larger issues that I have to deal with that build time does not affect.

But also unfathomably large because of the speed at which you could consume it was measured in kbps....

Project Binky (YouTube) ep 39 or 40 did this with their CNC mill

Respectfully, I disagree. An llm in my mind is a new compiler. Just it takes natural language and produces code.

So the issue with genetic algorithms / genetic programming is you need a good way to handle the path the population takes. It is more reinforcement than y = f(x) for deep learning f() is what the nn is computing. X and y is the training data.

Finding a good scoring algorithm is hard as it is so easy for a GA to cheat...

Source: experience

Yes, one is committing knowledge to neurons, the other is commuting knowledge to short term memory.

Put another way, if you took a llm with random weights. Do you expect you could rely on context alone?

Lol thanks for the correction! like I said... it had been 20 years.. I misremembered the amounts :P

Are you talking about teaching in the context window or fine tuning?

If it is the context window, then you are limited to the size of said window and everything is lost on the next run.

Learning is memory, what you are describing is an llm being the main character in the movie Momento, I.e. no longterm memories past what was trained in the last training run.

IIRC ( and it was 20 years ago now that I learnt this) the brain uses 20% of the body's resting energy usage. Most of that is keeping neurons polarised to the outside (ion pumps need ATP!!!).

The body uses 25w resting and thus the brain is about 5w.

Source: biology degree but like I said please take with the same amount of weight as a hallucinating LLM.