maciejzj

I wish these companies were more transparent so every third comment in this thread wouldn’t have to guess whether such shocking behavior is emergent or stems from a part of training.

BTW making DL models that generate weights for other models has been explored before in “hypernetworks”, though these have been far more simplistic and constrained.

FYI this phenomenon is also well known in my country in central/eastern Europe.

I would agree that a decent EE major is equipped with knowledge sufficient to grasp both Transformer architecture and gradient descent training. However, these elements do not fully cover the key parts of modern AI systems and also are not enough to predict the AI capabilities. You have to account for reinforcement learning from human feedback (RLHF) and with verifiable rewards (RLVR) which are crucial elements of the wider model creation/training process. These do not have obvious analogues in other engineering disciplines, but are still within your grasp if you are decent CS/EE/adjacent major. The issue is that you still can't deduce end result AI capabilities from first principles. You may have prefect understand of y = softmax(QK^T/sqrt(d_k))*V and it will give you no clue of the effectiveness of a given AI model. The range and complexity of training RLVR tasks may hint at the model capabilities, but the final results are very much emergent.

One of the better visualisations I've seen with the exception of Q/K/V weights and how they are presented. I believe they should be put more upfront since they are the core learnable parameter of attention. IMO, they should also be part of the "Head N of M" block since each head has its own weights (although they all can be collapsed into one huge matrix computation).

Had they issued an initiative on slowing down when they had the best model in the game, it would have been both more (anyhow?) effective and less dubious.

I know that this may be somewhat dramatised and even infantile, but my reflection is that in the world run by these reckless AI companies everyone looses. Navier-Stokes is solved but it feels like no one has won anything, controversy prevails, there is no glory in the math breakthrough. There is hardly anything to cherish, and even the guys at the top of it in OA who sit on the (supposedly) superhuman intelligence come across as massive losers and frauds.

I would advise any somewhat ambitious 17 year old to avoid tech and get into healthcare if they can stomach human interactions and bodily fluids. Sure, it is not all sunshine and rainbows, but there will still be plenty of work helping people who are ill or elderly. Even in the worst-case economic scenario, medicine will be a more socially rewarding and stable life path.

I've recently been connecting some machines to a new switch and my colleague has been monitoring web logs at the same time using Claude. He send me a Claude-generated observation that the machines that I was able to put my hands on simultaneously must be in different buildings due to high pings. Surreal experience.

On a side note, when I click the link it redirects me to machine-translated version of OpenAI website with completely botched meaning - the word “redacted” is translated to a false friend “redagować” which means to edit/refine text, not anonymize.

Can anyone more familiar with lambda calculus speculate why all models fail to implement fft? There are gazzilion fft implementations in various languages over the web and the actual cooley-tukey algorithm is rather short.

AFAIK – the input is (at most basic level) a matrix with L tokens (rows) and d embedding length (cols). The input tokens are initially coded into discrete IDs but they are turned into embeddings by something like `torch.nn.Embedding`. The embedding layer can be thought of as a "lookup table" but it is matrix multiplication learned through gradient descent (adjusted at train time, fixed values at inference time). The length of embedding (d) is also fixed, L is not. If you check out the matrix multiplication formulas for both embedding layer and attention you will notice that they work for any number of rows/tokens/L (linear algebra and rules of matrix multiplication). The context limit is imposed by auxiliary factors – positional encoding and overall model ability to produce coherent output for very long input.

When it comes down to the meaning of "bank" embedding, it cannot be interpreted directly, however, you can run statistical analysis on embeddings (like PCA). If we were to say, the embedding for "bank" contains all possible meaning of this word, the particular one is inferred not by the embedding layer, but via later attention operations that associate this particular token with the other tokens in the sequence (e.g. self attention).

Anecdotal evidence, but I do use Apple Maps in Poland and they work just fine for me, I guess the mileage may vary.

Is this gig-workification of the space industry?

Happy to see this architecture stil being handy for other people. Right now, I am using a heavily modified VAE for generation of synthetic satellite imagery. I am working with very limited dataset sizes, so I have chosen VAE over diffusion as a starting point.

Mind you, that regardless of your sentiment towards OpenClaw, not everyone is able to afford a sparse Mac Mini (especially given ram prices) and a ton of Claude tokens/super beefy GPU for local models to run this stuff. That's to the supposed "democratisation of knowledge and technology".

I don't mean "demo-like" in terms of poor technology. I meant that this technology doesn't yield products or services with global scales to an extent it happens in the US. Google Maps successfully uses both LANDSAT and Sentinel imagery. This is the wider problem of European failure to build companies/systems on top of technology.

ESA has done a lot of good for public benefit with the Sentinel-1/2 missions. I happen to work with remote sensing and Sentinel data has been my entry point to the field.

I hope that ESA keeps pushing forward even more. I am afraid that although Sentinel missions are great, ESA projects are a bit demo-like and limited in scope. Europe should focus on scaling up and applying the tech, not just proving that ambitious projects are possible for their own sake.

Mind you that the IT over investment sucked money out of other industries. My friends who chose different career paths do not seem to be particularly content either.

> Unfortunately, reading books for entertainment is ridiculous. You do not live in a log cabin on the prairie. You have Netflix, you have video games, you have TikTok, you have Twitter (you really spend too much time on Twitter anon). No one reads books for entertainment anymore, because paper is an inferior entertainment platform.

I see that this has already been criticised here, but to add my two cents, I believe that reading books has become one of the very last widely available (perhaps free?) entertainment media that is not anxiety inducing and mentally draining. 10 years ago reading books may have been a kind of a snobbish activity, but now it is one of very few things to engage in before you go to bed that will help you relax and get good sleep.

Yeah, irony put asisde, I believe that the reality is that people who already have power will consolidate it even more. Regardless if you were an average engineer, mid-level manager, consultant, whatever, you will be thrown under the bus any day. People can make up any identity and philosophy of entrepreneur/vibe engineer/manager/hardcore engineer/guy who run away to trades/anything but it doesn't change the fact that an average Joe is economically not viable in the current setup and he can't do anything particular to turn the tides.

Guys, is the current narrative that, due to AI, pure engineering is gone and we’re all supposed to be “managers”, or is it the other way around? I kinda lost the plot here.

If majority of housing is owned for profit by REITs or landlords they have such a leverage over ordinary person, that they can indefinitely hold the prices/rents at a level where they extract maximum of available resources from owning land while making sure people have enough processed food and cheap internet-provided entertainment that they don't rebel.

The prices will adapt, but the equilibrium will always be elite-oriented economy where accommodation of the masses is a second-tier goal.

BTW the whole plumber/electrician/whatever thing is ridiculous. I studied industrial automation before I joined tech. I checked the salaries for manufacturing maintenance engineer last month. The wages are a sad joke compared to the costs of living.

Yeah, kids change the landscape a lot. On the other hand, if you don't have any personal ties, its easier to grab opportunities, but you are unlikely to build any kind of social network when chasing jobs all over the country/world.

Either way, there is very little to no path toward "family + place to live + stable job" model.

TBH, it all feels like a huge gamble at this point. Neither skills, education, institutional ties, nor current employment can guarantee a stable foundation for life.

This hits harder depending on how much money, social capital, or debt you accumulated before this volatility began. If you’ve paid off your debts, bought a house, and stabilized your family life, you’re gambling with how comfortable the coming years will be. If you’re a fresh grad with student debt, no house, and no social network, you’re more or less gambling with your life.

I agree with you (and surprisingly so does Warren Buffet [1] if anyone doubts it). To add insult to the injury, I believe that people have lost some sense of basic self preservation instinct. Well being of ordinary people is being directly threatened and all that average person can do is to pick one of several social media camp identities you mentioned and hope that it will somehow pan out for them, while in fact they are at total mercy of the capricious owners class.

[1]: https://www.youtube.com/watch?v=yMD17EIk22c

I've been on a small adventure of posting more actively on HN since the release of Gemini 3, trying to stir debate around the more “societal” aspects of what's going on with AI.

Regardless of how much you value Cloud Code technically, there is no denying that it has/will have huge impact. If technology knowledge and development are commoditised and distributed via subscription, huge societal changes are going to happen. Image what will happen to Ireland if Accenture dissolves, or what will happen to the millions of Indians when IT outsourcing becomes economically irrelevant. Will Seattle become new Detroit after Microsoft automates Windows maintenance? What about the hairdressers, cooks, lawyers, etc. who provided services for IT labourers/companies in California?

Lot of people here (especially Anthropic-adjacent) like to extrapolate the trends and draw conclusions up to the point when they say that white-collar labourers will not be needed anymore. I would like these people to have courage to take this one step further and connect this resolution with the housing crisis, loneliness epidemic, college debts, and job market crisis for people under 30.

It feels like we are diving head first into societal crisis of unparalleled scale and the people behind the steering wheel are excited to push the accelerator pedal even more.

From the perspective of someone in their late 20s, the timing of this article feels quite off when entering 2026. The reality has become far different from the gist of what is presented here.

Career paths and opportunities have been getting broken and changing so much in recent years that I find it hard to plan anything. I don't even know what kind of "goal" is sustainable, let alone what the path towards it is.

The only sensible career that seems to offer a steady trajectory is medicine. Apart from that, my most successful peers were the ones that followed immediate money and speed-ran into owning some kind of real estate, which is a game changer. Besides that, people try to do as many side hustles as possible and diversify their income to save as much as they can and brace for a possible recession. I find it hard to apply any of these 8 steps in such volatile reality.

Great idea, I'm keeping my fingers crossed for this initiative.

I believe that the main challenge would be to get more traction and build a community. Hope you find a way to encourage as many people as possible to join the website.

My very minor nitpick -- I would add some kind of background colour to the main post list, something like #FAFAFA looks fine to me.

> Let’s be clear that taxing this bunch of people even more will have disastrous consequences.

?

> Top 10% includes everyone pretty much in HN.

I would suggest you take more caution with statements like that. There are people who read this page and don't have much luck in life. And foremost, what does HN mean in the scale of society? Should HN commenters high five and carry one because the are not the bottom part of the society?

> AI investment for job displacement is necessary for overall prosperity and efficiency.

AI will boost efficiency at expanse of society. I will give you some random examples to ponder about, maybe it will touch some people hearts:

1) What will happen to hundreds of thousands of managers and office workers in the US and EU who built mortgages and families when economy recovered after COVID if AI wipes out their jobs?

2) What will happen to hundreds of thousands of people in English-speaking counties in Africa who earned they daily bread by providing remote help desk services when AI takes heir jobs?

3) What will happen to hundreds of thousand of CS grads in India whose family spent life savings to give them "good" education in Bangalore when AI takes their jobs?

These and many more changes happening rapidly all at once?