Market forces do a poor job driving science. Its too long term, too risky, and you lose to much to the tragedy of the commons. All but the most trivial and mechanical research mostly happens due to government funding, or as a prestige byproduct of higher education (ie, not directly driven by their profit objectives). So in this case, maybe there is demand in a social sense, not a market sense.

Alternatively, large computer science companies may have just greased enough pockets to create regulatory capture of the process, creating the inequality in a non market driven way.

> Market forces do a poor job driving science. Its too long term, too risky, and you lose to much to the tragedy of the commons

Then how did we end up with things like nuclear fusion startups with hundreds of millions of dollars in funding? Market forces do a superb job of driving scientific innovation. We live in the most scientifically innovative period of our history.

I find that what people who say this sort of thing actually mean is “the things I perceive to be valuable are insufficiently resourced (according to me)”, rather than “things that are valuable fail to find resourcing”.

> Then how did we end up with things like nuclear fusion startups with hundreds of millions of dollars in funding?

By millions upon millions of dollars and man hours spent by governments and universities doing basic research. Markets are very poor at laying the groundwork required for scientific breakthrough, but quite good at leveraging that space to their own advantage, once it exists. This is, in fact, why the US government used to spend so much on basic research, so that its Capitalist economy could advance innovation more quickly.

> We live in the most scientifically innovative period of our history.

You’d have to state your measure for this. Scientific breakthroughs are not advancing at anywhere near the speed of the early 20th century. This may be because we’ve picked off a lot of the easy targets in the fundamental aspects that describe our reality, but would be contrary to this sentiment regardless.

I don’t really get why you think this isn’t a market solution, and I don’t really see where you think the problem is at all. Research institutions compete for fee paying students, endowments, and government funding. When they come up with marketable ideas, those ideas tend to make it to the market.

The fact that private capital can even find its way to funding ideas that have not and may never be proven to work completely disproves your point. Long term, high risk business propositions don’t seem unable to find investors. There is no tragedy of the commons here. If the market values something, there’ll be somebody willing to produce it. The only real room for disagreement is when the market doesn’t value something as much as you think it should.

> You’d have to state your measure for this. Scientific breakthroughs are not advancing at anywhere near the speed of the early 20th century

There’s more to science than quantum mechanics and a good approximation of gravity. It’s generally seen as rather obvious that our rate of innovation is increasing exponentially. I’m typing this comment on a pocket sized computer that’s orders of magnitude more powerful than the most powerful supercomputer in the world was on the day I was born.

https://www.nature.com/news/polopoly_fs/1.19431!/menu/main/t...

But what we're slowly learning is that a huge percentage, perhaps most of that "millions upon millions of dollars and man hours" ends up producing terrible papers that don't replicate (in science) or don't even say anything coherent at all (e.g. basically all of gender/critical race theory).

COVID-19 has been an excellent demonstrator of this: at a time when it's become clear that epidemiologists and virologists don't actually seem to know much about viruses, and know even less about stopping them, we have widespread riots and protests against 'systemic racism' that can't be found in any data sources. All this badness is coming from universities which look bloated, over-funded and nowhere near scrutinised enough.

Then finally you look at the tech sector, you look at commercial organisations at the cutting edge. The sad reality is they don't use much university output. They don't. Look at CS, most of the breakthrough papers are coming from corporate labs. The last 10 years saw multiple new programming languages take off and gain critical mass, none of them bear any resemblance to Haskell or other research languages popular in academia. A lot of the improvement in tech, whether it's electric cars, self driving cars etc, it's been coming from corporate labs.

China has four times our population, and we are falling behind on many fields. We need all the manpower we can get. On another note, I'd like to raise the point that bringing talent to the country also creates demand. Many of the next generation of startups will be biology-based.

THe bias I speak of is in regard to the attitude towards H1B. I have my own different bias, for in my field most people are in favor of it.

It's not necessarily a demand issue. It may be that companies are abusing the system to gain more such workers. I've certainly been part of hiring searches where qualified US candidates didn't get the job. It often came down to money. I'm in a higher cost of living area, so salaries tend to be higher than average. But the company was offering lower than the national average, much lower than the local average. The candidates from the US were often offered the job, then turned it down because the pay was so low.

"I can't afford or don't want to pay market rate salaries" isn't supposed to be a valid reason to grant an H1B visa.

I think there is something weird going on. I am heavily involved in my company's staffing process, and the vast majority of the referrals we get from our recruiting sources are from overseas (and often cheat and have cruddy IT skills).