Koromix

Interesting, it's similar to what I've done recently in Koffi 3, which is an FFI package for Node.js. I made my own C FFI layer, I did not use libffi. This works great, and got me close to statically-implemented NAPI modules.

Benchmarks are here: https://koffi.dev/benchmarks

It's still a little different because in my case, the instructions tend to do two things: decode JS value and prepare register/stack. For typical functions, only a few instructions have to run, with minimal overhead. So, for example, I have a PushBool instruction which calls napi_get_value_bool() and then puts the bool at the correct offset (pre-computed) so that it ends up in a register or on the stack.

A function like int atoi(const char *) ends up with only two bytecode instructions:

  - PushString
  - RunInt32 (combined macro-operation that defers to assembly to set up registers, call the function, and then directly decodes the value)

Or another exemple, void *memset(void *ptr, int value, size_t size) only needs four instructions:

  - PushPointer
  - PushInt32
  - PushUInt64
  - RunPointer

I've coupled that with a tail-call direct-threaded interpreter, with Clang's __attribute__((preserve_none)) ABI, just like Python did recently: https://github.com/python/cpython/issues/128563

The official solutions, node-ffi and node-ffi-napi, are extremely slow, with an overhead hundred of times higher than it should be. I don't know what they do to be so slow.

I'm making my own FFI module for Node.js, Koffi, as a much faster alternative. You can see some benchmark here, to compare with node-ffi-napi: https://www.npmjs.com/package/koffi#benchmarks

I think that in a proper API, we could have:

  int process_handle = spawn("/bin/true", SP_PAUSED); // create process but don't execute, and return a handle
  // most API would take a process handle, thus you could do stuff (such as prctl) on the new process
  unpause(process_handle);

There's a bit of a move in this direction in the Linux API with the PIDFD stuff.

Interesting, thanks for the tip! I'm gonna give it a go during the weekend.

I share this opinion as well. For centralized-repo style, which is what most teams need, the Git model has three problems:

- It is hard to use. There's a reason most teams have a single or two Git "experts" to help everyone else resolve repository / merge / pull / whatever problems that happen regularly. The onboarding experience for developers unfamilir with Git is very bad, and many of them end up never able to use it. If you have non-developers (e.g. artists), it is even worse.

- Binary files. Yes, there is Git LFS, but there are many small integration problems when you try to use it, and the simple fact that Git needs something special to manage binary files is a problem. And no, storing binary files outside the repository is NOT a good solution. In games for example, the binary files and the engine usually evole together and if you want to checkout an old version, the code and the assets need to stay in sync. You want to version the game, not just the engine.

- Partial (subtree) clones and permissions. There's a reason monorepos are popular: they're much simpler to manage. Unfortunaly, because Git barely does partial clones and cannot manage permissions (inside a repository) at all, monorepos are much more limited than in SVN or P4. That's why many projects end up divided in multiple repos, but this brings all sorts of problems and a large administrative overhead for all developers.

No, I mean that the scale of whale oil use was very very small compared to the scale of fossil fuels. So small, in fact, that if whale oil was on the graph I've linked above, you would barely see it moving above 0 in the 19th century.

Also, whale oil was not really used as an energy source anyway, except in lamps.

So it's completely irrelevant to my original point, which is that by and large, until now, in the thermo-industrial civilization, new energy sources have not been used to replace exisiting ones. They have been used to 'grow' the economy (more machines, more people, more production).

Maybe this graph can convince you: https://ourworldindata.org/wp-content/uploads/2014/03/global...

The volume of whale oil used back in the 19th century is laughably anecdotic compared to petroleum.

When we talk about our energy needs, scale matters a lot. So yes, when it comes to climate change and energy use, you can ignore the rest of human history completely.

https://en.m.wikipedia.org/wiki/fossil_fuel?wasredirected=tr...

Worldwide, most of the energy that comes from wind and solar is a tiny fraction of the primary energy supply mix. Where it's growing it's mostly used for additional energy, not to replace the exisiting fossil-fuel consumption. This does not help one bit for climate change.

First graph at the top left (which is actually energy consumption, close enough): https://www.bp.com/en/global/corporate/energy-economics/stat...

The important thing in this grap is the absolute values of each source, not their relative importance. You'll note that despite renewable growth (of which solar and wind is only a fraction, there's also biomass and various other stuff), fossil-fuel consumption is still growing faster. Indeed, historically, new energy sources have never really replaced 'older' ones. Each time, the new sources just provide additional power on top of the old ones.

So much for 'decoupling'.

Edit : the US energy article on Wikipedia has an interesting graph that shows something similar (except for a partial switch from coal to NG): https://en.wikipedia.org/wiki/Energy_in_the_United_States#/m...

Where the hell do you get your news from? The "massive wind and solar boom" exists only in the media. There's a rule of thumb you can follow here: the importance of a technology / energy source / whatever is inversely proportional to the number of articles about it in the general media (and on HN for that matter).

Geothermal, wind, solar, heat, tide, etc. currently amount to a staggering 1.5% of the total primary energy supply of the world. Coil is at 28%, oil is at 31%, natural gas at 22% (and climbing), nuclear at 5%, etc. CO2 emissions continue climbing year after year, and we should have started going down more than twenty years ago.

Our agricultural system is completely dependent on fossil fuels (oil). Because of that it is mostly unsustainable, and also because it tends to destroy fertile lands (topsoil loss is kind of a big issue).

You are kind of right that the current system can support a lot of people. Unfortunately, almost none of it is sustainable. So making more people is really not a good idea at the moment, because the situation will change drastically over the next 80 years.

You are right, there is pretty much nothing in biology that is. Humans are no exception, but until recently it was not a big problem because we did not have the power to significantly damage the climate and the biosphere (though the megafauna would probably disagree).

Fossil fuel powered technology has changed that, but we are psychologically and socially ill-equipped to deal with its transformative power.

> People are scared of nuclear energy for the same reason that they're scared of taking an airplane. Even though it's technically and statistically very safe, the perceived risk appears much greater. [...] I'm more worried about global warming than Fukushima and I'd gladly trade even a dozen of Fukushima-type incidents in the next decades (highly unlikely) if it could stop global warming and its dire, hard-to-revert consequences.

I could not agree more, but unfortunately global warming suffers from the exact opposite effect. The perceived risk appears very low to most people, because it is not very spectacular on human time scales. Even though it is by far the greatest existential risk we face.

Many / most people exisiting now also fail to grasp how bad the future will get on our current course. They fail to realize that modern life is anything but a short-term artefact of fossil fuel gluttonery (every person in the first world has uses the equivalent of 100 fossil fuel slaves. It's easy to mock human slavery when you can burn such a dense source of energy instead) and mild stable climate, both of which are likely to go very wrong this century.

The decision to put more weight on the present is mostly made in ignorance and wishful techno-optimism. Many people still think that we're somehow building a cheap-energy no-death space-faring future, or they're not thinking about it at all (though that does not stop them from making children). Going to space to escape the hard reality of life on earth and then never dying is basically the techno-optimist's replacement for the now outdated concept of heaven.

People on HN tend to go for techno-optimism. This is readily apparent when the AI, singularity delusion is treated as a grave and imminent civilizational problem but fossil fuel, energy shortage and climate change are dismissed with "PV will solve all of this. Evil greedy subsidy-loving coal and oil companies are the problem". Our fossil fuel powered civilization would quickly beg for these "evil" companies to resume their activity if they somehow decided to stop providing oil for a week.

Finally, this preference is also made at the expense of most of the wildlife and biodiversity that exists now (or not long ago for a significant chunk of it), which would very much like to continue exisiting too.

> Ultimately all life on Earth will die when the sun burns out or otherwise significantly changes its output. The only hope humanity has to outlive this is taking to the stars, and that won't happen without consuming a lot of resources.

There's 500M to 1 billion years left for complex multi-cellular life on earth, due to increased solar output. That's around 100000x to 200000x longer than human recorded history. So clearly, not an imminent problem any rational human needs to worry about.

Trashing the planet in a few decades for such a far away "problem" is absurd. All it does is significantly shorten the time we have here. We could decide that 500M to 1B years is enough, stay here and enjoy all the time left.

If we really want to escape our fate on earth, well we could simply take our time to slowly develop ways go to space. Assuming that it is possible for humans to reach anything beyond the solar system, which may not be the case. There are practical physical and thermodynamic considerations that may prevent us from ever colonizing much in space, even if we were to try hard. In which case, preserving earth would not just be the best thing to do, it would be the only thing we can do.

> If you want to look at it in terms of natural resources, personally I think irrational fear of nuclear power is the biggest problem facing the world. Modern reactors pose almost no risk to the biosphere compared to coal, oil and the like, nor do they contribute to global warming. More people die every year from coal-burning related illness than have ever been killed by nuclear power accidents.

No disagreement here. Though it only addresses some of the issues we face, nuclear fission is probably the only semi-viable alternative to fossil fuels.

Maybe, but I hope there are better arguments for the current system than "it works better than the Soviet Union". All you have done here is establish that an extremely dysfunctional economy can either waste or fail to use its resources, which would mean that America and Western Europe are not completely dysfunctional. This is a pretty low bar.

And as you said, we're talking short term here. Very short term. On slightly longer terms, it really doesn't look so good. Indeed, it takes an absurd level of denial / techno-optimism / magical thinking (whichever you prefer) to look at paleoclimatology and current climate indicators and think it can go on for much longer without global catastrophy.

At the end, if the best we can say of globalization is "For around 40+ years it was great fun for a lot of people. Well, for the most privileged countries anyway. And then it collapsed, and took a great chunk of the biosphere and the human population with it"... I don't see much to celebrate.

Burning through the vast energy and natural resources at our disposal is what has allowed us to "lift" so many people out of poverty. "Proposerity" and "progress" are not a miracle of economic thinking, or technology. We owe all of this to fossil fuels, nothing more, nothing less.

In fact, you'd need to slap an extremely inefficient economic system on top of this mind-blowing amount of resources to not enrich the global population at least a little bit. Or you'd have to purposefully design it to work that way.

Probably nothing.

This is wide-scale bike-shedding[1], basically. The real problems our unsustainable civilization faces (population overshoot, energy and fossil fuel shortage, ecological collapse, unsustainable agriculture, climate change, and so on) are between hard to impossible to solve at this point. They're also actually scary to think about.

So instead we talk about the "easy" and trivial stuff first. AI and singularity happen to be a nice kind of scary, because hardly anybody really believes it's a serious threat. It's kind of like watching a scary movie. You get a bit scared, but not too much, because you know there's no real danger.

[1] https://en.wikipedia.org/wiki/Law_of_triviality

Past civilizations did not fail because they slowed down. They slowed down while they were failing. I mean, this is what failing is. They failed at getting enough food and energy to feed their growing population, they failed at managing their growing social and technological complexity. The "people of yesteryear" you reference may have been wrong, until now, for our civilization. But it turns out these people also existed in failed civilizations and guess what, then they were right. Read "The Collapse of Complex Societies" by Joseph Tainter, it's a great book and it makes much more compelling arguments than I ever could about this subject.

Human ingenuity is definitely limited. There are levels of complexity we probably won't manage to get past. More importantly, physical laws have limits, and these are not negotiable. For example, we very probably won't ever get beyond the speed of light. We won't produce energy from nothing. We won't cancel gravity. We won't stop heat death. We won't travel in time. We won't teleport, or beam up as they say. We know that with a relatively high degree of certainty because science has progressed a lot, so we know a lot more about what this universe can do for us, but also about what it cannot do. This is not comparable to a few centuries (or even decades) ago, because then we knew a lot less about both.

We've got 20 years to address climate change and fossil fuel shortage, probably less, before they become catastrophic. There are reasons to think it may already be too late without active measures (carbon capture and so on). 20 years is less than the time it takes to go from brand new technology (let alone lab experiments) to widespread commercial use, which means that technologies that don't yet exist are of no use to address this problem, and that rules out fusion (which hasn't even proven it can produce more power than it consumes, let alone at economically viable scales), among others. I'd say thorium-based fission plants are the only semi-viable bet if we want to continue BAU, because uranium is probably a dead end (there's just not that much that can be exploited with an EROEI > 1).

Note that this is only the energy problem. We also need to deal with over-population, climate change, sea level rise, resource depletion, soil loss, aquifer depletion, species extinction, collapsed fisheries, ocean acidification and so on. At the same time, and at a time when our political institutions are reaching unparalleled levels of passivity and incompetence. If we solve all these problems, remember that our economists and leaders still insist on the need of exponential growth on a finite planet, which means it would soon prove not enough and the new problems we'd face would be even worse.

If you are part of the people who think we're destined to a Star Trek future, I can imagine that the thought of collapse can be painful to you. It used to pain me a lot, but now I'm okay with it. I'd prefer for our civilization to survive, but like with terminal illness, there comes a time when acceptance becomes the only good option.

Our civilization will fail, but eventually the biosphere will recover (though with the amount of damage we do, it'll take more time than with past collapses). Then life will go on for about 500M-1B years, after that it'll be toast and it will most likely be over for life in this corner of the galaxy.

"The pace of the world is here to stay, people will just figure out how to sustain it with new stuff".

I don't know how you can think that. The pace of our world requires incredibly huge amounts of energy, which we get from finite fossil fuels. Either they become economically unavailable or climate change becomes so severe that we can't use what remains in the ground. Oil companies don't go to ultra-deep water wells and shale gas just because they are evil money-eating bastards (which is the prevalent narrative). They exploit these economically mediocre sources for the same reason garbage starts to look appealing when you are starved: you are hungry and the good food has run out.

Too many people assume that technology alone is what has allowed us to reach 7.5 billion people. This misses a significant piece of the puzzle, because in reality it's fossil-fuel powered technology. It's a crucial distinction, as shiny but empty trucks and tractors won't help you feed billions of people. Without fossil fuels we have no realistic idea how to feed that many people. So we'll probably continue burning them as long as we can, because the alternative (mass starvation) is even worse.

If we were a rational species, we could fix all of it. We'd massively slow down our economies, have very few kids for some time to reduce our population to more sustainable levels, rely on local food, stop traveling all the time, and so on. Doing it smart, we could reach a relatively slower but very nice and sustainable way of life, augmented by sparse but useful technology. Something a lot more sustainable that the Rude Goldbergian machine we call "modern life".

Since we're not rational and obviously won't do the smart thing, instead it'll degenerate to resource wars (over food, water, gas, etc.) and massive refugee crises way beyond what we're already seeing. Our civilization will stumble from one crisis to the next, blaming this or that ethnic group for what is happening, each time cobbling a half-solution together that seems to work for a time, but gradually it will sched most of the modern things we currently take for granted.

The myth of humanity going from caves to space is just that, a myth. In the real world, countless civilizations have risen and fallen, gaining and then losing most of their culture and scientific knowledge in the process. We've done it bigger than anyone before due to fossil fuels, which for a limited time have replaced our need for human and animal labor, but it is unsustainable and soon it will go away. Human ingeniosity plays a small part in the real story of our world.

The real story is that nature has kindly stored millions years worth of solar energy as fossil fuels and we've gotten so drunk on it for 200 years that we've started to think that we've mastered the universe, with soon to come galactic civilization, godlike AI and the end of death itself. The hangover is not going to be fun for a lot of people, and these delusions will not survive it.

That's interesting, thanks for the information, I'll probably spend a good portion of tomorrow reading up on this. Geoengineering is scary.

Honestly, I am extremely pessimistic about mankind's ability to work on such a scale. Too many people seem to assume global warming is still a far-off problem, and that for some reason humanity is destined to "progress" forever, even though we know of many civilizations that have collapsed before ours.

I'd just like more people to grasp the dire reality of the situation, and stop assuming that somehow technology and/or progress will save us no matter what.

27% is better, though from a quick look it applies only to the US, and in Europe it looks closer to 15%. I need to take time tomorrow to find good sources on this.

I'm not convinced by carbon storage, though I don't know enough about it to be sure. It is my understanding that it is either energy-hungry (so useless because we don't and likely won't have enough carbon-neutral energy surplus) or pretty slow (also useless). And completely unproven at large or even moderate scales, too.

It was an answer to parent's "It's simply a matter of scaling up." comment.

I agree with you that the exact same fact is true of nuclear. And wind too, while we're at it.

I don't think there's a way out of it: our technological civilization lives WAY WAY beyond its means, and that is made possible only by burning through fossil fuels, among many other non-renewable resources. When that dries up (economically speaking) or enough ecosystems have been damaged, our civilization will most likely fall apart. It'll be a slow and ugly process, it'll happen over decades, and it's already under way.

Renewables would have been great in a simpler and slower world. Hopefully, that's how the next global (if any) civilization goes in a few hundred or thousand years.

Photovoltaic capacity is expected to be around 4674 GW in 2050 (currently around 150, I think) [1]. Let's be optimistic and say it'll actually be around 20000 GW, just for the fun of it, and let's ignore all of the variability problems that solar poses.

Considering that PV's capacity factor is around 15%, that's around 26 PWh of annual (electric) energy production, around 50% of the electrical energy we use up in a year today, nevermind what we will actually need in 2050 thanks to the neverending growth we're apparently trying to go for.

IIRC, electricity represents 20-25% of our civilization's energy use mix, so solar should solve around 10-15% of our needs in 2050 under extremely optimistic assumptions and ignoring ALL of the variability, energy grid, energy storage, solar panel production issues, none of which are minor limitations.

2050 is around 10 years beyond the date we need to be carbon neutral to stay below 2°C [2], if we were to peak in 2020 and quickly ramp down our emissions. It's 20 years too late in business as usual scenarios.

[1] http://www.iea.org/publications/freepublications/publication... [2] http://www.realclimate.org/index.php/archives/2017/06/why-gl...

Of course, everyone that does not buy into the myth of the Star Trek future that surely awaits mankind has to be a religious fool.

Perhaps some of us have realised that, like many (now disappeared) past civilizations, our technological civilization is entering a state of decline caused by resource overuse and population overshoot, among other things. It is barely maintained by burning through hundreds of millions of years of solar energy stored as fossil fuels in the span of 200 - 300 years, a behavior that is also wrecking the climate that allowed us to prospere in the first place.

In the face of the severe ecological, energy, food and population crises that await us (and indeed are already here in many places around the world), the Mars fantasy sounds foolish to me. Even a seriously damaged earth biosphere will remain infinitely more hospitable to human life than Mars. Damaged biosphere ≫ no biosphere (aka. Mars, and all other planets of our solar system).

I don't hate them. I've used python more than once, and will continue to do so. And I think it's great teaching material. It's a good scripting language. But I think its drawbacks far outweigh its advantages for large projects.

All your examples are programming gimmicks, and I've yet to see stuff that solves actual hard problems. I'm not interested in programming for programming's sake. I want to use it to make my computer do useful stuff.

I don't consider the Ruby ecosystem to be a good example of much. Idiomatic Ruby code is much slower and usually not more maintainable than C++. Actually, it may even be worse thanks to dynamic typing, which makes refactoring much more painful than it already is in large code bases.

Well I guess it's good at making CRUD web sites. Hardly rocket science.

> Like, for instance, class generation, and extending methods.

This is precisely the kind of thing that leads to unmaintainable "magic" code, even though it can still be useful but with extreme moderation. So I don't see the point of making that a core feature of any language.

If you have any example to the contrary, I'd love a link to a good open-source project that uses these things extensively.

> OO, dynamism, late binding

None of these are required for ease of development. At least the first two often result in precisely the opposite.

HLLs are not required to focus on slow abstractions. For example, homogeneous arrays of tagged unions can replace inheritance most of the time. And they avoid breaking your code in 10 files and 20 classes (though for some reason this metric is seen as a good thing way too often).

> That doesn't sound like a cache miss. Knowing Android, A cache miss is probably the least of your worries.

My example was meant to illustrate the user input problem. From what I know about Android, the absymal performance is very much a case of "death from a thousand cuts".

> It's the same argument as always: perf vs. development speed. You can be in the C and FP loop, or the Lisp and JS loop.

The fast(er) languages we have are old and full of warts, and that makes them slow to develop in. The heavily used HLLs such as Python and Ruby were made by people who did not care much (at all?) about performance, and it shows in many design decisions. But here's the thing: we could have both at the same time. I don't buy this dichotomy.

You're looking at this problem backward. For example, you mention user input. Users may need a second to click or touch a button, but when they do the software should react instantly, and that does not leave you that many cycles. My smartphone's lock screen is my go-to example: most times it fails to follow my finger, and I barely have anything running on it.

Most of the dynamic languages are data and instruction cache-miss machines. They chase objects and pointers all around the memory.