glangdale
The fastest way to detect a vowel in a string on any reasonable architecture (Intel or AMD equipped with SIMD of some kind) is using 3-4 instructions which will process 16/32/64 (depends on SIMD length) bytes at once. Obviously access to these will require using a Python library that exposes SIMD.
Leaving SIMD aside, a flat byte array of size 256 will outperform a bitmap since it's always faster to look up bytes in an array than bits in a bitmap, and the size is trivial.
If you have AVX-512, GFNI is faster for this task, but obviously many situations where you can't use it.
This has a huge impact on trying to hook actions, call-backs, etc (your "arbitrary procedure") to NFA states as you're frequently making many overlapping entries to these states, many of which go nowhere. Trying to figure out which entries correspond to which other entries isn't easy.
There are very stylized automata that are used in parsing that do interesting things beyond the finite automata space (like pushing things onto a stack), but they don't correspond to regex per se - instead they are generated from a grammar.
Doing the CSV version of the quote convention is actually wildly easier than the JSON version, as you can just treat a record like "foo""bar",hi,1,"hatstand,teakettle" as having 'left the quotes and rentered the quotes" at the double-quote spot when you're busy looking for ','. This isn't, of course, much help for normalization, but for the bit where you're hoping to simply find which ',' characters are separators, it's fine to pretend that there's a gap in your "quoted stuff bitfield" that happens at the "" in "foo""bar", as of course that gap isn't going to land on a ',' anyhow. So it's much cheaper than doing a tedious shenanigan to handle some crafty user who has hit you with 100 \ characters in a row (as opposed to 99 or 101), a la JSON.
IMO the cost of doing the CLMUL all the time vs taking a conditional to handle the 'I have no quotes today' case is pretty low. It makes for shorter and more easily understood (in performance terms) code. I wasn't allowed to take this to extremes in simdjson - we did handle the "common case" for UTF-8 validation (although I wonder if that's a very culturally determined notion of "common case").
Public universities are not well-protected in the United States from conservatives seeking to defund programs with which they disagree. One might be well-protected against being fired directly for saying something controversial, but if your whole department is closed, you can be gotten rid of, even with tenure.
https://www.boisestatepublicradio.org/education/2021-04-07/l...
There's an echo chamber out there that amplifies the notion that there is some sort of epidemic of "woke cancellations" and critical race theory etc. - this is heavily emphasized by conservative media. There's plenty of ridiculous behavior by students - e.g. at Reed College - which I am aware of a massive amount of left-on-left cancellation - someone I know had their career upended by the unlimited claims of the rather stereotypically woke students there. But stupidity at places like Evergreen and Reed will beget endless think pieces and have left cancellation magnified endlessly.
Meanwhile, conservatives' own role in cancellation is minimized to the point of ridiculousness: so Bari Weiss can straight-facedly claim to be a fan of academic freedom.
There's a quieter, but considerably more effective, campaign to defund entire departments among conservatives - the model is a narrow "university as career prep". This means getting rid of the humanities and social sciences in favour of business/law/engineering/CS (good for us, I guess?).
In Australia, the conservative governments have been fairly unabashed in raising the cost of a humanities degree (paid for via our "HECS" scheme) in a way that seems pretty much Amazingly, this gets considerably less press in the echo chamber than some jackass left students acting like jackasses. I wonder why...
I'll agree with you that most of the noisiest and embarrassing cancellations are left-on-right or left-on-left, but this a narrow slice of "actions taken against academics", and their noisiness is not indicative of their effectiveness. Some of these cancellations are simply part of right-wing grift: schedule a talk from someone with credentials designed to enraged the "woke left" somewhere on their home turf, then enjoy the newfound credibility when you get cancelled and get to be a Free Speech Martyr.
Your mind is going to get blown when you find out there exist whole departments at universities that aren't sciences, and that there's kinda of a tradition of academic scholarship that goes back centuries in the humanities.
Also, it's interesting that even the very ideological "FIRE" group (not exactly a neutral player in all this) shows roughly 60% of user-submitted cases being deplatformed "from the left", which is far from overwhelming.
A right-wing think tank with strong influence on Idaho legislators produces this gem of a report:
'The report doesn't just call for eliminating individual courses, however. It calls for the elimination of five whole departments — Gender Studies, Sociology, Global Studies, Social Work and History — that it claims are infused with "social justice" ideology. (A sixth blacklisted department has since been added: Criminal Justice.) Eight other departments (later updated to nine) are on a watch list of sorts, judged to be "social justice in training."'
"Arkansas Senator Tom Cotton, in a tizzy over the discontinuation by the estate of Dr. Seuss of six lesser-known books with racist content, had the audacity to gripe via social media that “the woke mob” is trying “to erase our history and cancel anyone who disagrees.” This is the same Tom Cotton who wrote a whole legislative act aimed at banning schools from teaching the 1619 Project, the initiative exploring how the United States was indelibly shaped by slavery—or what Cotton blithely describes as “the necessary evil upon which the union was built.” Cotton is not concerned about the censoring of history; he’s just picky about what parts of history get erased. What the Arkansas senator really means when he gets prickly about preserving “our history” is making sure the mythical white-supremacist recollection of American events is the only version schoolkids can read. Along with racist Dr. Seuss books, of course."
If this seems tendentious and unlikely, let's remember that Bari Weiss, now advocating for the University/grift that is University of Austin as a beacon of academic freedom, made her reputation attempting to get academics "cancelled" for having the wrong views on Israel.
There are a lot of people with a vested interest in beating the drum to claim that universities are a mess of "cancel culture", but I'd be intrigued to read some actual statistics on this, rather than heavily publicized anecdotes.
I tend to agree with the poster who complained this was some sort of blame-shifting ("he started it, mom!"). But the portrayal of normal academic life as being rife with endless left-wing cancel culture is a project being done for a reason. I don't like left-wing cancel culture either - and some of the leftiest people I know dislike it from a practical perspective (I know people personally teaching at elite institutions whose teaching has become unmanageable from constant weird student political demands). But any analysis of this that doesn't take into account the fact that universities are under pressure from the whole political spectrum is dumb.
It's also worth noting that "right wing cancel culture" sometimes just manifests itself by quietly shutting whole departments in favour of, say, expanding the "trade school" elements of a university. Just nuke the whole history department and double the size of business... obviously an apolitical act, right?
This is a good sentiment, but it's worth noting that the overwhelming majority of pressure to "cancel" books and academics is directed at the left by the right, regardless of what gets column inches and think-pieces.
I'm not going to denigrate noodling around with uxn, which seems quite neat, but it's not in any way shape or form something that most people would recognize as "research": it might be cool, and it might be an effective way of learning things, but it is not going to let anyone do anything substantially new they couldn't do before.
The point of "popularity" is not to win internet brownie points: it's to get outside your own head and find out what other people's workloads look like. If "modern" workloads sounds too much like a cliche (and I admit it does sound a bit like I feel like all research OS work should be centered around supporting an Electron stack or some hipster NoSQL database running in Kubernetes when I talk like that) - it could just as easily be "scientific workloads" or even, god help us, COBOL workloads.
Just something that you didn't yourself make up. That's noodling. That's a hobby. We're all really good at asking ourselves questions and providing good answers for them.
Anyhow, enjoy your noodling. I'm sure you'll have some absurd and reactive response to this, and yell and scream some more about how I don't have any substance and how your notion of research is Super Awesome and we all should invest more time playing with 8-bit computers, but I have some 256-bit and 512-bit stuff to get to, so have a nice day.
The reason that Plan 9 failed as a platform for exploring OS design (not strictly speaking "OS research" in a narrow sense) is that modern workloads didn't run there.
Honestly, who cares which platform supports a handful of processes running the moral equivalent of xclock, a few shells, a 1980s looking terminal and the occasional build? This isn't a proxy for anything that anyone cares about, and all modern hardware is ludicrously overpowered to do that. Give me a modern machine, and I could probably write you something that simulated a decent-looking 1989-level experience in pure Python (I am not saying this is a good idea).
It rapidly became impossible to test any meaningful ideas on Plan 9 because what consensus reality regards as "software" doesn't really run there. This filters down into potentially bad decisions about design of OS and runtimes. You can't test out ideas that come from anywhere outside your narrow circle of "the Unix room and miscellaneous fanboys and trainspotters".
For example, is "everything is a file" a good abstraction for a modern graphical interface of any kind? Who knows, we'll never see one ported to Plan 9, just a bunch of bitblt-level fingerpainting.
If Plan 9 had stayed popular (there's that word) enough to attract enough users to matter, maybe we could have explored these ideas. Not necessarily building a dumb clone of Windows and doing everything the same way, but having enough interoperability to make it a plausible daily driver.
I'm deeply affected by this because I loved Plan 9 in 1992-1994. I wrote multiple user level file servers for my silly honours project and for summer research projects. I built compiler passes for Thompson C while I was at the Labs between ugrad and starting my PhD (mixed caller save, callee save register allocation and a retargeting to the R4000). I wanted it to succeed because of the things that were good about it - but even then was starting to realize how worrying the NIH stuff in Plan 9 was. I started CMU still wanting to talk to people about Plan 9, but it didn't last.
My complaint comes from disappointment: Plan 9 was exciting because it was a clean sheet design, coming from the best judgement of a bunch of very smart people looking around at what was going on with hardware and software in around 1989. People poking the corpse now are working in the opposite of that spirit.
Everyone is free to do what they want, of course (I'd love to fire up a Commodore 64 or an Amiga!) but it feels like a tremendous waste of time to me (subjectively, of course).
As a bonus, you'd get a vast array of device drivers and be able to run actual computer programs used by more than a handful of people, which I hear is neat for attempting to build an innovative research operating system with some relevance.
I'm not going to argue that a standard Linux distribution is not a giant mountain of cruft, but imo it's the user space and system software that's alarming, not the kernel per se. And to the extent that the kernel is crufty: well, it's supporting orders of magnitude more things that Plan 9 ever did, and that's a good thing.
I share your irritation at all the things you outline, but you don't need to be running a wacky Thompson C codebase on top of an old kernel to fix any of these things.
I'm also far from convinced that there's any aspect of the Plan 9 design that should be accepted uncritically for a clean sheet new OS design. Saying "nothing has moved forward" is hyperbole, and plenty of things have changed since 1990, even if we accept your claim about OS stagnation as a given. I'm far from convinced that the insights of the Plan 9 guys circa 1989 are so wonderful and timeless that they are better than what could be done with a clean sheet design on top of modern hardware and a minimalist Linux kernel now.
Also intellectually incurious: assuming that P9 expired due to POSIX eating its lunch. My personal take on it is that P9 expired partly due to bad licensing decisions but mainly due to a preposterously shitty attitude to Other People's Code. I vividly remember people in the Unix room clustered around the one Windows machine there that could do radical stuff like "run games" and "run a browser". If just a bit more effort had been put into Howard Trickey's APE Plan 9 might well have kept evolving and become useful, but NIH was more important. Or "Invented Here, But Not By The Right People" (C++).
POSIX could have been an asset to Plan 9 - a target to track instead of having to emulate a bunch of disparate Unixen.
The thing that still appeals to me about Plan 9 as a concept - is the willingness to do a clean-sheet design and not be bound by the way that everyone else is doing things.
A better design would also not fetishize uselessness. It's one thing to have a cool environment but it's quite another to point-blank refuse to maintain a compatibility layer that allows people to run significant programs that people want to use. Plan 9 didn't have to allow the APE (ANSI POSIX Environment, a project from Howard Trickey) to run down into uselessness.
There are a lot of interesting system design questions that arise from letting people use computers the way they want to, for 2021 workloads. I think we probably know how to let people use shell, ed/vi/emacs/... and cc to make hand-built astronomy databases and manually paint a time of day clock into a window. But you don't get to find out the demands of modern computing without having enough compatibility layers to let someone run a database or a reasonably-complete 3D engine or a web browser on your system.
They are remarkably honest about it: "There is no advanced auto-completion of program names and flags, like hipster zsh and fish users might be accustomed to. But this really isn't an issue since Plan 9 has virtually no programs or flags to speak of, as you will discover soon enough."
Those with any kind of spidey sense for going down tech rabbit holes might also notice how front-and-center features for, ummn, connecting/emulating computers not running Plan 9 are in this guide.
Randomly cutting into the document presents you with a plethora of the usual fetishization of the fact that a nearly useless operating system, window manager, etc. doesn't take nearly as many LoC as something that more than a handful of people use, not to mention a giant list of things that you apparently shouldn't do, like use files made in Office: "There are a great many office suits (sic) on most operating systems, and other office utilities besides too numerous to count. So many are the choices in fact that it's easy to forget that "office" is just a fancy word for working with text."
It seems to close on some daft explication of how everything can be a database and how you don't need a real one, presumably after having invested thousands of words on how you don't need a spreadsheet, don't need a web browser, don't need an office suite, etc.
I cannot think of anything that captures the spirit of Plan 9 less than attempting to revive a dead, nearly 30 year old operating system and pretend that it's a useful daily driver. At the time Plan 9 was built it was a quirky and clean take on technology of the time, built on cutting edge and interesting hardware. At a time when distributed systems in Unix (not really Linux, then) were a tooth-grinding exercise in frozen NFS mounts and awkward, irritating incompatibilities, Plan 9 was a breath of fresh air. Honoring its spirit would be building a new operating system that applies this kind of simplicity to modern hardware (a hell of a task) - and learns from its mistakes, not trying to set up a homestead in this utter wreck.