This is so true, all commercial OSes that take i18n seriously do this, while most open source OSes' community (which communication revolves around English) decided that IMEs are an add-on for CJK people.

It's a pity, and this one reason is enough for Linux to be never adopted for ordinary users in the non-western world.

My biggest problem with IMEs in free software land is that we've had groups like Gnome lean on IME developers and push through their vision of how it should work -- even though the people pushing their vision don't use IMEs. I ended up migrating to FCITX just because it was the last hold out not to cave to pressure.

Microsoft and Apple have the incentive of selling to billion-user markets. The open source community, on average, appears to have demonstrated a lack of interest in opening up the user-base further (and expecting that user-base to just roll their own solution creates multiple catch-22 and tragedy of the commons problems).

Why one of the commercial open-source vendors hasn't taken this on as a core challenge, I do not know.

Technically true, but for practical purposes IBus is the standard. That's what Fedora and Ubuntu use out of the box. Firefox beta telemetry shows about 89% IBus vs. 11% FCITX.

> My biggest problem with IMEs in free software land is that we've had groups like Gnome lean on IME developers and push through their vision of how it should work -- even though the people pushing their vision don't use IMEs. I ended up migrating to FCITX just because it was the last hold out not to cave to pressure.

What wrong thing did the Gnome folks push for?

Some things evolved, some things didn't. O hi X Windows!

They are all very nearly the same age.

Windows 1.0 was fairly primitive, but Windows 2.0 supported overlapping windows (!) in 1987, coincidentally the year that X11 was released.

It's completely fair. Most of the developer community just doesn't care.

Does Windows really?

> while most open source OSes' community (which communication revolves around English) decided that IMEs are an add-on for CJK people.

IIRC, Fedora ships an iOS/Android like Latin-script IME with autocomplete. It's not mandatory, though.

> It's a pity, and this one reason is enough for Linux to be never adopted for ordinary users in the non-western world.

The Korean situation doesn't really generalize. Among CJK, AFAICT, the Korean IBus IME on Ubuntu 18.04 is pretty broken but the Japanese IME and the various Chinese IMEs appear to be at least OK. (It's a rather surprising situation considering that the Hangul part of a Korean IME should be much simpler than the Japanese and Chinese IMEs.)

Yes, the signs that even English is passing through Windows' IME infrastructure in Windows 10 is pretty minimal by default (in Desktop mode, Tablet mode immediately turns on a couple more), but at this point Windows 10 makes almost all of it opt-in. Some of it is referred to as accessibility tools from an English perspective, because IMEs are also useful for accessibility.

The "big one" IME for most English users is the Emoji Keyboard accessible with Win+. or Win+; (whichever you prefer). It's really interesting how well emoji have helped Latin script users with further understanding the complexities of Unicode, fixing old ugly bugs in Unicode handling, and even introducing some such users to an IME that they want to use (sometimes every day).

Under Settings > Devices > Typing > Hardware Keyboard you can turn on the IME "Show text suggestions as I type" even on a hardware keyboard in Windows 10, which gives you mobile-style auto-suggestions (you can also turn on mobile-style autocorrect even on a hardware keyboard).

No, API-wise the on-screen keyboard generates keystrokes for emoji (astral keystokes, multiple keystokes for multi-scalar-value emoji) even to an IME-aware app. In contrast, the emoji picker built into the Windows 10 Pinyin IME enters emoji via IME API.

> Under Settings > Devices > Typing > Hardware Keyboard you can turn on the IME "Show text suggestions as I type" even on a hardware keyboard in Windows 10, which gives you mobile-style auto-suggestions (you can also turn on mobile-style autocorrect even on a hardware keyboard).

Thanks!

That said, even if it isn't using the IME APIs directly in most cases, it's still useful as a teaching tool/analogy tool/example tool to English speakers of what an IME can be like to use, even if a nice-to-have for an English writer versus a necessary required tool for other languages.

Windows (10 at least) has native support for these IMEs; setting it up on Windows was much easier than on Linux (which doesn't even have a standard IME). Windows also comes with basic CJK fonts, which different distros may or may not have. On Debian I have to install noto-cjk or Adobe's source han fonts.

I originally tried fcitx but had to switch to ibus. Definitely not a great experience.

In my experience, Debian is worse than Fedora, Ubuntu, and openSUSE. Fedora is better than Windows 10 when it comes to IMEs: Fedora installs the IMEs by default. Ubuntu, like Windows 10, installs IMEs when you request the addition of an IME-requiring language. OpenSUSE gives you an IME for the language you use at install time if you install openSUSE using an IME-requiring language.

I haven't tried Debian 10, but when I installed Debian 9 _in Japanese_ with a _Japanese keyboard layout_ chosen, the installer didn't bother to set up a Japanese IME!

Fedora comes with an OK set of Noto CJK fonts by default. Ubuntu comes with a minimal set of Noto CJK fonts by default, but when enabling Chinese, Japanese, or Korean, Ubuntu drops more language-appropriate fonts on the system, like Windows 10.

Which is why it's infuriating when the built-in widgets of a platform don't include simple operations for obvious workflows like filtering characters, masked input, etc. That's what leads developers to have to roll their own implementation using keydown/keyup.

I can't tell you how many GUI frameworks have forced me to roll my own numeric input widget.

I have a note-taking system that I use to store all this data, and after I set it up years ago I looked at implementing an IME to search my notes and get a link to an entry, but every platform has its own byzantine IME API and I never got anything working. I've been thinking it might be easier to just make a cross-platform program that interacts with the clipboard rather than being a true IME.

In general one effect I'd like to see from having flexible IMEs would be allowing logging and chat could be left to separate apps. Imagine a birdwatching (or whatever) keyboard that lets you pick the birds you saw, their age, behavior, and punch it in while you're in the field, and have it output in structured form into whatever app is handy.

I can't agree here (excepting perhaps the unicode char replacement which a non-english speaker needs to comment on the viability of, vs actually having a non-english keyboard (excepting huge alphabets like chinese where that is used already AFAIK)).

Writing is fundamentally a thinking process, the text entry for a touch typist is relatively quick. If you are to propose expanding abbreviations, how much time do you expect to save? I mean, actually measured it?

> runs little scripts and inserts the output

What is the purpose of this?

> gives you quick dictionary/thesaurus lookups

I use those perhaps once a week or less. If you use it say 3X a day, you'd save little time, perhaps a couple of minutes.

Dunno what emmet powers are though.

MS word & libreOffice does some of what you want and the first time I install them, I spend several minutes tracking down each setting and turning them off - they drive me bonkers. They think they know what I want but they don't. Touch typists can hit many keys a second and kind of pipeline their typing. Having the input modified automatically is rarely useful IMO.

Your idea may be good but like many other ideas such as graphical programming, except in restricted cases they don't work. Perhaps if you measured it I'd be convinced but I can't accept it now as an obviously great benefit.

Not quick enough. I think faster than I type, and I type fast. It's fine until I start getting impatient with myself. Call it micro-impatience, a flash of irritation in which you're suddenly conscious of not having finished typing in the thought. It's distracting.

Honestly, I like parent's idea; a good chunk of Emacs's awesomeness and the reason people like me use it as an operating system is because of that - unified, fully configurable and expandable text-based interface. I often wish to have something like it system-wide, because standard UIs are very far from optimum ergonomy-wise. But then again I wouldn't trust Apple or Microsoft to do it right; they'd quickly find a way to dumb it down, or restrict the extensibility in the name of security.

I can totally imagine that. Underneath all the GUI layers, every operating system and application has (or has the potential of) a fully text-based interface. There's just no standard or integration, and tools that allow that (like a system-wide middleware) haven't caught on, I guess. Maybe in an alternate historical timeline, such a feature could have been a fundamental layer of an OS.

From the grandparent comment:

> a system [with powerful text input methods] that expands abbreviations, replaces easy-to-type sequences with proper Unicode chars, runs little scripts and inserts the output, gives you quick dictionary/thesaurus lookups, gives you emmet-style powers, etc., whenever you're writing and in any app.

Yes, yes - and the last point: in any app. I picture it like how TCL can script other programs, even ones that weren't designed to be "remote controlled".

Why do you say this? There is nothing fundamental about text. Is there something fundamental about text in Smalltalk? Or AmigaOS?

> There's just no standard or integration, and tools that allow that (like a system-wide middleware)

COM on Windows. Scripting interface to apps on MacOS. They're there.

This is a reality within Emacs (which really is a 2D text OS running lots of applications inside, including a text editor), and being text-based does play a role. When it's very hard to draw arbitrary pixels on screen and most of all apps deal with text, it's easy to make a large set of very powerful interface tools, and it's easy to pull data out of an app and put data into it, whether the app intended it to happen or not.

In the back of my mind, I sometimes wonder how something like Emacs could be made with modern browser canvas, to enable cheap rich multimedia, while retaining the ability for inspection and user-programmability. Introducing arbitrary GUIs is hard, because next thing you know, half of the stuff is drawing to canvas directly and it's all sandboxed away from you.

I think this is why it reminded me of TCL, specifically the "expect" command that can script apps that know nothing about it. From the Wikipedia page, the TCL Expect extension: "automates interactions with programs that expose a text terminal interface".

So how I imagine this "Emacs as an OS" paradigm you're describing, is that it mediates interactions with any and all apps that expose a text input/edit interface, to allow programmatic customizations.

Like I'd love to script my own shortcuts for Firefox (or other apps) - possibly with multiple steps, taking input from some config file, or sending a link to another app.. Or, as you mentioned, Emmet-style expansions that work in any input field or textarea..

If you want larger clumps of code then you have various options such as skeleton mode, but again that's something the user has to ensure happens - again they remain in control.

> But then again I wouldn't trust Apple or Microsoft to do it right

Oh hell yes!

For the sake of completeness, you can always make it automatic. All it takes is to add a function to post-self-insert-hook, and make it e.g. call dabbrev-expand if you pressed space twice. So you can have it any way you like - manual, automatic, semi-automatic. You're in full control.

> If you want larger clumps of code then you have various options such as skeleton mode

Yes. I currently use yasnippets for code. Still, my favourite yasnippet is one I use in comments - it expands "todo" into: "TODO: | -- [my name], 2019-10-29.", and similarly for "note", "hack" and "fixme". | is where the caret ends after expansion.

That's the kind of flexibility I wish my OS had. Unfortunately, it goes against the commercial interest of mainstream OS providers.

Actual example, was working with vis studio with another guy. Open a bracket and VS automatically added a closing bracket. That is fucking annoying and saves you a whole keystroke while breaking muscle memory and interfereing with our work. We had trouble turning that off.

> I use those perhaps once a week or less. If you use it say 3X a day, you'd save little time, perhaps a couple of minutes.

Maybe you only use them so rarely because they're not convenient to use.

For people learning, perhaps it could be a good thing.