News for hackers. Stuff that matters.
Re:Modernizing the OpenBSD Console by saagarjha on Monday August 31, 2020 @11:16PM (#24338203)
Interestingly, some of the newer commits seem to have come from John Carmack.

While it's not quite related to consoles, I've noticed that the non-GUI interface to a lot of operating systems often don't get a lot of love. Linux has had a fairly nice boot scroll, but I think part of that is because it's become an icon and as such is part of the public face of the OS; but on macOS for example it's an tiny awful experience because the text doesn't scale to 2x (you'd think Apple would be the first one to fix this…) and the output isn't well formatted at all–it's just the various subsystems spewing text as they start up. And a lot of modern systems use kind of old fonts that in many cases seemed to be picked as an afterthought…

Re:Modernizing the OpenBSD Console by formerly_proven on Tuesday September 01, 2020 @06:01AM (#24340074)
Re:Modernizing the OpenBSD Console by abrookewood on Monday August 31, 2020 @11:51PM (#24338372)
If I recall correctly, there was an article on HN not so long ago about Carmack discovering BSD. Edit: found it - https://news.ycombinator.com/item?id=23224584
Re:Modernizing the OpenBSD Console by hnlmorg on Tuesday September 01, 2020 @07:18AM (#24340577)
s/discovering/committing to/

Carmack has been aware of and actively involved in UNIX systems for a long time now. This isn't a recent discovery. However it has been recently noticed that he's committing to OpenBSD.

Re:Modernizing the OpenBSD Console by notaplumber on Tuesday September 01, 2020 @08:38AM (#24341114)
Contributing to OpenBSD, not committing. He is not a committer.. yet.
Re:Modernizing the OpenBSD Console by hnlmorg on Tuesday September 01, 2020 @10:29AM (#24342104)
Apologies, I hadn't realised there were a distinction. What's the difference?
Re:Modernizing the OpenBSD Console by kasabali on Tuesday September 01, 2020 @10:38AM (#24342222)
I guess it's about having a direct commit access to their CVS tree.
Re:Modernizing the OpenBSD Console by notaplumber on Tuesday September 01, 2020 @12:03PM (#24343262)
Precisely. OpenBSD developers have committed John's diffs.
Re:Modernizing the OpenBSD Console by rbanffy on Tuesday September 01, 2020 @08:08AM (#24340892)
> you'd think Apple would be the first one to fix this…

To be fair, they have one of the very few terminal apps that support double-width/height characters and that passes the VT torture test with flying colors.

Re:Modernizing the OpenBSD Console by saagarjha on Tuesday September 01, 2020 @01:56PM (#24344668)
I mean, I would think this would just be using a larger font size for their bitmap, no?
Re:Modernizing the OpenBSD Console by rbanffy on Tuesday September 01, 2020 @02:44PM (#24345291)
Apple's terminal doesn't seem to use a specific bitmap codepath - it renders the characters at a doubled size for DECDHL and it looks like it stretches the geometry of the font horizontally for DECDWL.
Re:Modernizing the OpenBSD Console by rbanffy on Tuesday September 01, 2020 @08:16AM (#24340952)
What really disappoints me is that, with all those pixels, and bits per pixel we now have, it still can't have proper bold, italic, underline, overline, Tek graphics, sixels, and smooth scrolling like any ancient terminal made by any well regarded hardware company.
Re:Modernizing the OpenBSD Console by HugoDaniel on Tuesday September 01, 2020 @09:07AM (#24341346)
indeed :O

here are the VT100 ansi escape codes: https://espterm.github.io/docs/VT100%20escape%20codes.html

Bold, Underline, Double height, double width, etc... are all there. VT100, from 1978.

Re:Modernizing the OpenBSD Console by msla on Tuesday September 01, 2020 @09:34AM (#24341559)
ncurses gives you all of that in a much easier-to-use fashion.
Re:Modernizing the OpenBSD Console by rbanffy on Tuesday September 01, 2020 @02:40PM (#24345222)
True, but not really relevant in this context. The fact is that graphical consoles could support all that because, in reality, there was a time they actually did - they were just independent embedded computers connected to the host via a serial port.

In my dreams, when switching to Tek mode, the framebuffer would have a dark green background, the lines would have a brighter green, and drawing would push the HDR for the brightest green it'd be able to do (because to store a vector in a Tektronix DVST, you flash it with a high energy beam). Erasing the screen would flash it light green and there would be no scrolling.

Re:Modernizing the OpenBSD Console by unixhero on Tuesday September 01, 2020 @02:13AM (#24339022)
"Optimize character rendering in 32bpp mode (John Carmack, June 2020)"

Awesome

Re:Modernizing the OpenBSD Console by JdeBP on Tuesday September 01, 2020 @03:23AM (#24339333)
It's the sort of optimization that one can read about in games and DirectX programming books.

A better optimization comes with the realization, as MacOS people had many years ago, that copying from system memory to video memory is slow. So one instead pre-loads the glyphs into off-screen video memory, possibly on-demand, and then uses the display hardware to blit them into the on-screen part.

Of course, this requires that code know about blitter hardware, which does not necessarily exist. If one is doing an in-kernel terminal emulator, such as wscons in OpenBSD here, then one can rely upon the fact that the EFI Graphics Output Protocol provides a Blt() function that is supposed to abstract away the hardware. Indeed, EFI GOP can in theory not provide direct framebuffer access in the first place and require that everything be blitted (PixelBltOnly).

We who write application-mode terminal emulators are not so fortunate. (-:

Re:Modernizing the OpenBSD Console by mjg59 on Tuesday September 01, 2020 @03:32AM (#24339384)
GOP functions aren't available after ExitBootServices() has been called, so the kernel doesn't get to take advantage of Blit() - the assumption is that the kernel will have a native driver for the hardware. Additionally, you can't really use off-screen graphics RAM in the EFI environment because GOP doesn't tell you how much video RAM is actually available, only the size of the framebuffer. You can guess which PCI device is associated with the framebuffer and look at the size of the BAR, but you have no way of knowing which addresses the EFI driver is using for its own purposes. Apple have enough control over their platform that they could possibly make this work, but it's tough when you're dealing with a vendor-provided driver on an add-in card.
Re:Modernizing the OpenBSD Console by JdeBP on Tuesday September 01, 2020 @04:12AM (#24339575)
Hah! I've been doing too much load-time stuff, obviously. (-:

But if one is looking at PCI device IDs, then one is already at the level of the code having detailed knowledge of the specific hardware (even if only via some "miniport"-like system) and the point stands.

Re:Modernizing the OpenBSD Console by mjg59 on Tuesday September 01, 2020 @04:43AM (#24339701)
Oh, you can look at the device that the GOP instance is bound to and get the PCI device from there. At that point you just walk the BARs looking for one that's mapped in a region that covers the framebuffer address that GOP gives you. This doesn't require you to know anything about the device in question, and most of the time you'd probably get lucky by throwing glyphs in space that exists outside the framebuffer and blitting them onto the screen - up until you find a driver that implements blt() by using the 3D engine and mapping some buffer objects on top of the space you're trying to use.
Re:Modernizing the OpenBSD Console by JdeBP on Friday September 04, 2020 @02:45PM (#24377723)
The code will still have to know about blitter hardware, as I said.

Any design that depends from "you'd probably get lucky" is not a wise one. (-:

Re:Modernizing the OpenBSD Console by xvilka on Monday August 31, 2020 @09:17PM (#24337570)
Will it support "true colors" (16M colors)[1][2]?

[1] https://github.com/termstandard/colors

[2] https://gist.github.com/XVilka/8346728

Re:Modernizing the OpenBSD Console by watersb on Monday August 31, 2020 @09:37PM (#24337672)
The article uses 32 bits per pixel to illustrate the growth in performance requirements:

> In framebuffer however, to fill a 4K UHD-1 (3840x2160) screen in 32bpp mode we need to send 3840 * 2160 * 4 bytes of data, which is 33177600 bytes (approximately 33 MB).

but I don't know if that is in fact the definition a "framebuffer console" versus a terminal that uses character addressing.

That's a different issue that actually supporting 32-bit ANSI codes, but I think it's an indication of the scope of a an effort to update the console implementation.

Re:Modernizing the OpenBSD Console by asveikau on Monday August 31, 2020 @10:10PM (#24337874)
Some of the code linked in the article seems to emulate the VGA style text console (1 value for char value, 1 value for color and attributes) and render it onto a 32bit framebuffer.

The diffs that are attributed to Carmack seem to deal with this. scr->rs_bs[off].uc and scr->rs_bs[off].attr appear to represent the gylph and attributes/color respectively.

So I would expect that would severely limit the amount of colors available to the input buffer. OTOH I just found the declaration in the code and the attributes are 32 bits wide which is more than the 8 that VGA provides:

    struct wsdisplay_charcell {
       u_int       uc;
       uint32_t    attr;
    };
Note I would expect the 32 bits of attr to be divided between fg and bg color, and it seems like they carved out bits for things like underline.

Edit: after reading some more it seems like the color values range from 0-7.

Re:Modernizing the OpenBSD Console by JdeBP on Tuesday September 01, 2020 @03:57AM (#24339519)
32 bits isn't really enough. One uses 24 bits for RGB, and the remaining 8 bits aren't really enough for the remaining graphics renditions that a modern terminal emulator should support.

Factor in things like the KiTTY underlining system, invisible, the little-recognized new (sic!) ECMA-48:1991 graphic renditions (encircled, framed, and overlined), and strikethrough; and one actually needs 13 to 14 bits (depending from how far one takes the KiTTY underlining system).

Moreover: No-one doing this style of terminal emulator really does Unicode properly. To do so actually involves having a string of one or more code points per cell, not simply exactly one. Some of the terminal emulators based upon external GUIs (e.g. X11) do do that, or approach it. (Although at the same time they take a giant leap backwards from video terminal semantics to paper terminal semantics, so it is a matter of swings and roundabouts. See https://unix.stackexchange.com/a/593613/5132 .)

Re:Modernizing the OpenBSD Console by asveikau on Tuesday September 01, 2020 @12:12PM (#24343363)
> 32 bits isn't really enough. One uses 24 bits for RGB,

Yes, and as I mentioned you need to divide between FG and BG colors.

All a little moot because it seems to me like the code uses 3 bits for color.

> No-one doing this style of terminal emulator really does Unicode properly

Indeed, both multi-char and multi-codepoint glyphs. I also think the whole "matrix of glyphs" thing breaks down pretty severely when you want BiDi support for instance. The article mentions someone working on UTF-8 support. I wonder what that looks like.

Re:Modernizing the OpenBSD Console by JdeBP on Tuesday September 01, 2020 @03:40AM (#24339430)
The "I" in "ITU" does not stand for "ANSI". (-:

* https://itu.int/rec/T-REC-T.416

As M. Kochkov will attest, supporting 24-bit RGB (or even 32-bit ARGB) colour involves (a) recognizing and processing the new (sic!) control sequences from the 1990s, and (b) actually supporting having the display hardware in a non-palettized true colour mode (as opposed to using a 256-colour palettized mode and pretending).

The definition of a framebuffer virtual terminal is, rather, that the display hardware is in its graphics mode rather than its text mode. (This was the norm for some architectures, which didn't even have text mode display hardware. But the MDA/Hercules/CGA/EGA/VGA on the IBM PC and compatibles did.) The operating system, that is providing the virtual terminal, draws the glyphs by manipulating pixels, rather than relying upon "character generation" hardware to do it in the video output stage of the adapter.

Re:Modernizing the OpenBSD Console by formerly_proven on Tuesday September 01, 2020 @06:15AM (#24340156)
I expect none of this (text mode, indexed colors, 16 bpp, hardware GDI/2D etc.) actually exists in hardware any more. I expect GPUs emulate all of this through shaders baked into the VBIOS/driver.
Re:Modernizing the OpenBSD Console by JdeBP on Friday September 04, 2020 @02:43PM (#24377702)
I doubt that very much. I cannot see people successfully making a business case to spend development time and money re-implementing MDA/CGA/EGA/VGA text mode, with the concomitant on-going support costs when inevitably the re-implementation misses stuff, for pretty much zero additional revenue. No, this wouldn't be implementable through drivers. It has to work during POST and IPL.
Re:Modernizing the OpenBSD Console by TimSchumann on Monday August 31, 2020 @09:51PM (#24337758)
Just curious, but does anyone know a business using OpenBSD in production other than the Pushover App?
Re:Modernizing the OpenBSD Console by LeoPanthera on Monday August 31, 2020 @10:20PM (#24337923)
Virtually everyone uses bits of OpenBSD. Android uses a lot of its C standard library, LLVM uses its regular expression library, and Windows 10 uses OpenSSH with LibreSSL.

Armorlogic, Calyptix Security, GeNUA, RTMX, and .vantronix make network appliances based on the complete OS.

Re:Modernizing the OpenBSD Console by smabie on Monday August 31, 2020 @11:16PM (#24338206)
Don't forget tmux, cwm, openbgpd, and openntp. The OpenBSD guys consistently make very high quality tools.
Re:Modernizing the OpenBSD Console by aaron_m04 on Monday August 31, 2020 @11:44PM (#24338340)
cwm is a diamond in the rough for sure. Doesn't look like much but it's so easy to configure and use.
Re:Modernizing the OpenBSD Console by smabie on Monday August 31, 2020 @11:52PM (#24338386)
I absolutely love cwm. Best floating window manager, hands down.
Re:Modernizing the OpenBSD Console by runbsd on Monday August 31, 2020 @10:22PM (#24337940)
Especially OpenSSH...
Re:Modernizing the OpenBSD Console by jlgaddis on Tuesday September 01, 2020 @08:19AM (#24340975)
Re:Modernizing the OpenBSD Console by jlgaddis on Tuesday September 01, 2020 @08:13AM (#24340928)
I used OpenBSD in production at my previous job (an ISP), in a few different roles. Lots of ISPs use OpenBSD for various purposes.

HSMs from nCipher used to run on top of (a slightly modified, IIRC) OpenBSD. Just as JunOS (the OS on (some) Juniper network devices) runs on FreeBSD, I'm sure there are many other commercial products built on OpenBSD as well -- we just don't hear about them very much.

---

Related (from the same author as this submission): "Why OpenBSD?" [0].

---

[0]: https://www.cambus.net/why-openbsd/

Re:Modernizing the OpenBSD Console by daneel_w on Tuesday September 01, 2020 @01:33PM (#24344342)
An acquaintance is currently building the tech part of his and some friends' start-up, entirely on OpenBSD - PHP-based web front end, back end, Postgres-based database, mail server etc.
Re:Modernizing the OpenBSD Console by mgreenleaf on Tuesday September 01, 2020 @03:48AM (#24339469)
Not sure if this counts, but I use it in production in yakdocs.com and yak-mu.com. I love how easy it is to maintain and I love the pf firewall. Hetzner has images for OpenBSD on their cloud service and you can use a custom image on the dedicated servers.
Re:Modernizing the OpenBSD Console by brobdingnagians on Tuesday September 01, 2020 @04:00AM (#24339530)
Vultr also has support for OpenBSD
Re:Modernizing the OpenBSD Console by protomyth on Tuesday September 01, 2020 @06:43AM (#24340325)
Works well too. We've moved a lot of boxes from local to them. I do wish their console supported cut and paste on a Mac.
Re:Modernizing the OpenBSD Console by ravar on Tuesday September 01, 2020 @02:54AM (#24339177)
Netflix uses freeBSD does that count? https://www.youtube.com/watch?v=KP_bKvXkoC4
Re:Modernizing the OpenBSD Console by Aloha on Tuesday September 01, 2020 @12:33AM (#24338589)
am I the only one who prefers a non-framebuffer console?
Re:Modernizing the OpenBSD Console by ansible on Tuesday September 01, 2020 @04:00AM (#24339529)
I hate the framebuffer console on most of the systems I have.

In some cases, the font is way too small. Changing the font size is confusing, because it isn't clear what size of text you will end up with.

And on some of the systems I have, the framebuffer scrolls so slow!!!!! Drives me up a wall. If it takes more than a tiny fraction of a second to display an entire screen of text, that is too slow!!!! Hate it, hate it, hate it.

Re:Modernizing the OpenBSD Console by icedchai on Tuesday September 01, 2020 @12:25PM (#24343498)
Sun workstation framebuffer consoles were like this. Slow scrolling! I'm thinking mainly of Sun-3's and early SparcStations...
Re:Modernizing the OpenBSD Console by Aloha on Tuesday September 01, 2020 @01:14PM (#24344112)
even on an Ultra 5 I had, it was slow.
Re:Modernizing the OpenBSD Console by icedchai on Tuesday September 01, 2020 @03:24PM (#24345861)
Now that I think of it, I have an Ultra 10 rotting in my basement. I haven't turned it on in over 15 years now.
Re:Modernizing the OpenBSD Console by throwaway2048 on Tuesday September 01, 2020 @01:24AM (#24338815)
Its not even an option on newer UEFI machines, all consoles are a framebuffer.
Re:Modernizing the OpenBSD Console by Aloha on Tuesday September 01, 2020 @02:19AM (#24339042)
Let me rephrase, I don't want a huge screen with tiny fonts for my console, the default should be at most 132x48
Re:Modernizing the OpenBSD Console by rvense on Tuesday September 01, 2020 @02:36AM (#24339115)
Isn't that exactly what this patch addresses? Updating the framebuffer console so it's not horrible on modern framebuffers.
Re:Modernizing the OpenBSD Console by Aloha on Tuesday September 01, 2020 @01:13PM (#24344100)
the OS/Distro should chose sensible options out of the box, they usually do not.
Re:Modernizing the OpenBSD Console by unixhero on Tuesday September 01, 2020 @02:14AM (#24339023)
Interesting, I didn't know that. Thanks!
Re:Modernizing the OpenBSD Console by kjs3 on Tuesday September 01, 2020 @02:15AM (#24339029)
Not the only one, but one of the few.

glances at the VT-320 hooked to the OBSD firewalls

Re:Modernizing the OpenBSD Console by Aloha on Tuesday September 01, 2020 @02:18AM (#24339037)
I have a WY-185 sitting around, I feel like I should set it up again, on a terminal server maybe
Re:Modernizing the OpenBSD Console by kjs3 on Tuesday September 01, 2020 @02:49AM (#24339161)
It's a moral imperative. :-)
Re:Modernizing the OpenBSD Console by rbanffy on Tuesday September 01, 2020 @08:10AM (#24340907)
An elegant tool for a more civilized age.
Re:Modernizing the OpenBSD Console by Aloha on Tuesday September 01, 2020 @01:12PM (#24344093)
its also alot of effort and consumes more of my limited desk space ;-)
Re:Modernizing the OpenBSD Console by rbanffy on Wednesday September 02, 2020 @01:48PM (#24355762)
What is desktop space compared to having a proper console? Do it!

I'm personally after a small HP terminal that was nicknamed the "munchkin" for its diminutive size. My chances of finding one that can be shipped here are slim, but from the US you could have a better chance.

Re:Modernizing the OpenBSD Console by JdeBP on Tuesday September 01, 2020 @04:25AM (#24339625)
Almost certainly not, but on the other hand there are people who conversely enjoy proper boldface and italics, as Unix text processing tools encompassed pretty much from the get-go; or more than 256 different glyphs in use at once, with none of that faffing about with loadable Unicode-to-8-bit character maps; or MouseText. (-:
Re:Modernizing the OpenBSD Console by Koshkin on Tuesday September 01, 2020 @12:47PM (#24343747)
Hopefully not too off-topic, I have a question: shouldn't we stop using terminal emulators already (i.e. why do we still require emulating the terminal hardware designed in 1970)? Wouldn't a text editor-like user interface be more sensible? (Perhaps, something similar to emacs' M-x shell but more universally adopted.)
Re:Modernizing the OpenBSD Console by hideevidence on Monday August 31, 2020 @11:28PM (#24338263)
I want to note kmscon, it's really distraction free terminal. Best of all it easily supports unicode points with no added user effort.