News for hackers. Stuff that matters.
Re:Show HN: A retro terminal text editor for GNU/Linux coded... by jll29 on Saturday August 31, 2024 @04:54PM (#41411884)
Apart from the learning experience, there is still value in a small-sized, fast TUI text editor for Linux.

However, my two suggestions would be:

- add Unicode support (it's the 21st century, so ∑, ®, etc. should finally work); this is not easy, but the earlier you do it the less dramatic the changes will be (check out https://github.com/unicode-org/icu/tree/main/icu4c);

- don't allocate the lines individually (as you do now with malloc, having looked at the code); at least use an arena allocator, i.e. allocate larger chunks of memory and the provide your own alloc_line function that uses the larger chunks (called arenas) piecemeal. This will speed things up and reduce fragmentation. A more advanced approach would be not to use per-line buffers but to switch to rope data structures e.g. https://github.com/josephg/librope).

Re:Show HN: A retro terminal text editor for GNU/Linux coded... by badsectoracula on Saturday August 31, 2024 @06:35PM (#41412562)
> This will speed things up and reduce fragmentation

Does this really make much of a practical difference these last decades? I wrote a text editor back in the late 90s in DOS using DJGPP with per-line allocation and it worked fine on the Pentium MMX i had at the time (and my approach to optimization at that time was be ignorant of the concept :-P), i'm not sure it'd really make much of a difference on any desktop or laptop CPU released past 2000 - at least as far as users editing text files are concerned (might make a difference if you try to open a multiGB file with barely a line break and accidentally press a letter :-P).

Re:Show HN: A retro terminal text editor for GNU/Linux coded... by HexDecOctBin on Saturday August 31, 2024 @11:18PM (#41413988)
No, it won't. You might want an arena for other reasons (better reasoning about lifetimes, preventing nest of pointers, etc.), but an array of line array approach to text editors is totally valid (except in the case of millions of extremely small lines).
Re:Show HN: A retro terminal text editor for GNU/Linux coded... by lanstin on Sunday September 01, 2024 @11:07AM (#41417561)
I do from time to time open files of millions of small lines. To this day vim handles it better than emacs. These would be log files.
Re:Show HN: A retro terminal text editor for GNU/Linux coded... by HexDecOctBin on Monday September 02, 2024 @09:16PM (#41430254)
Log files have lines that are long enough, I was thinking more of one/two-character long lines (such that the overhead of malloc bookkeeping would be more than the line size in memory). Regardless, Emacs also stores the data in a single array with a gap in middle (and memcpy is relatively cheap), so what slows it down is probably not the choice of data structure but everything else built around it.
Re:Show HN: A retro terminal text editor for GNU/Linux coded... by lanstin on Tuesday September 03, 2024 @10:26AM (#41435305)
Yeah I have very high limits for paren matching and syntax highlighting so probably those scanners.
Re:Show HN: A retro terminal text editor for GNU/Linux coded... by simjnd on Saturday August 31, 2024 @09:13PM (#41413383)
As an alternative to ICU, there is suckless's libgrapheme (https://libs.suckless.org/libgrapheme/) which is more than a 100x smaller and provides full Unicode compatibility.
Re:Show HN: A retro terminal text editor for GNU/Linux coded... by wwalexander on Saturday August 31, 2024 @11:08PM (#41413938)
Heh, I maintain the MacPorts port for this lib! Like all suckless projects, it’s written in very simple and portable C which makes it a breeze to package.

https://ports.macports.org/port/libgrapheme/

Re:Show HN: A retro terminal text editor for GNU/Linux coded... by frign on Monday September 02, 2024 @04:45PM (#41428666)
Thanks for your work on packaging my library. Please let me know if I can make the process simpler for you; I take great care to make packaging as simple as possible for the packagers. Likewise I have no sympathy for those writing software that is almost deliberately hard to package.
Re:Show HN: A retro terminal text editor for GNU/Linux coded... by wwalexander on Monday September 09, 2024 @04:13PM (#41493100)
Thank you for your work on the library! It really couldn’t be easier to package but thank you <3
Re:Show HN: A retro terminal text editor for GNU/Linux coded... by HexDecOctBin on Saturday August 31, 2024 @11:16PM (#41413973)
Are there any similar options that handle collation and normalisation? No small Unicode library seems to implement them, unfortunately.
Re:Show HN: A retro terminal text editor for GNU/Linux coded... by frign on Monday September 02, 2024 @04:53PM (#41428730)
Author of libgrapheme here: Both collation and normalisation are non-trivial and have many gotchas thanks to the way the Unicode consortium likes to write their specifications. I sometimes get the feeling that they don't even care about implementers and just document what is done in the reference implementation ICU.

The only sensible normalisation one can implement is the full decomposition (NFD), and maybe the full composition (NFC). You rely on the full decomposition if you want to collate correctly, which is a problem because the amount of memory needed to store the decomposition is unbounded in general. I don't want to make the libgrapheme users jump through hoops, and I also don't want to do any memory allocations in libgrapheme either.

There is an idea floating in my head on how to solve this, but I'm currently busy finalising Unicode 15.1 support (Unicode 16.0, released on the 10th, will be trivial to upgrade to) and releasing my already fully-compliant implementation of the Unicode bidirectional algorithm.

Re:Show HN: A retro terminal text editor for GNU/Linux coded... by HexDecOctBin on Monday September 02, 2024 @09:21PM (#41430276)
I see, thanks for replying. I agree, Unicode specs are hell to work with (I tried doing a auto-codegen thing based on them and just gave up due to the size of tables generated and the seemingly-arbitrary edge cases). libgrapheme looks pretty good otherwise, I'll keep an eye on it for whenever I have to wrangle with Unicode on a low level again (hopefully not for a long time).
Re:Show HN: A retro terminal text editor for GNU/Linux coded... by velorek on Sunday September 01, 2024 @06:21AM (#41415688)
Interesting. Cool library, I'll check it out. :) Thanks for sharing
Re:Show HN: A retro terminal text editor for GNU/Linux coded... by frign on Monday September 02, 2024 @04:44PM (#41428660)
Thanks for recommending libgrapheme. I am honoured, being the author of this library.
Re:Show HN: A retro terminal text editor for GNU/Linux coded... by wolfspider on Sunday September 01, 2024 @12:50AM (#41414389)
Well yeah, Seph is right as usual I’ve been updating my MLIR generation code to use Twine lately: https://llvm.org/doxygen/classllvm_1_1Twine.html#details

LLVM has been using rope data structures for some time. There is more discussion about it here:

https://lists.llvm.org/pipermail/llvm-dev/2011-July/041639.h...

In my situation when parsing lines of code Twine stores the file path and concats the line number and column when getting the location within a file. It does this a lot so using Twine there really helps. The rope data structure can be used in a number of ways.

Re:Show HN: A retro terminal text editor for GNU/Linux coded... by velorek on Sunday September 01, 2024 @06:20AM (#41415684)
I agree. Unicode support is top priority. Thank you for sharing those links.