News for hackers. Stuff that matters.
Re:Write plain text files by imgabe on Tuesday March 01, 2022 @07:49PM (#30522175)
Another advantage to plain text files: source control. You can check your writing into git and get a history of all your edits.

It’s something programmers take for granted, but it would be amazing if this got more widely adopted outside of tech. The number of files with names like “Report Final Final draft v3.docx” is truly staggering.

“Git for everything“ would be a multi-billion dollar startup easily.

Re:Write plain text files by phreack on Tuesday March 01, 2022 @11:57PM (#30523914)
I once thought git for humans would be a great idea but never got around to speccing it out. Later on, a lawyer friend showed me the software they used 'for backup' (that they paid thousands per month for) and it turned out everything about it was just exactly like SVN. The terminology was different, the UX was laser focused to the intended users, but at the end of the day it was commits, syncs, merging, pretty much everything but branches, just laid out with domain specific language instead, professional office UI and simple UX.

So, uh, yeah I agree absolutely with you.

Re:Write plain text files by awr on Wednesday March 02, 2022 @07:04PM (#30535294)
What was the tool called? We're currently looking to solve a problem for a client to better manage contract negotiation between his lawyers and the purchaser's lawyer (apartment units). ATM it's all done via email but I imagine there'd be heaps of tools out there.
Re:Write plain text files by ben174 on Wednesday March 02, 2022 @06:57AM (#30526340)
I tried to git my resume once upon a time. It's still something I'd love to finish doing, it just makes sense to have a git repository as a timeline of your life:

https://github.com/ben174/bugben

Re:Write plain text files by robin_reala on Wednesday March 02, 2022 @09:05AM (#30527396)
My official CV is a page on my site, that I print to PDF if someone asks for it. The page itself is as source-controlled as the rest of the site.
Re:Write plain text files by thefunnyman on Thursday March 03, 2022 @02:43PM (#30545769)
I do the same by having my resume in latex which I’ve checked in to git. Been meaning to write a GH action to generate the pdf artifact automatically.
Re:Write plain text files by tomrod on Thursday March 03, 2022 @09:24PM (#30549872)
I've decided to migrate my CV to Markdown instead -- pandoc can dump to LaTeX, PDF, you name it.
Re:Write plain text files by chrisfinazzo on Friday March 04, 2022 @10:14AM (#30555933)
I eventually settled on LaTeX - although I considered dropping all the way down to groff/troff and having TeX be an intermediate step in the process.

Custom LaTeX classes made this more trouble than I was willing to deal with, so I decided to not pursue it further. I suspect Markdown might have similar challenges dealing with this, although given that the "verbose mode" is just HTML, I might be able to make it work.[1]

The print media type has been around for eons, but the @page rules don't support everything I need and are generally absent in WebKit browsers.

[1]: https://github.com/chrisfinazzo/resume

Re:Write plain text files by vertis on Wednesday March 02, 2022 @05:00AM (#30525644)
Enterprise Content Management like EMC Documentum has this sort of thing baked in (it's been a long time, but they're definitely on top of it). But that's for big companies.
Re:Write plain text files by Zak on Wednesday March 02, 2022 @10:15AM (#30528251)
I had a similar thought some time ago and concluded it's an impossible task. The problem is that it would need to understand every file format its users care about and be able to represent changes in a useful way.

How do you merge destructive edits to image files?

Re:Write plain text files by yencabulator on Friday March 04, 2022 @01:18PM (#30558335)
Many version control systems aimed more toward digital assets do some sort of locking, so only one user edits a file at a time. Advisory or mandatory, with support for breaking locks, etc.
Re:Write plain text files by tracker1 on Wednesday March 02, 2022 @07:24PM (#30535443)
git-lfs with appropriate file types set to use it.
Re:Write plain text files by sslayer on Wednesday March 02, 2022 @11:09AM (#30528876)
LOL,sweet summer child, its all 0's and 1's
Re:Write plain text files by munk-a on Wednesday March 02, 2022 @01:51PM (#30531199)
Please explain, in 0's and 1's how to combine the change to add a dark splotch over here and a patch of blue over there so that the image you get out of the merge has both the dark splotch and the patch of blue... without teaching git how to comprehend image file formats (and assuming you're not using a bitmap which is sort of a free win).
Re:Write plain text files by D13Fd on Wednesday March 02, 2022 @01:00AM (#30524265)
Honestly I think modern Sharepoint/OneDrive based .docx files have already leapfrogged git for the average user.

In an office environment, modern MS Word gives you a really nice version history, with automatic or manual/named versions, plus multi-user simultaneous editing, with a slick and fast UI. It’s even cross platform - I send a coworker a link to a document, and they can open it and work on Word for Windows while I edit the same doc on Word for Mac or even on the web-based Word.

It’s shockingly good.

Re:Write plain text files by CorrectHorseBat on Wednesday March 02, 2022 @02:10AM (#30524625)
After not using Word for years we've been using it for a few months now for a project, expecting it would be decent.

While some of the ideas are great, the overall experience was pretty bad. Leaving autosave on often slowed down Word to an unworkable pace and after a few other issues (I'm not exactly sure what, wasn't involved) we just made sure only one person at a time works on the document.

Some other annoying small issues I experienced: when working together sometimes your undo history is gone. I guess that's because someone else edited too and correctly undoing something someone else edited in between is not simple. IMO it's impossible and any system doing merges automatically will fall apart at some point.

Track edits cannot track deleting rows of a table

Wed based word works actually pretty fine, but it misses random features. Adding captions seems not to be supported.

Re:Write plain text files by yarky on Wednesday March 02, 2022 @06:46AM (#30526238)
I agree, I also expected the whole office365 ecosystem to do what's supposed to, but I also found it useless due to the slowness and errors the whole team kept getting while working on the same file. That and the fact that it somehow feels like the Mac version of their products always comes with a bonus bug. Don't event get me started on their web versions.

It's the little things...

Re:Write plain text files by D13Fd on Wednesday March 02, 2022 @07:20AM (#30526518)
Were you using it on Linux or something? Or maybe with very large documents?

I use it every day and really haven’t had slowness issues, even when running via Parallels on the Mac side. Of course, typically my docs are <100 pages and light on images.

I have seen undo history go away when edits collide with multiple users, but that’s really the only way it could work. And you can still view old versions of the doc if you need to retrieve/restore something.

MS Word as a whole is definitely very far from flawless, but it’s still better than the alternatives I’ve seen, and it’s extremely entrenched. That’s why it’s hard to see “git for office docs”succeeding.

Re:Write plain text files by CorrectHorseBat on Wednesday March 02, 2022 @08:38AM (#30527151)
>Were you using it on Linux or something? Or maybe with very large documents

~250 pages, ~100 images, not something I expect a modern computer to struggle with. We used the web version on Linux sometimes, but I think it also happened when we didn't.

>I have seen undo history go away when edits collide with multiple users, but that’s really the only way it could work. And you can still view old versions of the doc if you need to retrieve/restore something.

The other way is like git does it, manual merging and conflict resolving. I don't think there's a way around that once you work with enough people together on a file.

Re:Write plain text files by D13Fd on Wednesday March 02, 2022 @10:28AM (#30528408)
That makes sense, although obviously the downside to a manual check-in/merge process is that it greatly slows down the editing flow.

Personally for the kinds of documents I work on, I'd rather risk losing my undo history than have to deal with manually merging changes from multiple users.

Plus, without real-time concurrent editing, the changes are much more likely to conflict.

Re:Write plain text files by srazzaque on Wednesday March 02, 2022 @03:26AM (#30525126)
It's great, until it isnt. For small and simple documents (sub 20 pages), there's no better option.

In my experience it starts to creak on large documents with multiple editors. I did a project many years ago where the key deliverable was a large report, worked upon by 3-4 contributors over 12 weeks.

That last week was absolute madness. Document got corrupted, and we tried all sorts of things. We ended up importing the corrupt document into Google Docs, which, oddly enough, did a better job of working with a corrupt Word document than MS Word did.

Re:Write plain text files by jmmcd on Wednesday March 02, 2022 @05:43AM (#30525892)
I've seen online Word become extremely slow even for small documents with a few editors.

It doesn't help that there are lots of versions of online Word - there's the one that lives in Sharepoint, the one I get to via Outlook, the one in Teams, and (maybe? somehow?) the desktop Word when editing an online file.

Re:Write plain text files by jsmith99 on Wednesday March 02, 2022 @02:21AM (#30524716)
Word also includes a nice GUI diffing tool for docx files. It's possible to make git diff use it using something like https://github.com/ForNeVeR/ExtDiff
Re:Write plain text files by thrwawy283 on Wednesday March 02, 2022 @03:04AM (#30524979)
Thank you for sharing this. I had no idea it existed! I was getting pandoc cleared at my work, but for the Windows environment that it is, this is another solution :)
Re:Write plain text files by bodge5000 on Wednesday March 02, 2022 @08:50AM (#30527277)
I dont use word, but I do know people that do use it (the modern version) and they still have files called "...final 3 copy(4) really final.docx".

These features are nice and all, but the thing is, most users not only don't care about using them, but don't care about even the idea of them to know they exist, so they continue with the old workflow.

Most of the time its better to get out of the users way, but in cases like this, if you truly do want to get rid of the infinite finals, they need a bit of a nudge

Re:Write plain text files by wildzzz on Wednesday March 02, 2022 @09:44AM (#30527845)
We use SharePoint for revisions for casual documents and another CM system for tracking releases of official documents. The CM system is meant really for CAD models but it lets you upload Word docs that it displays as PDFs with watermarked revision information so you have to have a revision table at the front of the document for listing changes. The SharePoint system is a lot easier to use but you don't get the same sort of change approval system since anyone can edit. Instead we use multiple folders for WIP, waiting for QA approval, and final versions. We have to follow ISO 9001 but anything developmental or that is just derivate of documents in the CM system can live on much more casual systems like git, SVN, or SharePoint.
Re:Write plain text files by xerox13ster on Wednesday March 02, 2022 @01:48AM (#30524505)
That's not plain text and not easily parse-able with mutli document search
Re:Write plain text files by zouhair on Wednesday March 02, 2022 @12:32PM (#30530024)
It's a proprietary format, it's not gonna last.

Microsoft word can't even open files from old version of the same software.

None of these word document will be readable in 50 years from now, a text file though.

Re:Write plain text files by catach on Wednesday March 02, 2022 @08:11PM (#30535835)
A modern MS Word document is pretty much zipped XML and mostly ISO standard-compliant.
Re:Write plain text files by thrwawy283 on Wednesday March 02, 2022 @03:09AM (#30525008)
Hmm. Sometimes I step back and I look at the version history in modern Office documents and it does feel convenient. I like the CRDT editing a lot. It still feels too loose and prone to issues, but I recognize its value to the average person.

Most of what bothers me is trying to view, read-only a previous version. Usually I have several Word or Excel documents up on my screen, and it's easy to begin modifying a past version that then becomes the latest. It's not reflex to open a previous version and untoggle autosave. It bites me all the time.. In Office's pursuit to help people avoid losing data, I wind up overwriting what I need to preserve. A lot..

I have not seen manual version names in Word doc history. Maybe I'm sheltered :s

Re:Write plain text files by yakubin on Wednesday March 02, 2022 @06:02AM (#30525990)
The idea is pleasing, but when you specifically look at Git, its UX is rather disappointing. I'm not even talking about the CLI (after a while you can get used to it). I'm talking about resolving conflicts. If I have a sequence of changes A -> B -> C, then I change A and rebase B on top of A resolving any conflicts, then rebasing C on top of B will most of the time mean that Git will most likely ask me to resolve the same conflicts all over again and more. The kind of conflicts is also staggering. Each time I look at the conflicts it generated, I cannot explain how it could come up with that craziness. And all the while it also merged some changes without signalling conflicts, but instead breaking something in significant ways, without notifying me in any way, i.e. silently breaking something potentially important.

From what I've read, Pijul should fix at least the problem of needing to resolve the same conflicts multiple times over and over again. However, I feel that version control focused on text breaks, because the merging algorithm doesn't know anything about the semantics of what's represented in text. So yes, I'd say that version control in more areas would be nice, but one based on binary formats understood by version control. One glimpse of that may be KeePassXC, which can merge password databases and has never done it wrong in my experience.

Re:Write plain text files by Hackbraten on Wednesday March 02, 2022 @06:20AM (#30526084)
While I agree in that Git’s UX is abysmal for the beginner or casual user, no one should have to endure resolving the same conflicts over and over again.

In my opinion, the following configuration should have been the default with Git:

    git config --global rerere.enabled true
    git config --global rerere.autoUpdate true
The `enabled` part means: transparently record all resolutions in a database, and re-apply them whenever bumping into the same conflict with the same pair of files in the future.

The `autoUpdate` part means: every time you finish re-applying a recorded resolution, please `git add` the result automatically for me so I don’t have to look for a "conflict" that’s actually no longer there.

Re:Write plain text files by FreezerburnV on Wednesday March 02, 2022 @10:12AM (#30528222)
I had no idea these options were a thing. Thank you for pointing them out, adding them to my config immediately and hoping it makes some merges and whatnot easier going forward.
Re:Write plain text files by pmeunier on Wednesday March 02, 2022 @03:34PM (#30532702)
Hint: rerere doesn't always work. It's still guesswork, like the rest of merges/rebases/cherry-picking in Git. This is because looking only at the tips of branches doesn't work, and that's the only thing Git can do (with or without diff3 as the merge algorithm).
Re:Write plain text files by Hackbraten on Friday March 04, 2022 @04:24AM (#30552450)
The rerere database is content-addressed, and rerere will apply a resolution only if the pair of conflicting files is a bit-exact déjà-vu of a conflict it has seen before.

I don’t see how that is guesswork. My impression is that rerere is perfectly deterministic, and only repeat things the user has done before.

The downside is: once the user has manually created a faulty resolution, then rerere will possibly replay that faulty resolution.

That’s how things can go wrong, and I see how one might blame that failure on rerere itself. And I think the blame not entirely unjustified, because rerere could do a much better job in explaining how it interacts with certain scenarios. For example, let’s say I’m aborting a rebase. Will rerere roll back the resolutions it recorded? I honestly don’t know. The manpage vaguely claims that it will, but I’ve been unable to reproduce it.

I wish rerere made more transparent how it interacts with `--abort`. Just printing an informational line would already go a long way. I also feel a command like `git rerere log`, which would print the recent activity of the `rerere` database, might help with that.

Re:Write plain text files by greyhair on Wednesday March 02, 2022 @08:44AM (#30527199)
The git diff process and conflict process really goes off the rails at times, and I have never figured that out. Modify one line in a file, and add one new line two lines after that modification, and the git diff is two 40 line chunks with some common lines, a bunch of additions, and a bunch of deletions.

Really? One new added line, and one delete/add two lines above it.

it usually works fine, it really does, but when it messes up, it really messes up big time. As long as it doesn't create a conflict, you just shrug and move on, but when it does create a conflict, holy-moley!

Re:Write plain text files by Nullabillity on Wednesday March 02, 2022 @12:08PM (#30529703)
The biggest UX sin of Git is that rebasing is featured so prominently. If you had just done a normal merge commit then Git could have realized the common history, done a three-way merge, and resolved the conflict automatically.

But no, people always seem to insist on that linear history is the only concern that matters, explicitly delete the history, and then wonder why Git is so annoying and easy to screw up.

Maybe Git should have locked rebase behind a feature flag of some kind. `git pull --rebase` should certainly never have been added.

Re:Write plain text files by yakubin on Wednesday March 02, 2022 @03:03PM (#30532236)
When I have a chain of commits pending review on Gerrit, and I fixed some flaws in the first commit in chain and need to rebase the dependent commits, merging is not an option. And rightly so: when I look at the master branch, I don't want to see random corrections someone made during review, they're just irrelevant once the reviewed changes are merged.

There are many reasons why linear history is important. Rather than saying that "you shouldn't want to do that", I'd prefer it if the tools people use were fixed to better serve the things people actually want to do.

A merge would only make sure that I don't need to resolve the same conflict multiple times. It wouldn't result in the conflicts not being generated in the first place, or them being saner. The only difference in conflict resolution between rebasing and merging is that the sides are flipped (what's shown on left in rebase, is on the right in merge, and vice-versa). Which doesn't address the second issue I listed: conflicts are not only repeated, but the conflicts themselves are pretty crazy. In my example, often-times C would never even touch files which were detected as having conflicting changes with A or B. It was pretty absurd that I would have a 3-line change in C, several hundred-line changes in A and B, yet the biggest conflicts would be triggered when rebasing C on top of B, and those would be in files C did not touch. Other times diffs in conflicts would have most of the lines added and removed actually identical, with only a couple in the middle different. Why would git mark them as conflicting, is beyond me. And then there was the issue of git not detecting conflicts, when it should have, instead merging changes in a way that broke source code. None of those issues is better handled by merging than by rebasing. Most of them however have pretty good solutions in a patch-based version control system, as opposed to snapshot-based like git (conflicts in files which weren't changed? not going to happen). An even better solution would be binary formats with domain-specific merge logic, like the one in KeePassXC.

Re:Write plain text files by Nullabillity on Wednesday March 02, 2022 @04:50PM (#30533689)
> When I have a chain of commits pending review on Gerrit, and I fixed some flaws in the first commit in chain and need to rebase the dependent commits, merging is not an option.

No, you just add it at the end of the commit chain?

> And rightly so: when I look at the master branch, I don't want to see random corrections someone made during review, they're just irrelevant once the reviewed changes are merged.

git log actually has the --first-parent for this, which hides all of the commits that were merged into the branch, without destroying the history when you try to go back and try to understand why the choices were made. The idealized version of history created by constant rebasing serves neither purpose.

> Rather than saying that "you shouldn't want to do that", I'd prefer it if the tools people use were fixed to better serve the things people actually want to do.

Agreed, tools that cope poorly with merges should be fixed, rather than forcing people to hack around it by rebasing.

> None of those issues is better handled by merging than by rebasing.

Not quite, rebasing generates more false positives, since it tries to merge every intermediate commit instead of only looking at the end states and the common ancestor.

Re:Write plain text files by pmeunier on Wednesday March 02, 2022 @03:32PM (#30532684)
> However, I feel that version control focused on text breaks,

Pijul doesn't, by the way, the diff algorithm is customisable. I wrote one industrial application of Pijul that uses spaces as breaks, and semantics-aware breaking is totally doable.

Re:Write plain text files by vlovich123 on Wednesday March 02, 2022 @06:14AM (#30526053)
Have you tried git rere?
Re:Write plain text files by yakubin on Wednesday March 02, 2022 @06:24AM (#30526103)
No. That's the first I'm hearing of it. Thanks. I'll try it in the future.
Re:Write plain text files by bob1029 on Tuesday March 01, 2022 @07:58PM (#30522260)
> “Git for everything“ would be a multi-billion dollar startup easily.

In addition to plaintext documents, I have found that a simple JSON diff is also a very effective way to demonstrate changes between 2 complex biz objects. Non-developers can cope with this as long as the differences are visually obvious (red=removed, green=new, etc) and the object graph is reasonably flat. Everything can be trivially serialized to a JSON document, so this scales super well in my experience. We use a port of Google's DiffMatchPatch to generate human-friendly HTML reports of object diffs in our latest administration tools.

Re:Write plain text files by jollybean on Wednesday March 02, 2022 @08:46AM (#30527222)
Git is barely useable by technical people.

Git would be a byzantine disaster for 'everyone else'.

The graph and abstractions involved introduce enormous unnecessary complexity.

Now, 'historical changes for everyone' - yes.

But not Git. Git is very powerful, but ultimately a questionably valuable product in many cases overall. And we've now settled on it, so it's a bit difficult to displace.

Re:Write plain text files by yarky on Tuesday March 01, 2022 @08:01PM (#30522300)
I was actually surprised : last time I used Git Bash on Windows it did work quite well with .docx files.

Then I switched jobs to avoid touching .docx files ...

The best alternative I've found to generate .docx output is R markdown, which uses pandoc under the hood and let's you program the whole document the way LaTeX would.

Re:Write plain text files by greyhair on Wednesday March 02, 2022 @08:53AM (#30527297)
Pre LaTeX (I am old...) I wrote a huge command definition document for an embedded telecom product in nroff/troff. The whole thing was text files, with all the common parameters documented once, and the whole thing was assembled using a Makefile. So a typical page would be the command name, a descriptive paragraph, an include list of the parameters it used, and an include list of the possible return errors. Very little writing for a new command that mostly used existing parameters and errors.

And all tracked under CVS with a management/tracking layer on top.

# make command_doc

and the pile of text became a lovely 250 page postcript ready for the laser printer.

Re:Write plain text files by TuringTest on Wednesday March 02, 2022 @02:54AM (#30524903)
> The best alternative I've found to generate .docx output is R markdown

I do the same with markdown in Zettlr, which has a nice inline-preview format and can be used by my non-tech partners.

The generated .docx has a limited range of styles, but we see that as an advantage.

Re:Write plain text files by newbamboo on Wednesday March 02, 2022 @12:11PM (#30529739)
Powershell can generate and manipulate word docs, as another alternative to keeping content in a markdown file.
Re:Write plain text files by IceWreck on Wednesday March 02, 2022 @12:57AM (#30524252)
Isnt docx binary ? So git is storing a new version of the file every commit. That way too much storage space youre wasting.
Re:Write plain text files by testermelon on Wednesday March 02, 2022 @01:34AM (#30524435)
I've heard somewhere that docx is actually gzipped xml. But I never really confirmed that myself. But it's binary once gzipped, so your point still stands.
Re:Write plain text files by simion314 on Wednesday March 02, 2022 @02:24AM (#30524731)
>I've heard somewhere that docx is actually gzipped xml. But I never really confirmed that myself.

docx and epub are zip files, you can rename them with a .zip at the end and open to see what is inside. It might not be as simple to zip them back, at least for epub is very important to zip the files in a certain order but I forgot the details, but is easy to do from command line.

Re:Write plain text files by bellweather49 on Wednesday March 02, 2022 @03:06PM (#30532280)
Or use Vim; it has built in support for zip files, so you can just type something like `vim my_file.docx` and it will open the files in netrw (the built in file explorer). Move to the file you want and hit enter. "word/document.xml" has the main document contents in it.

The xml will probably need to be run through a formatter to be readable. You can type `:%!xmllint --format -` if you have xmllint installed.

Now prepare to spend several hours trying to make sense of the xml. :-p

Re:Write plain text files by simion314 on Wednesday March 02, 2022 @04:17PM (#30533286)
Vim is not for me, I unzip the epub/docx and then open the xml/html files in Kate, I had to do this to examine what is saved or how it saved or if my custom epub exported worked correctly. For epub I think you need to make sure the metadata file is the first one in the archive for it to work(so not sure if editing stuff directly in vim will preserve the order)
Re:Write plain text files by JonathonW on Wednesday March 02, 2022 @02:28AM (#30524753)
Docx (and pptx, and xlsx) is a zipped (not gzipped) composite of several XML files plus any other attached resources. It extracts out to a whole folder structure.

Definitely binary once compressed, though, and even when extracted not an easy format to parse. It might be XML, but it’s still representing the full complexity of an MS Office document.

Re:Write plain text files by zouhair on Wednesday March 02, 2022 @12:37PM (#30530095)
Juzipped. Just take a docx file and append .zip to it and open it.
Re:Write plain text files by tashbarg on Wednesday March 02, 2022 @01:56AM (#30524533)
Git stores a new file anyways. It’s only for packs, that delta compression is applied. And delta compression handles binary files.
Re:Write plain text files by BerislavLopac on Wednesday March 02, 2022 @01:23AM (#30524377)
That's exactly what the actual git does too.
Re:Write plain text files by copperx on Wednesday March 02, 2022 @01:59AM (#30524559)
Doesn't git do binary deltas?
Re:Write plain text files by thrwawy283 on Wednesday March 02, 2022 @12:57AM (#30524243)
Okay. Agree entirely.

I love some of the collaborative nature of Microsoft Teams and CRDT editing Word/Excel, but I'm usually pretty remote. Text over a tenuous WAN connection is ideal.

I work at a government agency and I was /just today/ getting them to review and approve Git and VS Code for our staff use - and pandoc. A couple years ago we never would have gotten open-source software approved. I wish I knew of an equivalent to SELinux or AppArmor for Windows so they could lock down things a semi-trusted application can do. VS Code will be used in a few different departments, but I mostly want it to help those unfamiliar with Git and its CLI (Git Graph is nice).

There's a trick out there to first convert things like Word documents to Markdown, and then do a diff of that intermediate output: https://hrishioa.github.io/tracking-word-documents-with-git/

Pandoc can only do so much. I'm trying to convince my part of the gov to put policy documents we disseminate to staff in a git repo, so we can track who did what and why (based on commit messages). This will be a big step, but thankfully one part of our org is already moving toward version control for IT and data analytics so I'm proposing this and suggesting we hop on their bandwagon. Momentum.

There's a Word template that we commonly use for legal purposes where it gives you line numbers and you write text to align with the numbers. When reviewing it's easy to say "change line 23 to read ....". This - for example - will not translate well through pandoc because the numbers and the text are separate text elements.

There's a market for making pandoc better. Visually translating how elements "flow" from 1 format to another, instead of simply transpiling XML to Markdown.

I'm still excited thinking in a year I might be able to git-blame the legal department for things. Or see a diff from 1 administration to the next.

Re:Write plain text files by lifeisstillgood on Wednesday March 02, 2022 @02:24AM (#30524730)
>>> There's a market for making pandoc better.

Some thoughts:

1. Policy makers think they write policy in English (or other human language). But more and more it is the software.

2. "policy engines" aren't going to cut it - we need to introspect code to decide what the code does - and translate that back to policy.

3. at my work the best solution i have got is using unit tests to explain what the policy is based on test comments - and again that's using english and again it's terrible ("best")

I think the real solution is both wider software literacy so that discussion happen "in code" and code that is more like policy (composable functional languages are thus likely to be useful here)

But great to hear you are taking any steps at all - would an HN letter writing campaign to your ministers help?

Re:Write plain text files by copperx on Wednesday March 02, 2022 @01:58AM (#30524553)
I love pandoc and git and writing text files, but oh my, unless the users in your organization are tech savvy, your scenario sounds like a recipe for disaster.
Re:Write plain text files by thrwawy283 on Wednesday March 02, 2022 @03:05AM (#30524981)
Admittedly, if there's any chance to this I think it will start with tech-savvy assistants helping to preserve documents on the backend of things. When a process takes hold budgets usually get restructured and a dedicated application is contracted around the existing process or data. I'm at minimum trying to prove the value of strict version control in my space :)
Re:Write plain text files by goosedragons on Wednesday March 02, 2022 @10:42AM (#30528558)
At least for LaTeX there are packages like lineno that will add line numbers to the compiled document which can then be referenced with a label automatically similar to equation or figure captions. So if somebody tells you to change line 23 you can reply with "Please see line \lineref{whatever} for change." and it will automatically fill where that change is to "Please see line 25" or whatever in case it's been moved by other changes.
Re:Write plain text files by JohnL4 on Wednesday March 02, 2022 @07:40AM (#30526676)
I have long thought writing law is a LOT like writing code. When you actually look at modern laws, they're basically legal patch files.
Re:Write plain text files by jandinter on Wednesday March 02, 2022 @02:45AM (#30524851)
> “Git for everything“ would be a multi-billion dollar startup easily.

Worked on a “Git for Word” project [1], which is currently on hold.

The diff part was manageable, though not trivial to get diffs that make sense for prose/regular text.

The hard parts are UX/UI (making Git concepts transparent to “normal” users) and merging. Yet without automatic merging, branching is not very convenient.

Would love to collaborate on this in the future again. Reach out if you are working in this space, happy to share.

[1] https://julesdocs.com

Re:Write plain text files by koolba on Wednesday March 02, 2022 @07:40AM (#30526673)
I’ve had better than expected success with diffing word files by converting them to markdown via pan doc. It’s nowhere near perfect as you lose nearly all formatting, but if only the actual text content is changing it allows you to automate the display of those changes.
Re:Write plain text files by willis936 on Wednesday March 02, 2022 @09:24AM (#30527605)
I don't think merging will ever be fully solved by software. It's a problem created and solved by process. How annoying merges are is entirely dictated by process.

Sourcetree is the best git GUI I've used. That could be used as a model.

I think an old-style solution to merging would be fine: output a word file that uses a unique font style to indicate which user made what conflicting changes, have the user edit the document and remove all of the "merge styles", then continue.

Re:Write plain text files by Seirdy on Tuesday March 01, 2022 @08:01PM (#30522290)
Yeah, in high school I and another student did our essay peer reviews using annotated diffs once. It was so much easier than Google Docs, since we didn't have to leave our editors.

Wrapping version control with a non-tech-friendly porcelain could help a lot of people escape user domestication from vendor lock-in.

Re:Write plain text files by copperx on Wednesday March 02, 2022 @02:02AM (#30524576)
I don't care how much porcelain you add. Using git effectively beyond add and commit requires undestanding quite a few concepts. Mass adoption of git in the consumer space ain't happening anytime soon.
Re:Write plain text files by greyhair on Wednesday March 02, 2022 @09:01AM (#30527366)
It cracks me up that this many years later, you put five people in a room to establish a git branch/merge/release strategy, and you get five very different opinions, and such strong opinions they are.

Luckily, that responsibility is not in my wheelhouse, I just have to live with whatever decisions are made.

So yeah, mass adoption in the consumer space, I don't see that happening.

A revert or a rebase gone wrong and wails of "what happened to my file?"

Re:Write plain text files by Swizec on Tuesday March 01, 2022 @08:14PM (#30522405)
> The number of files with names like “Report Final Final draft v3.docx” is truly staggering.

iirc this was the original pitch for Dropbox

> “Git for everything“ would be a multi-billion dollar startup easily.

Dropbox is valued at 8.5 billion

Re:Write plain text files by ovao on Tuesday March 01, 2022 @08:46PM (#30522624)
Dropbox version history is limited to 30 days (free plan) or 180 days (paid plan)[1].

I like Dropbox, but for documents, 30 days worth of version history is not fantastic.

[1]: https://help.dropbox.com/files-folders/restore-delete/versio...

Re:Write plain text files by squeaky-clean on Tuesday March 01, 2022 @08:51PM (#30522663)
The original pitch was "throw away your usb drive", I don't think file history came about until after 2010, but I'm not too sure and modern Google makes it impossible to find anything older than 2 years ago...

https://news.ycombinator.com/item?id=8863

Re:Write plain text files by slightwinder on Wednesday March 02, 2022 @11:51AM (#30529436)
It's not that hard to search for old content on google. Just limit the year in the search filter. Anyway, I found mentions of this feature up to 2008 back. I guess it was there since mostly the beginning.

From Sep 15, 2008: https://www.maketecheasier.com/dropbox-backs-up-and-syncs-fi...

Re:Write plain text files by vishnugupta on Wednesday March 02, 2022 @05:25AM (#30525800)
> “Git for everything“

Just to show how useful this is here’s Indian Constitution with amendments as commits:

https://github.com/anoopdixith/TheConstitutionOfIndia

Re:Write plain text files by Matrixik on Wednesday March 02, 2022 @09:30AM (#30527674)
I mourn that StackEdit [1] got abandoned. It's online markdown editor that can use git as a backend. Fully cross platform editing (in browser) with synced all text. I used it with GitHub private repository for all my notes but editing on mobile was really buggy. So I moved to notion (unfortunately).

[1]: https://github.com/benweet/stackedit

Re:Write plain text files by pokstad on Wednesday March 02, 2022 @01:27AM (#30524397)
NB does exactly this: https://github.com/xwmx/nb
Re:Write plain text files by anta40 on Thursday March 03, 2022 @12:02AM (#30537341)
Interesting, as much as I admire Evernote, I want something that is open source and can be installed locally. Of course, Git integration is another plus point.
Re:Write plain text files by dpcan on Tuesday March 01, 2022 @09:51PM (#30523066)
I think my Dropbox is already basically this. It has a revision history of my text files I think.
Re:Write plain text files by dpcan on Wednesday March 02, 2022 @10:13AM (#30528226)
Dropbox does this but it’s limited to a certain number of days, so not the exact same thing. I love the idea of a simple hit for everything.
Re:Write plain text files by yjftsjthsd-h on Tuesday March 01, 2022 @08:14PM (#30522403)
> "Git for everything“ would be a multi-billion dollar startup easily.

AIUI, git is already prepared to be that, it just needs diff programs that can handle whatever format. I mean, other than git's interface being its own impediment to non-technical users.

Re:Write plain text files by loudmax on Tuesday March 01, 2022 @08:19PM (#30522443)
"Report Final Final draft v3.docx" is in a proprietary format format controlled by a single company. The spec may be open in some sense, but it's far from a level playing field.
Re:Write plain text files by sgjohnson on Wednesday March 02, 2022 @01:51AM (#30524517)
> is in a proprietary format format controlled by a single company.

It's not proprietary by definition. A nightmare to implement? Yes.

But definitely not proprietary.

Re:Write plain text files by _dain_ on Wednesday March 02, 2022 @07:50AM (#30526753)
It doesn't follow the published open spec so it is de facto proprietary.
Re:Write plain text files by yjftsjthsd-h on Wednesday March 02, 2022 @02:06AM (#30524600)
Right, but if that company released a diff tool for docx (or someone reverse engineered one), git would work with it. I don't expect that to happen, I'm just saying git isn't the limiting factor.
Re:Write plain text files by sgjohnson on Wednesday March 02, 2022 @03:49AM (#30525259)
Pre-commit hook to convert to plaintext or markdown[0].

And there you go, a human readable diff.

[0]https://github.com/benbalter/word-to-markdown

Re:Write plain text files by conductr on Wednesday March 02, 2022 @02:33PM (#30531780)
I feel like this is a solution without a problem. Or a problem without pain.

Most people have no issues navigating a full folder of revisions to get to “Report Final Final draft v3.docx” but anything resembling version control would simply be unused. At corporate level, the version features of box.net, egnyte, and others are rarely used. I'd say most people don't even know they can navigate the revisions until they are in a data loss situation and asking about how to recover a corrupted file (which occurs most frequently with Excel files in my experience)

Re:Write plain text files by kumarsw on Wednesday March 02, 2022 @08:37AM (#30527144)
I wound up writing dupver https://github.com/akbarnes/dupver after getting frustrated with the lack of versioning tools for binary files. One neat thing about .docx files and their ilk is that they are "just" zip files so it isn't hard to add special handling to pull out their contents and run deduplication over that.
Re:Write plain text files by judge2020 on Tuesday March 01, 2022 @08:35PM (#30522552)
> “Git for everything“ would be a multi-billion dollar startup easily.

Version control with online MS Office and Google Docs seems to be going pretty strong.

Re:Write plain text files by ho_schi on Wednesday March 02, 2022 @08:12AM (#30526934)
git --everything-is-local

Which is the next thing. Git works just local and uses servers to sync. The purpose of servers is syncing and not depending on them. From this everything else comes, autonomous usage, speed, reliability, recovery.

Re:Write plain text files by deepGem on Tuesday March 01, 2022 @11:48PM (#30523856)
"Git for everything" is the first thought I had when I opened this article. I am trying to keep all my notes in Google docs and it's all good as long as you stick to their format. I can't edit text files in Google docs ! I mean, how absurd is this.

I feel like building a Google docs clone for simple text files. Just one feature - versioning (A Time machine like interface) and perhaps add collaborative editing later. I just want to write and store a text document without having to create multiple files. Automatic versioning of snapshots so I can go back in time and refer to any timestamp.

Re:Write plain text files by jsnell on Tuesday March 01, 2022 @11:57PM (#30523916)
Have a look at Etherpad.
Re:Write plain text files by chubot on Wednesday March 02, 2022 @10:55AM (#30528693)
Yup, that's basically the same argument I'm making with this diagram -- with text, you can get useful operations "for free"

http://www.oilshell.org/blog/2022/02/diagrams.html#text-narr...

https://news.ycombinator.com/item?id=30483914

It's generally not something you want to reimplement ...

Re:Write plain text files by accrual on Wednesday March 02, 2022 @01:08AM (#30524306)
> "Git for everything"

macOS Time Machine and Windows Shadow Copy aren't perfect git-based versioning systems, but it's nice something in the general direction exists.

https://en.wikipedia.org/wiki/Time_Machine_(macOS)

https://en.wikipedia.org/wiki/Shadow_Copy

Re:Write plain text files by m_eiman on Wednesday March 02, 2022 @05:46AM (#30525908)
macOS also has support for local file history, even if many apps try to ignore these features: https://support.apple.com/sv-se/guide/mac-help/mh40710/12.0/...

When they introduces this they changed the standard file operations and replaced "Save as…" with "Save a copy" - this was not universally welcomed.

Re:Write plain text files by trevormeier on Wednesday March 02, 2022 @09:28AM (#30527651)
> “Git for everything“ would be a multi-billion dollar startup easily.

Didgets (recently surfaced here on HN) seems like a sane approach to this, from the file system up. Pretty incredible performance too.

https://didgets.substack.com/p/where-did-i-put-that-file

Sadly not (yet) open source, though the developer is considering it.

Re:Write plain text files by chupchap on Tuesday March 01, 2022 @08:16PM (#30522419)
Isn't that what Google Docs does already?
Re:Write plain text files by ryanqian on Tuesday March 01, 2022 @08:29PM (#30522523)
I like this too, easily share is really a huge plus for any type of format text.
Re:Write plain text files by brightball on Tuesday March 01, 2022 @11:05PM (#30523542)
One of the big things I love about Dropbox is automatic version control of the file.

Google docs has a similar feature built in too I think.

Re:Write plain text files by innomatics on Tuesday March 01, 2022 @11:12PM (#30523598)
I switched from dropbox to mega years ago when the former removed Linux support. TIL mega have version support so I need to enable that and try it out.

First thing I do on a new device is setup my mega shared drives - and there is a lot of plaintext files there!

Re:Write plain text files by slightwinder on Wednesday March 02, 2022 @11:11AM (#30528893)
Dropbox never removed linux-support at all, or sync specifically. They removed support for some esoteric filesystems which lacked certain features. For most people dropbox on linux did not stopped working.
Re:Write plain text files by brightball on Wednesday March 02, 2022 @07:25AM (#30526555)
Been using it on Linux for many years but it does feel like their Linux support has gotten worse.
Re:Write plain text files by orhmeh09 on Wednesday March 02, 2022 @02:35AM (#30524788)
Dropbox removed Linux support? It seems to work fine for me on arch Linux with btrfs.
Re:Write plain text files by innomatics on Wednesday March 02, 2022 @02:44AM (#30524849)
It was sync specifically, and four years ago. Maybe it works now.

https://news.ycombinator.com/item?id=17732912

Re:Write plain text files by 10729287 on Wednesday March 02, 2022 @05:30AM (#30525826)
If I remember well, they removed the ability to host Dropbox folder on a encrypted drive.
Re:Write plain text files by nly on Wednesday March 02, 2022 @04:08AM (#30525367)
It works well if the text format in question is line oriented, otherwise it doesn't.
Re:Write plain text files by rmbyrro on Wednesday March 02, 2022 @09:01PM (#30536196)
I thought Google Doc had solved this with single-version model and automatic change history with easy restoration.
Re:Write plain text files by niutech on Thursday March 03, 2022 @06:36AM (#30539670)
As for "git for everything", try SparkleShare or LogicalDOC CE.
Re:Write plain text files by nuker on Wednesday March 02, 2022 @12:57AM (#30524253)
But can you use git for the .docx files?
Re:Write plain text files by sgjohnson on Wednesday March 02, 2022 @01:48AM (#30524503)
You can, but .docx is binary (it's .zip, and the Word document itself is OpenXML)

So you won't have a neat revision history, unless you implement something in the pipelines to also convert .docx to something human readable, because .docx and OpenXML isn't.