News for hackers. Stuff that matters.
Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by sunshine-o on Thursday December 26, 2024 @09:04AM (#42515206)
Looks cool, but as I have been on this knowledge management / productivity journey like everybody. Here are my findings:

If you are reasonably comfortable with computers / Unix.

- You need to first rely on a directory structures, filenames, plaintext, lists and maybe markdown. Stick with a "File over app", Unix approach.

- Try to sort things with universal concepts: locations, things, people, events, metrics, howtos. A bit like the 5Ws approach.

- Leverage good Unix tools: unix commands, make/justfiles, (rip)grep, git, fzf, etc.

- Do not try to solve the problem through the Web. Because you will end up trying to solve web problems instead of basic knowledge management and productivity issues.

- The smartphone/touchscreen is a major problem, but as with the Web do not try to solve it. Use your file manager or even fzf in termux can be adapted to be reasonably usable on a touchscreen.

Something I have been wondering about is the "backlink" feature. It would be cool to link items/notes together through references. What I would be looking for is a Unix tools that can scan my text files for references to other files in the hierarchy.

Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by skeeter2020 on Thursday December 26, 2024 @10:01AM (#42515595)
I've been experimenting with Flatnotes (https://github.com/Dullage/flatnotes) for a while and really like the design. No notebooks or even folders, just a single directory with markdown files and decent search & tagging. It feels a lot like what happened to email when we gave up all the up-front structuring with deep hierarchies and just said index it and we'll find it when we need to.

The project is just "good enough" for what I need, and aside from tiny bugs whenever I find a gap I can either work around it or live without. Constraints are a powerful motivator for both creativity and getting stuff done.

Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by zelphirkalt on Thursday December 26, 2024 @10:14AM (#42515674)
Tagging is a strictly more powerful tool than hierarchies, at least with same amount tags vs directories/categories, because an item can be tagged using multiple different tags, but can only be in one directory, unless you create duplicates or symlinks or whatever.
Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by cal85 on Thursday December 26, 2024 @12:08PM (#42516323)
I find the opposite. Being forced to consider hierarchical categorisation leads to a more powerful system in practice. It helps me create a mental reference to the item being stored. And it often causes me to see a better way to form an item in the first place — eg maybe this note is really two notes, maybe this idea can actually just be discarded, etc. Or, on rare occasions, a new item doesn’t fit anywhere (but is important) so I need to tweak the hierarchy itself — and that’s a good thing, once in a while, as it can lead to creative insights about the domain as a whole.

I’ve found tagging systems usually become a kind of dark swamp where things go never to be seen again. The lack of structure means I have little memory of what’s gone into the system, so I end up with too much duplication of ideas and inconsistency of style (mess). All this makes it uninviting and difficult to ‘explore’, so I don’t use it much except as a dumping ground, and the swampiness compounds over time.

Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by kortilla on Thursday December 26, 2024 @02:30PM (#42517166)
Hierarchy doesn’t make sense if you care about multiple dimensions, which I often do with notes.

Sometimes I want to look at all notes relating to c++, sometimes everything related to a personal project. Directories don’t support that without symlinking everything that mentions c++ into a folder for that.

Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by Charon77 on Friday December 27, 2024 @07:06PM (#42527293)
Hierarchy can do that too, once you allow a single file to exist in multiple hierarchies at once. Think of database, we have a single table, yet we have a lot of different indexes to search on.

We could do the same, either with symlinks or hardlinks

Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by sunshine-o on Thursday December 26, 2024 @03:50PM (#42517700)
I really like tagging but somehow the concept could never become central to any filesystem:

- I believe BeOS tried,

- MS tried with WinFS but cancelled it in Windows Vista,

- I am sure some cloud storage service bet on it but can't cite any.

Actually tagging is probably mainly successful in cases where we can reliably automate the tagging such as email or photos.

Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by setopt on Thursday December 26, 2024 @06:14PM (#42518615)
MacOS has had first-class support for file tagging for years: You can add arbitrary tags either from “Save as” dialogs or from Finder, and can then browse by tags in Finder or search through your file system filtered by tags.
Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by maroonblazer on Thursday December 26, 2024 @08:34PM (#42519255)
Does the native file tagging see much adoption? I'm a lifelong Mac user and tried using the tags when they first launched, but eventually it was too much work and I resorted to using search to find what I'm looking for. Not macOS native search but tools like LaunchBar, Alfred, et al or apps like EasyFind.
Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by nxobject on Thursday December 26, 2024 @09:02PM (#42519377)
Colours, yes – but then again I worked in a print shop that'd been using Apple since MacOS 8, so they might be used to those quirks.
Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by zelphirkalt on Thursday December 26, 2024 @07:45PM (#42519056)
I like tagging for bookmarks in Firefox and derivatives of Firefox.
Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by bloaf on Friday December 27, 2024 @09:41PM (#42528127)
Sharepoint tries to push people towards tags, but doesn't always succeed.
Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by speedupmate on Friday December 27, 2024 @02:57AM (#42520619)
Everything is a hierarchy, even the tags in your mind do belong under some tree/node like imagination. Tag is essentially just what you described "duplicates or symlinks or whatever."
Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by bloaf on Saturday December 28, 2024 @12:21AM (#42528803)
This isn't really true. Even mature, sophisticated attempts at making hierarchical classification systems (e.g. the UDC[0]) don't claim to have created a system where each document gets a unique leaf on a tree, and indeed instead attempt provide some syntax for adding additional metadata into its "nodes," i.e. the nodes become the sum of their tags.

[0] https://en.wikipedia.org/wiki/Universal_Decimal_Classificati...

Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by Y_Y on Friday December 27, 2024 @05:45AM (#42521251)
> but can only be in one directory, unless you create duplicates or symlinks or whatever.

Non-sym links do exactly this. Starting with your file in foo;

    ln foo/file bar/file
will have the file (really a reference-counted pointer) appear in both directories. It's the same location on disk and there's no overhead until you run out of inodes.
Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by WillAdams on Thursday December 26, 2024 @10:12AM (#42515666)
For a long while I was trying to keep text notes in a structured set of directories, Tombo.exe made that a lot easier/nicer to use:

https://openhub.net/p/p_7697

sadly, it looks to have dropped off the 'net.

Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by echelon on Thursday December 26, 2024 @10:01AM (#42515592)
Obsidian lets you live on top of the filesystem structure using markdown files, and it gives you links, tags, topic clouds, and more.

If you use it with reasonable conventions, you can compile and deploy a web version with ghost or zola or something automatically run in CI.

It's backed up, synced, and comes with desktop and mobile apps. It has vim key bindings and git plugins.

It's a pretty sweet system that works nicely with Unix, and it adapts to multiple navigational and organizational paradigms.

I've tried a lot of things, and nothing has ever come close to Obsidian.

Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by ruthmarx on Thursday December 26, 2024 @10:27AM (#42515731)
I just use TexStudio. I don't use proper tex files, but it never complains unless I were to try and compile. But their navigation tree and tabbed interface for multiple files makes organizing things really nice.

I can have multiple instances at once, and it's all managed via files and my filesystem.

Eventually I want to build a web frontend to index and reference my data, but for now the above approach is working well.

Unlike Obsidian, TexStudio is open-source, which I consider a plus.

Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by heavensteeth on Thursday December 26, 2024 @12:35PM (#42516489)
I'll leave my imcomplete mind dump below for posterity but I'll make it known that while writing it, I did realise that what I've invented is a text editor.

----------------------------------

I had an idea for a deeply unix-y note taking software a while ago and I've continued to think it would actually work very well.

Notes could either be an sqlite database (nets you compression, performance, easy backups, etc.) or just text files on the filesystem (easy searching, less lock-in, whatever). The "editor" just creates a new temporary file and opens it in `$EDITOR`. You can optionally name it after, otherwise it's named the current date.

Want linking? Use a filesystem-aware editor. Want syntax highlighting? Use an editor with syntax highlighting. Want to sync your notes? Combine it with Syncthing. Want to search your notes? fzf.

Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by packetlost on Thursday December 26, 2024 @01:26PM (#42516791)
I do this except with git, fzf, and a simple CLI tool:

https://codeberg.org/ngp/tsk

Contributions welcome :)

Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by exe34 on Thursday December 26, 2024 @12:43PM (#42516542)
org-roam gives you backlinks. directory of text files.
Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by skydhash on Thursday December 26, 2024 @12:59PM (#42516628)
After using Bear.app and Things.app for a while, I've moved firmly in the file over app camp. Apps can be enjoyable to use until you don't have access to the app anymore (platform, longevity,...). Now I'm using Emacs with org files, but could just as well use any tools that favors files. Any novel use case I want, I can script it out in a moment, while still enjoying the benefits of viewing my information on most platforms.

> Something I have been wondering about is the "backlink" feature. It would be cool to link items/notes together through references. What I would be looking for is a Unix tools that can scan my text files for references to other files in the hierarchy

The only thing you need is some kind of structure, like the wiki link format or org link format, and you could 'rgrep' the notes directory for a particular link (vim and emacs can put the result in a buffer).

Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by packetlost on Thursday December 26, 2024 @01:17PM (#42516735)
This is exactly the conclusion I came to. I still use Obsidian for a lot of things, but I've developed my own task management software that uses plain text files and fzf for everything: https://codeberg.org/ngp/tsk

It has worked great for me! That being said, sync and mobile usage is still a bit of a sore spot with it. It works beautifully with Termux, but I wish there was some way to slap a basic UI on top of a CLI application for mobile. Tk/Tcl is the closest but there's only options on Android (I'm mostly an iOS user) and even then it's not really ideal. For sync, I at least have a reasonable plan: IMAP4 or git-based sync. The roadmap and tasks for the project are tracked in-repo with itself, so it definitely works.

Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by withinboredom on Thursday December 26, 2024 @03:15PM (#42517454)
If you want some windows-forms-esq app development, I highly recommend checking out Flutter Flow. It isn’t exactly cheap, but I’ve used it for some basic personal apps.
Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by packetlost on Thursday December 26, 2024 @03:24PM (#42517506)
I kinda doubt FlutterFlow would meet my needs, it looks like it's primarily build on web technologies which is a nonstarter for this.
Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by vopi on Thursday December 26, 2024 @03:40PM (#42517626)
As far as I know, it's the opposite (using Skia to render). Flutter for the web is a second-class citizen, whereas for mobile apps, it's quite productive.
Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by withinboredom on Thursday December 26, 2024 @04:53PM (#42518149)
FlutterFlow is just the UI to build Flutter apps, which yes, is built on web-tech. However, Flutter runs natively on mobile devices.
Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by sunshine-o on Thursday December 26, 2024 @08:22PM (#42519213)
> but I wish there was some way to slap a basic UI on top of a CLI application for mobile

I have only relied on fzf so far but have played with the various components of the Termux:API [0] for basic UIs, and it works well.

There is also Termux:GUI [1] with Bash bindings which allows to build more advanced UIs.

- [0] https://wiki.termux.com/wiki/Termux:API

- [1] https://github.com/termux/termux-gui

Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by packetlost on Thursday December 26, 2024 @08:44PM (#42519299)
I'll have to look at Termux GUI again. I've tried poking around with Tasker and it's Termux plugin but it seems so confusing and poorly documented that I'd rather just write a native app at that point lol
Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by ankitrgadiya on Thursday December 26, 2024 @03:26PM (#42517522)
I have gone through the whole journey similar to you as well. It is so frustrating to keep searching for that perfect app only to find that every other app lacks something or the other. For the last year or so, I've settled on just plain markdown and the directory structure for hierarchy just like you.

I use my preferred editor (Emacs) to modify the files. I get to use all the functionality like key-bindings, search, version control. I push my notes to my self-hosted Forgejo instance (but it can be Github). Forgejo already has a web-interface the notes and links just work there.

On my laptop, I have mdbook configured to watch the directory and build the notes into static website. So if I just want to go through by notes or read something I can use that as well.

I tried finding solutions to take notes on my phone. But I realised that if its longer than a few lines than I prefer using my laptop anyways. I only ever read my notes on the phone. I've setup the same mdbook behind VPN so I can access it on my phone as well. If I ever want to modify notes on-the-go then I can use the Forgejo web interface as well.

Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by engfan on Friday December 27, 2024 @03:02AM (#42520628)
You should check out the HyWiki part of the Emacs Hyperbole hypertext package available from melpa or elpa-devel. It utilizes Org mode for notes and WikiWords to automatically interlink them as you type. The WikiWords work in any kind of text buffer as well, even in programming file comments. It even lets you link to many other kinds of Emacs entities like bookmarks, Org IDs and headings, etc. One command to build a web-based wiki. With a little practice, it might amaze you.
Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by melagonster on Thursday December 26, 2024 @08:46PM (#42519302)
>Something I have been wondering about is the "backlink" feature.

I trust this is why everyone tries to rely on network-like structure.

Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by cdaven on Thursday December 26, 2024 @11:51PM (#42520022)
For backlinks, have a look at Andy Matuschak's note-link-janitor: https://github.com/andymatuschak/note-link-janitor/

I wrote my own version of it a few years ago, but have moved on, and currently use LogSeq instead: https://github.com/cdaven/noteexplorer

Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by v3ss0n on Thursday December 26, 2024 @11:58PM (#42520050)
All falls Short when it comes to cross referencing, sub notes , rich capabilities.
Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by Abishek_Muthian on Friday December 27, 2024 @12:02AM (#42520071)
Great suggestions.

I'm quite happy with Obsidian on Linux, especially due to great plugin ecosystem.

Granted that it would have been great if the clients were open-source; but Obsidian is one of those non-FOSS apps which are so consumer oriented that they get an exception from me.

I sync the files from Obsidian using syncthing to Android & use Markor for editing. But this wouldn't be possible on iOS.

Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by Propelloni on Friday December 27, 2024 @07:21AM (#42521596)
Or, you could just fire up Zim Desktop Wiki [1], a local application you can run on any one of the major OS, and be mostly done with it. It gives you bullet 1, 2, 3[2], 4, and backlinks out of the box.

You get 5 if you are willing to compromise. Since it's just all text files in directory trees, you can sync the tree to your mobile phone (e.g with syncthing) and just use any text editor.

[1] https://zim-wiki.org/index.html [2] git with an add-on.

Re:Siyuan: Privacy-first, self-hosted personal knowledge man... by nchmy on Saturday December 28, 2024 @12:57AM (#42528940)
What do you mean by "the Web"? The internet, or some sort of network graph of notes (such as can be made in Obsidian)?

Either way, what is the problem with it?