News for hackers. Stuff that matters.
Re:Gains I'm seeing from my second brain tool by endisneigh on Monday January 31, 2022 @02:12PM (#30152677)
I wish there were a tool that could just record everything - text, web history, clicked links, video chat, messages, etc. and just index it.

fundamentally I believe "taking notes" is the wrong way to think about it. if you "know" everything, then notes serve no purpose.

Re:Gains I'm seeing from my second brain tool by Vetch on Monday January 31, 2022 @03:00PM (#30153421)
I run into this sentiment often and I believe it's fundamentally mistaken as to the purpose of notes.

Recording is not knowing, at best it is memorization of largely useless trivia. Sherlock has a good discussion on this (Sherlock solar system). To know something you have to have processed and at the very least achieved some compression of it.

Notes help with processing and thinking things through, aiding conscious and active understanding.

Unless recordings feed into one's process, they're a waste of space. Sturgeon's Law: The vast majority of what we encounter is junk, much of it even self-contradictory. The more you let into your corpus, the harder the problem of retrieval relevance.

My clippings are actively curated, carefully sized to fit in working memory while also independently comprehensible and guaranteed to have met my minimal standards on factuality. Because I have read them carefully before, I expand my mental canvas but am not slowed down too much by processing demands. The throttling on note growth means an easier time for information retrieval and much fewer irrelevant results to ignore.

Re:Gains I'm seeing from my second brain tool by bhelkey on Monday January 31, 2022 @05:27PM (#30155570)
Recording isn't knowing but archiving is better than losing.

As an exercise, look back at bookmarks that are 5+ years old. How many of them still load? Look back at your history of liked videos on YouTube. How many of them have been deleted?

Re:Gains I'm seeing from my second brain tool by gumby on Monday January 31, 2022 @02:46PM (#30153234)
> fundamentally I believe "taking notes" is the wrong way to think about it. if you "know" everything, then notes serve no purpose.

I used to think that but the act of taking the note is itself important, as it integrates some of the info into your decision making process.

The indexed database would also be useful, but you have to understand enough to search in the first place.

Re:Gains I'm seeing from my second brain tool by igammarays on Monday January 31, 2022 @02:43PM (#30153186)
That's what I use DevonThink for. Local macOS app with encrypted local filesystem. It just indexes basically everything from email to webpages to actual directories of plaintext files on my filesystem. And it can do automatic OCR of images/PDFs etc.

I don't do actual note-taking there, I use NotePlan for that, which is just a directory of plain text files on my local filesystem. Automatically indexed by DevonThink.

I just dump everything there, including a regular weekly cronjob to archive my email accounts from different IMAP servers including Gmail.

I use DevonThink regularly and it never fails to deliver (except syncing, which sucks, my databases are too large). I have 25+ GB of data there in all sorts of file formats, from PDF receipts to over 1 million+ text documents including emails, and DevonThink can still find stuff instantly (under 500ms) for any full text search.

Re:Gains I'm seeing from my second brain tool by 9dev on Monday January 31, 2022 @02:16PM (#30152719)
Wouldn't that essentially be a system that tracked the entirety of your digital life in a sort of timeline, with metadata attached, making it easily accessible and searchable? I have thought a lot about this and curiously, it's also what many science fiction depictions of computer systems seem to aim for - a single, integrated, computing experience, across all devices and tools.
Re:Gains I'm seeing from my second brain tool by shepherdjerred on Monday January 31, 2022 @02:22PM (#30152841)
This is something I think about often when it comes to tools. The idea that Google had all of my location history for the past 8 years means I can see everywhere I've ever gone. I can lookup any email I've ever sent/received. I can look through my browser history for any particular day. With Spotify + Last.fm I can see stats about my listening history, and with Tract.tv I can see my TV/movie watching habits. Strava tracks exercise, and YNAB tracks budgeting/spending. There's an incredible amount of health data from my iPhone + Apple Watch.

It's really cool to think that all of this data exists, but I'm not really sure that it's useful. Occasionally I've gone back through my old search history for that one old post that I wanted to re-read, or go through my Spotify listening history to find a song I really enjoyed but forgot the name of, but 99% of that data goes to waste.

Quantified Self [0] is an interesting project to me since it seems to make use of data from all of these disparate sources, but I'm still not sure what the outcome of tracking all of this data is. It seems cool but not useful.

[0]: https://quantifiedself.com/

Re:Gains I'm seeing from my second brain tool by anamax on Monday January 31, 2022 @09:59PM (#30158099)
> I can lookup any email I've ever sent/received.

Not with gmail you can't.

As of a couple of years ago, gmail search was a crap-shoot. It would consistently fail on some things, occasionally fail on others, and work just fine on cases that looked (to me) to be the same.

Yes, this was on a paid account.

Re:Gains I'm seeing from my second brain tool by shepherdjerred on Monday January 31, 2022 @10:00PM (#30158107)
I use an IMAP client with Gmail and it works great. I agree that the web version of gmail has pretty poor search (which is extremely ironic)
Re:Gains I'm seeing from my second brain tool by anamax on Monday January 31, 2022 @10:19PM (#30158224)
I didn't know that IMAP supported search.

https://datatracker.ietf.org/doc/html/rfc1730#section-6.4.4

Re:Gains I'm seeing from my second brain tool by Jeff_Brown on Monday January 31, 2022 @03:13PM (#30153596)
It's exciting that the data exists, but Google's (and every similar organization's) business model is not compatible with full data transparency. They want you locked into their platform, using their tools.
Re:Gains I'm seeing from my second brain tool by shepherdjerred on Monday January 31, 2022 @05:31PM (#30155641)
Yes exactly! It would be nice if this data were more opened up so that tools around the data could be created.
Re:Gains I'm seeing from my second brain tool by hateful on Monday January 31, 2022 @02:43PM (#30153187)
I think it's the same with a lot of other things, such as backups or proper testing - you wouldn't say backups were a waste because you never used them and you wouldn't say tests were a waste because they always passed!

It's also like all the wires I have in boxes and draws at my house. As soon as I throw them out I need them!

Re:Gains I'm seeing from my second brain tool by EvanAnderson on Monday January 31, 2022 @05:17PM (#30155446)
I've dumped my SMS logs into my IMAP mailbox since 2010. I've used the "History Trends" browser extension and religiously archive my browser history. I store client IP addresses and timestamps from my IMAP server as another data-point on my physical location in the world (i.e. what network my phone is connected to when it checks-in for email). My credit card transactions dump to email as well. EXIF from photos is helpful but I haven't automated dumping those into my email.

Most of my personal "note taking" is either time tracking entries for Customer billing or emails sent to myself with keywords in the subject line and the notes in the body.

I wish I could get at my phone's (iPhone) browser history, call database, and location services database easily to amalgamate into my "digital life panopticon" too.

I love being able to search over even this limited corpus of data. I can "remember" where I was and what I was doing on a given day pretty easily. I discovered that I can often get "organic" recollection when I prime my brain with a few context details gleaned from the digital corpus. ("I was at xxx site that day, I had lunch at yyy restaurant, and when I got back I looked at these websites... Ah, and now I remember what Bob and I talked about that afternoon!")

I'd love to have audio and low frame-rate video recording on my person during all my waking hours. If I had that, though, I'd keep it completely secret (and use parallel reconstruction if I need to divulge "memories" gleaned from it to others).

Re:Gains I'm seeing from my second brain tool by primarydonkey on Monday January 31, 2022 @03:58PM (#30154226)
Re:Gains I'm seeing from my second brain tool by rzzzt on Monday January 31, 2022 @05:14PM (#30155391)
"There is no worrying about whether a document belongs in the Letters folder or the New Project folder. It simply takes its rightful place in the time-ordered stream." https://www.wired.com/1997/02/lifestreams/
Re:Gains I'm seeing from my second brain tool by echelon on Monday January 31, 2022 @02:40PM (#30153150)
> fundamentally I believe "taking notes" is the wrong way to think about it.

Hard disagree for a few reasons.

Even with history, sometimes I forget how to look up a paper I saw six months ago. A lot of stuff sits at the tip of my tongue, and I'm unable to adequately describe it well enough to find it again. Or if I can find it, it takes a great deal of effort. I'm pretty sure this problem isn't unique to me.

But even more importantly, writing helps your brain build connections. When you have to analyze, synthesize, and summarize, your brain is actively grappling with, contextualizing, and relating new concepts to the world you're already familiar with. It becomes an active part of your worldview.

Coupled with spaced repetition, note-taking is one of the best tools for learning.

Re:Gains I'm seeing from my second brain tool by Jeff_Brown on Monday January 31, 2022 @03:44PM (#30154039)
> analyze, synthesize, and summarize

Yes. Summary in particular is surprisingly mind-expanding.

Also don't forget classifying! Note taking apps for e you to do that, and it can lead to important insights.

Re:Gains I'm seeing from my second brain tool by endisneigh on Monday January 31, 2022 @02:45PM (#30153218)
we'll have to agree to disagree - everything you're saying is also true even if you take notes, except your notes are just a lossy form of reality itself. reality that could just be captured directly and indexed. computers can form the connections with human assistance. no notes required.

notes have a purpose, but most people that I talk to use notes to basically just record reality to remember details

Re:Gains I'm seeing from my second brain tool by echelon on Monday January 31, 2022 @02:53PM (#30153310)
No hard feelings :)
Re:Gains I'm seeing from my second brain tool by sergiotapia on Monday January 31, 2022 @03:26PM (#30153787)
I am building something like this. It's my most ambitious project to date and I know it will require a large team of people to support it's various connections.

Think automatically downloading all youtube videos you liked, all tiktok videos you heart, all tweets you favorite, make a crisp HTML backup of every bookmark you have. It will grab all the videos you "save" and automatically ffmpeg to the resolution of your choosing (in my case x265 720p).

It's a large effort, but one that I think it 100% worth it. People are losing their connection to the past on the whims of a few corporations. It's time to decentralize that.

A swarm of my apps all online, all uploading/downloads the content as people search for it and consume it. No taking it down since there is nothing to take down.

Coming soon!

Re:Gains I'm seeing from my second brain tool by karlicoss on Monday January 31, 2022 @03:24PM (#30153753)
This is my approach!

I'm using HPI [0] as a sort of universal API for almost all of my data (manual notes, bookmarks, instant messages, internet comments, etc)

Then I use it in tools like Orger [1] and Promnesia [2,3] which function as my second brain

[0] https://github.com/karlicoss/HPI

[1] https://github.com/karlicoss/orger

[2] https://beepb00p.xyz/promnesia.html

[3] https://news.ycombinator.com/item?id=23668507

Re:Gains I'm seeing from my second brain tool by jasode on Monday January 31, 2022 @03:03PM (#30153479)
>I wish there were a tool that could just record everything [...] if you "know" everything, then notes serve no purpose.

I guess the completeness of your proposed "auto-memorize-everything" solution depends on what style of notes a person writes.

A lot of my notes are annotations or personal commentary that don't exist in the sources of the url text or video.

Here's an example of that type of notes I often take... E.g. I found some links that explain how one can grok "Docker" by going through the learning exercise of re-inventing it from basic os features with homemade scripts. But I wrote my own personal notes (in "##")about the limitations of the explanations in the urls:

  ##  these tutorials explain the verb (runtime) but not noun (artifacts of image file)
  https://github.com/p8952/bocker/blob/master/bocker
  https://www.youtube.com/watch?v=8fi7uSYlOdc
Because content cannot meta-analyze itself and enumerate all the ideas and concepts that are not embedded in its own text, it means a hypothetical indexing engine that converted the 2 links above to searchable data still would not retrieve the extra meta information I wrote. A lot of good notes are generated from your own brain.

EDIT add to categorize different types of "notes" to aid discussion:

(1) notes as memorization aid such as copying facts: a user writes a note that says "1 inch equals 25.4 millimeters" after seeing it on a webpage or something. So the "universal-record-and-index-everything" would help make these type of notes obsolete because one could just skip writing that note down and just recall it later by searching for "inch to mm conversion" in the digital archive

(2) notes as synthesizing/interpreting/connecting/commenting the content. These type of idiosyncratic notes generated by the user's brain often don't exist on the internet so they cannot be replaced by a universal recording tool.

Re:Gains I'm seeing from my second brain tool by sdepablos on Monday January 31, 2022 @03:13PM (#30153604)
Of course the best option is doing BOTH things: indexing the context automatically via a modern personal search engine that accepts all kind of media and does vector / full text and relation indexing, plus the information wroten by the user
Re:Gains I'm seeing from my second brain tool by flurie on Monday January 31, 2022 @02:34PM (#30153035)
The closest thing I've seen to that is https://github.com/novoid/Memacs
Re:Gains I'm seeing from my second brain tool by thinkling on Monday January 31, 2022 @03:28PM (#30153808)
There have been a number of research prototypes in that direction including Susan Dumais' Stuff I've Seen [1], which was Windows desktop-oriented but I think had the right idea.

> much knowledge work involves finding and re-using previously seen information. We describe the design and evaluation of a system, called Stuff I’ve Seen (SIS), that facilitates information re-use. This is accomplished in two ways. First, the system provides a unified index of information that a person has seen, whether it was seen as email, web page, document, appointment, etc. Second, because the information has been seen before, rich contextual cues can be used in the search interface.

Might be worth revisiting their papers to see what they learned.

[1] (PDF of SIGIR paper) https://www.microsoft.com/en-us/research/wp-content/uploads/...

Re:Gains I'm seeing from my second brain tool by randito on Monday January 31, 2022 @02:48PM (#30153253)
Ink and Switch have done a lot of thinking and a lot of experiments around this idea.

https://www.inkandswitch.com/capstone/

Re:Gains I'm seeing from my second brain tool by avian on Monday January 31, 2022 @02:31PM (#30152995)
Maybe APSE is what you're looking for [1]. A while back the founder sent me a link after one of my blog posts hit HN. It's a tool that continuously records your desktop and offers text search of everything through OCR.

I personally found the idea interesting, but I was too afraid to ever try it out. The mere idea of a video record existing of everything that's going on on my computer, even if it's never supposed to leave the computer, was too scary for me.

[1] https://apse.io

Re:Gains I'm seeing from my second brain tool by gopalv on Monday January 31, 2022 @03:06PM (#30153509)
> I believe "taking notes" is the wrong way to think about it

Recording only the inputs doesn't help me much, my notes are mostly always about the part of my brain it ticked off.

A lot of it is "I wonder if this will help me with X" connection points left off for me to follow up later rather than to go back to refer to the conversation in exact terms.

Good notes for me are a way of letting off my internal narrative pick up from there later and carry on paying attention to the conversation I'm actually having right now.

Re:Gains I'm seeing from my second brain tool by mattlondon on Monday January 31, 2022 @02:26PM (#30152897)
Pretty sure that is what happens with Google if you use their ecosystem (i.e. use Chrome, Google search, Gmail, hangouts, meet, docs, YouTube etc)

But people tend to dislike this idea that one company knows everything.

Re:Gains I'm seeing from my second brain tool by XorNot on Monday January 31, 2022 @04:20PM (#30154554)
tracker in theory should be really useful on the Linux desktop because it did pretty much all of this.

In practice I don't know how or why it currently feels so unusable.

Re:Gains I'm seeing from my second brain tool by trenchgun on Tuesday February 01, 2022 @12:03PM (#30165559)
There is. It's called programming.
Re:Gains I'm seeing from my second brain tool by barapa on Tuesday February 01, 2022 @01:36AM (#30159318)
I worked on such a tool at Google
Re:Gains I'm seeing from my second brain tool by inshadows on Monday January 31, 2022 @06:58PM (#30156657)
Have you heard of Google? :-)