steve1820

Thanks, reading it now. Good points on memory usage and latency.

Thanks! I appreciate the encouragement :-)

Apologies for that Colin! I'm using Squarespace for my personal blog.

I'll investigate further to make it more readable.

EDIT: spelling

I'll have a read...very interesting!

Haha! Talk about learning something new every day!

Interesting points regarding the internal mental model!

I agree, for effective information retrieval we (as humans) need to remember the context/ mindset where/when we consumed the knowledge.

I haven't really thought about this problem. It is definitely something to ponder on.

I probably won't be maintaining this unfortunately :-(

You have a good point about interoperable components!

Awesome open source project as well!!

Good find.

It was my first time playing around with Electron! Overall pretty decent experience.

What makes you hate it if you don't mind me asking?

Vaporware is better than going nowhere! (Get it...noware...haha).

Congrats on getting started.

I agree with Obsidian - I think that most people forget the maintenance time it takes to build a lifelong Knowledge Management System.

I like your idea - document similarity is a well known area in ML.

Feel free to take my Chrome Extension and use the parts where it tracks key paragraphs in an article (using a user's click/ hover/ attention behaviour) and use that as the corpus for your ML similarity models.

Intuitively it makes more sense to run document similarity on key points/ paragraphs than the whole web page.

If you want the whole web page though, there's code in the Chrome Extension that use's Mozilla's readability lib (https://github.com/mozilla/readability) to purify the web content.

Forgot to add: It also tracks what you click so it can build relationships between articles/ blogs/ anything else you read on the internet.

For example, if I’m reading an article about big data/ ETL pipelines and then I click on a link from within the article to a resource on machine learning, this relationship should be tracked and digested.

No worries! I'm always leeching from the open source community so I'd thought to give back haha.

The visualisation library I'm using is by Ant Design (Alibaba).

You can see some examples below from the documentation.

https://g6.antv.vision/en/examples/gallery

Re syncing with real CouchDB - I haven't implemented that yet but from what I read, its definitely possible with PouchDB.

Re I have a bunch of Express APIs that expose PouchDB so I can do basic CRUD operations such as creating resources etc.

Awesome that we came up with similar ideas!

So right now, the Chrome Extension tracks all link clicks as well so the visualisation actually builds relationships between articles/ blogs/ anything else you read on the internet.

I also had another idea that once we build our Knowledge Maps, we should be able to compare and share it will one another.

For instance, I'd love to see what our software engineers around my age/ experience are reading and the insights they are drawing from articles.

Obviously some sort of security/ privacy mechanism will need to be implemented as well.

Thanks man! I really appreciate it :-)

This blog is a summary of a fun 1 month adventure I had with Knowledge Management Systems and building a POC that I thought had potential. It was inspired by so many of the products I see on Hacker News.

I’ve open sourced all the code + written down some notes on my insights/ architecture. Just a warning, the code leaves much to be desired as this was a mini project over 2-3 weekends.

At the moment, I have 3 key layers of the application.

The first is Chrome Extension which allows data collection. It tracks all sorts of data such as attention, time on page, scroll/ click and hover behaviour. It then sends this data to the Electron app.

The Electron app receives this data through an API and saves it down locally using PouchDB (data structure is compatible with Apache CouchBD and thus allows easy cloud storage).

Within the Electron app, I use an Express/ NodeJS web server to expose endpoints for functionality with the front end/ receiving data from Chrome Extension.

For the front end I use ReactJS.

If you’re interested, please read the full article! There are lots of videos of the application in action.

Done! It was an issue with my gitignore not committing build related files. Fixed that. Thanks!

Let me fix this when I get home. My bad!

Proof of concept!

This blog is a summary of a fun 1 month adventure I had with Knowledge Management Systems and building a POC that I thought had potential. It was inspired by so many of the products I see on Hacker News.

I’ve open sourced all the code + written down some notes on my insights/ architecture. Just a warning, the code leaves much to be desired as this was a mini project over 2-3 weekends.

At the moment, I have 3 key layers of the application. The first is Chrome Extension which allows data collection. It tracks all sorts of data such as attention, time on page, scroll/ click and hover behaviour. It then sends this data to the Electron app.

The Electron app receives this data through an API and saves it down locally using PouchDB (data structure is compatible with Apache CouchBD and thus allows easy cloud storage).

Within the Electron app, I use an Express/ NodeJS web server to expose endpoints for functionality with the front end/ receiving data from Chrome Extension.

For the front end I use ReactJS.

If you’re interested, please read the full article! There are lots of videos of the application in action.

This blog is a summary of a fun 1 month adventure I had with Knowledge Management Systems and building a POC that I thought had potential. It was inspired by so many of the products I see on Hacker News. I’ve open sourced all the code + written down some notes on my insights/ architecture. Just a warning, the code leaves much to be desired as this was a mini project over 2-3 weekends.

At the moment, I have 3 key layers of the application. The first is Chrome Extension which allows data collection. It tracks all sorts of data such as attention, time on page, scroll/ click and hover behaviour. It then sends this data to the Electron app.

The Electron app receives this data through an API and saves it down locally using PouchDB (data structure is compatible with Apache CouchBD and thus allows easy cloud storage).

Within the Electron app, I use an Express/ NodeJS web server to expose endpoints for functionality with the front end/ receiving data from Chrome Extension.

For the front end I use ReactJS.

If you’re interested, please read the full article! There are lots of videos of the application in action.

I'd definitely encourage more regulation in this area.

It's going to be a balancing act but at the moment, its is way too skewed towards the big corp in the corp vs consumer arguement.

Quite an interesting read!

Definitely travel to Japan! Do it while you're young.

You literally have your whole life ahead of you - 1 or 2 years in Japan will give you experiences you've never even thought about!