dogline

This is a breakdown of how I built a book scanner using my iPhone camera, and the problems and issues I ran into. With pictures. Normally, book scanning can take excessive time and ideally, a scanning rig, but AI and OCR can handle page skew and motion pretty well now.

Remember, if all you need is a git remote to push to somewhere via ssh, `git init --bare` on your server should be available and is solid if you don't need a web interface. Now you've got something you can push to on a shared server.

Unless you really also want a web interface, issue tracking, et. al. Then, use any of these fine tools. Go wild!

Good, rational points from an expert. Not AI slop. It's silly that we have to separate this now.

Well, this post made me reassess using zswap vs zram. I've expanded my zswap and disabled zram. Not sure if that makes any difference to others, but it's probably good for me.

Wait, what? This sounds important, but I can't quite parse it.

In my case, I'm just trying to make a list of official sources, so there's minimal data that can go stale. Although I'm using Claude to help put this together, I'm reviewing most everything, or I gave Claude the list I want to include from my own searches.

Is there liability for posting something like this? I don't think so, but can't say that I'm certain. It would be sad if there is liability.

UDF: User Defined Function

Except for history, I don’t find much that stops you from switching back and forth on the CLI. They both use tools, each has a different voice, but they both work. Have it summarize your existing history into a markdown file, and read it in with any engine.

The APIs are pretty interchangeable too. Just ask to convert from one to the other if you need to.

I think it's becoming clear we're not anywhere near AGI, we figured out how to vectorize our knowledge bases and replay it back. We have a vectorized knowledge base, not an AI.

With lots of people showing how Saas apps can be easily written these days, I'm not as interested in those articles, as people showing off new ideas of what I can do with these new found abilities. This is cool.

I would drive hours to get to the nearest Fry's to me, to pick up some new gear. Being able to browse everything and look around was great. For me, online ordering of parts probably hurt Fry's, but the real reason was after a while, you were never sure if the video card you were buying was new, or actually a return item, and after a couple times having to drive all the way back for something that was missing parts, the whole thing just seemed way too risky. Amazon and Newegg nailed that door shut.

You're right, and it's a good idea. The summary started out small, as a header to the actual daily pages, but then I realized I could have AI do a lot more work here, including silly things like collect weather references and assemble them together. My prompt kept getting bigger to find trends in the data. But, it takes away from the view-ability of the site, which is not good.

LLM's ability to take 7400+ handwritten entries and try to make a narrative out them is amazing. With all of the AI experiments on HN lately, we're figuring out the power of LLMs, but it most cases, it still needs a human refining touch, and we need to remember that. Or else it just looks like AI slop.

That's cool! I've noticed when asking for Claude for a website, it does have a certain look, like our two sites, if you don't give it any more guidance. I'm not sure if that's a good thing or not.

Digitizing history in different ways, with different resources that are unique or only known to small groups, might be a new development area, and that's exciting. As I've shown, and how other people have shared, using AI tools to digitize things which haven't previously been done before is now possible. Are there ways to make this easier for everybody? New techniques to discuss? I don't know, and I'd love to talk about it.

Concerning OCR: I used Mistral because of a posting here describing advancements with handwriting recognition a month or so ago. I didn't actually compare them. And I've got my setup that I can rerun everything again later if there are advancements in the area. Again, another area to keep track of and discuss.

Beej! Thank you very much! Your networking guides have long been a great contribution to everybody, and collectively improves what we know.

These diary pages come largely from Stirling City, just north of Chico, and later from the Hat Creek district, on Hwy 89 north of Mt. Lassen. Nearby, many historical records were lost in the Paradise Camp Fire, and digitizing some of the records in some of the local museums is something this is a test run for.

  —Lance (CSU Chico ‘93 Computer Engineering)

Yeah. If there are groups that want the high resolution images, talk to me.

Hadn't thought about it, but will take a look. Also, the two Forestry type links look very interesting. I figure there must be interest in this sort of thing - this is one resource, and the Stirling City Historical Society (Lassen NF) has a bunch of other documents I'd love to digitize soon.

I did a quick search, mules are mentioned 75 different times. Like this one at random from Sept 1942: https://forestrydiary.com/page/019bd90a-f176-713f-9999-b14b6...

"Fix up my packs. Load the 2 mules with 225# each. Take the 2 loads to trail camp at Lake Everett, Unload. Have lunch with the Trail cook. Haze mules & ride to 7 1/2 PM."

Horses are mentioned 2586 times. That'd be a whole study on how they're used in the back country. (Edit: horse number is inflated since part of the diary form at one point asks for "Horse Mileage". Will have to refine search).

Oh boy. #3 on front page, 19k page hits in the first hour. 8243 static html pages, 15728 webp images (10k-50k each).

I've never had one of my sites with this much traffic. With everything as static files, website is still holding. Thank you all.

Also, just to clarify, I scanned all 7488 pages in personally (Fujitsu ScanSnap ix500). With Claude's help, I found some undocumented SANE features to auto crop and fix the scans, then had a Python script in Linux auto scan them and put them into a Postgres database as I went. Other scripts would add transcription, summaries, and auto index everything.

"mistral-ocr-latest" did really good handwriting transcription, considering how tight and small some of the handwriting is. Then back to Claude API calls to summarize by month and collect people and places from all of the entires.

Claude then created static html pages from what started as a Flask app. Published on Dreamhost.

I like this. It's rather creative.

I have custom scripts I use at home to keep track of various personal data, assisted by an LLM. The idea of using Telegram as a way to have a global, quick, and personal interface from my phone or tablet, is perfect and easy to set up.

Claude is making it easier to have bespoke data and dashboards for anything. We're going to make a lot of them, for all reasons. I've also made apps with Django interfaces, but quick, global interfaces are going to become in demand.

Isn’t that how a lot of us learned — buy typing the code out of back of a magazine? Then spending hours trying to debug a typo somewhere.

I didn’t realize how close LLMs are to the old magazines. Let it give you a seed, then use that springboard to learn everything else.

Until just now, I've been trying to figure out why people think that JSON is necessary in the database? Yes, lots of data is hierarchical, and you just normalize it into tables and move on. The fact that some people don't work this way, and would like to put this data as it stands into a JSON tree hadn't occurred to me.

What problem does normalization solve? You don't have to parse and run through a tree every time you're looking for data. You would, however, need to rebuild the tree through self joins or other references in other cases, I suppose. It depends how far you break down your data. I understand that we all see data structures a bit differently, however.

We'll have to get the old (webrings)[https://en.wikipedia.org/wiki/Webring] back in fashion.

You gave an IOCCC snippet as an example of a C99 coding trick you know? I mean, the code looks visually cool, but it's funny to explain a code concept using code shaped like an anime character. (At least that's what I think it is.)