News for hackers. Stuff that matters.
Re:Manualslib – Database of More Than 2.6M Manuals by Mister_Snuggles on Monday June 05, 2017 @03:15PM (#14490483)
Backups are one of those things that I don't do well at the moment. Fortunately, Mayan doesn't have any special requirements - basically back up the database and back up its media directory[0].

My ideal solution is to have everything backed up "in the cloud" plus have a backup at home. So far I've got the home backup happening - every machine uses 'attic' to back up onto my NAS. I'm still figuring out what the "in the cloud" side looks like.

One thing you can do is keep copies of important documents elsewhere - like Dropbox, a USB drive in a safe deposit box, etc. It all depends on what technologies you trust and how much you trust them.

For OCR, I can't say I've had much trouble with it. Mayan breaks the PDFs into pages, then feeds each page to tesseract. All of that happens in a worker process and I haven't found that it has a significant impact on the use of Mayan. The OCR section for the document will be incomplete for a while though.

[0] http://mayan.readthedocs.io/en/v2.1.4/topics/backups.html

Re:Manualslib – Database of More Than 2.6M Manuals by coaxial on Monday June 05, 2017 @05:39PM (#14492380)
I don't trust dropbox, at all. I'd like to store encrypted backups online for the Mayan db and every document.

For the OCR, my issue is that if I import several documents Mayan will feed them all to tesseract in parallel. Then I have two available cores, but 40 tesseract workers and the system comes down because it runs out of RAM and CPU. How did you set up your OCR?

Re:Manualslib – Database of More Than 2.6M Manuals by Mister_Snuggles on Monday June 05, 2017 @08:30PM (#14493696)
I just used the instructions here[0], with some minor changes (pathnames, db server, etc). I've never seen more than two tesseract processes running on my two-core VM, no matter how many documents I give it.

Maybe something changed at some point to limit the active workers to the number of cores.

[0] http://mayan.readthedocs.io/en/stable/topics/deploying.html

Re:Manualslib – Database of More Than 2.6M Manuals by siloraptor on Tuesday June 06, 2017 @12:45AM (#14494797)
Mayan EDMS issue to add rate limit support to the OCR backend. https://gitlab.com/mayan-edms/mayan-edms/issues/287
Re:Manualslib – Database of More Than 2.6M Manuals by Mister_Snuggles on Tuesday June 06, 2017 @11:12AM (#14497833)
I wonder if there was a change in celery that limits the number of active workers to the number of cores. This is the behavior I see on my instance.
Re:Manualslib – Database of More Than 2.6M Manuals by coaxial on Tuesday June 06, 2017 @01:37PM (#14499196)
That could explain it, it would be a nice addition