News for hackers. Stuff that matters.
Re:Borb – A Python library to read, write, and edit PDF files by Syzygies on Saturday September 18, 2021 @07:23AM (#28575290)
I am a math professor with a scanned exam grading workflow that I hacked together as Bash scripts using various open source command line tools. I feed all the exams through a sheet-fed scanner, decode bar codes to identify problems and students, add radio buttons for entering and tracking scores (0-6 per problem), and create PDF "books" per problem for grading and annotating.

Having grad students help grade paper is a consistency nightmare: It's look once, never look back. Instead, after each of several provisional passes I recreate the PDF "book" for that problem, with a chapter for each score, and students randomized within each chapter. In the same spirit as "checking your work lets you work three times faster" this is actually both more consistent and faster that a single pass over paper. Almost all of my attention is on the math, which I'm good at, rather than locating problems and finding again the ones I know I misgraded, which I'm not good at.

Then each student's exam needs to be extracted from these problem PDFs, scores recorded, and annotations frozen.

There are cloud services for grading. They're hopelessly primitive, with cloud lag. Like a gamer, I used to reject wireless mice because of the lag. I reject these services. I can grade everything myself faster than using a team of grad students, with the right local tools.

The PDF format is a morass. My hats off to anyone who will work with it. There are many evolutionary layers and no formal specification or verification; one tests a PDF by seeing if most programs accept it.

It's time for me to rewrite my grading system in a modern scripting language, so others could use it. I prefer Ruby, but that's mainly to stave off boredom when I'm not using Haskell. I can use Python. This would permit a more robust workflow, such as adding late exams in mid-grading without losing grading in progress.

I can't find documentation for Borb, to check off the list of features I'd need. I suspect from this being a one-person project that I might need to continue to patch together external tools.

Re:Borb – A Python library to read, write, and edit PDF files by Svetlitski on Saturday September 18, 2021 @12:39PM (#28577476)
You should consider looking into Gradescope (Gradescope.com). As a former TA, I can attest to it making grading much more pleasant and streamlined than it would be otherwise.
Re:Borb – A Python library to read, write, and edit PDF files by Evidlo on Saturday September 18, 2021 @08:33AM (#28575717)
> There are many evolutionary layers and no formal specification or verification;

There is a specification, but it's very complicated.

Re:Borb – A Python library to read, write, and edit PDF files by hyperpallium2 on Saturday September 18, 2021 @09:17AM (#28576013)
PDF uses postscript, which is Turing equivalent. It's a document format with the halting problem.
Re:Borb – A Python library to read, write, and edit PDF files by mkl on Saturday September 18, 2021 @11:26AM (#28576966)
PDF does not use Postscript, and is not Turing complete. Its drawing model is based on Postscript's (with additions), but its instruction set is focused on drawing, and can't do programming. Here is the instruction set, with equivalent Postscript commands where applicable: https://www.adobe.com/content/dam/acom/en/devnet/pdf/pdfs/PD.... Most parts of the Postscript language have no PDF equivalent.

Other things like JavaScript and Flash can be embedded, but they are extras on top of the document.

Re:Borb – A Python library to read, write, and edit PDF files by dunham on Saturday September 18, 2021 @11:25AM (#28576963)
It's based on postscript, like json is based on javascript. I don't believe there are any control flow (or even general arithmetic?) instructions in the content streams, so I don't see how it could be Turing complete. It's just a sequence of drawing and transformation commands, like SVG.

I presume the extensions to embed javascript are Turing complete, and IMO do not belong in a PDF file.

I've also heard that some of the embedded font formats have features that are Turing complete, but I don't know the details on that.

If there is a way to implement a Turing machine in PDF, outside of the fonts and javascript, I'd love to see the details. (I know somebody managed it with just the macro expansion bit of TeX.)

Re:Borb – A Python library to read, write, and edit PDF files by maxerickson on Saturday September 18, 2021 @10:44AM (#28576649)
Postscript files are often rendered into PDF, but PDF doesn't use postscript.
Re:Borb – A Python library to read, write, and edit PDF files by Syzygies on Saturday September 18, 2021 @04:17PM (#28578948)
See dunham's answer. Anyone who has seen a PDF file in text form would swear they're looking at Postscript. There's a common intersection that's identical.

Very roughly speaking (this is a semantic debate where everyone is wrong from someone else's perspective), a PDF file is a restricted subset of Postscript, with added indexes so one can render pages in the middle without having to process the code from the beginning.

The hardship in generating PDFs from scratch is getting those indexes right. It's far easier to convert a Postscript file using standard tools.

Re:Borb – A Python library to read, write, and edit PDF files by maxerickson on Sunday September 19, 2021 @03:09PM (#28587004)
That's fair. I was waffling about using stronger language, "pdf isn't postscript", but didn't get there. It would have been more correct.
Re:Borb – A Python library to read, write, and edit PDF files by kaba0 on Sunday September 19, 2021 @04:07AM (#28582595)
The same way as JSON is a subset of JS, it has radically different properties that way, so comparisons to postscript is not really meaningful.
Re:Borb – A Python library to read, write, and edit PDF files by einpoklum on Saturday September 18, 2021 @10:16AM (#28576448)
That is not a realistic concern for GP. The scanned exams won't even involve any Postscript.
Re:Borb – A Python library to read, write, and edit PDF files by Syzygies on Saturday September 18, 2021 @04:06PM (#28578866)
I do insert the score radio buttons using the "pdfmark" mechanism via Postscript.
Re:Borb – A Python library to read, write, and edit PDF files by selfhoster11 on Saturday September 18, 2021 @04:14PM (#28578927)
HTML uses JavaScript which is also Turing complete. Lack of the halting problem is nice, but having it is not a show stopper.
Re:Borb – A Python library to read, write, and edit PDF files by Tijdreiziger on Saturday September 18, 2021 @08:19AM (#28575639)
Perhaps you might be interested in the Zesje project: https://gitlab.kwant-project.org/zesje/zesje
Re:Borb – A Python library to read, write, and edit PDF files by mkl on Saturday September 18, 2021 @11:56AM (#28577174)
More information: https://sandbox.grading.quantumtinkerer.tudelft.nl/, https://zesje.tudelft.nl/about/

It sounds like it's still mostly a prototype?

Re:Borb – A Python library to read, write, and edit PDF files by Tijdreiziger on Sunday September 19, 2021 @05:21PM (#28588032)
Sorry, I should have elaborated in my initial comment.

I was briefly involved as a developer several years ago (as part of my bachelor's thesis). At that time, it was mostly beta-quality, but it was already in use by multiple professors for grading. I haven't been involved with the project since, so I'm not sure about the current status.

I think the homepage [1], which you linked to and where it mentions that it's still a prototype, is at least somewhat outdated; it has a screenshot of a very old version of the software. At least the 'support' section still looks accurate, though.

If you're interested in using it, I would advise getting in touch via the Mattermost channel or mailing list (both linked to from the homepage [1]) and asking about the current state of the project. Tell them Jamy sent you :)

[1] https://sandbox.grading.quantumtinkerer.tudelft.nl/

Re:Borb – A Python library to read, write, and edit PDF files by jhgb on Saturday September 18, 2021 @08:14AM (#28575603)
> Having grad students help grade paper is a consistency nightmare: It's look once, never look back.

Captcha-ize them, with several of them grading the same result, and with checking their responses against each other?

Re:Borb – A Python library to read, write, and edit PDF files by gettalong on Monday September 20, 2021 @04:23AM (#28591423)
If you want to do something like this in Ruby, have a look at HexaPDF - https://hexapdf.gettalong.org/ - which provides a full-blown implementation for reading and writing PDFs and is quite mature already (n.b. I'm the author).

It is licensed AGPL+Commercial but if you just use it for yourself, this does not matter as you can use the AGPL.

Re:Borb – A Python library to read, write, and edit PDF files by strzibny on Saturday September 18, 2021 @12:53PM (#28577568)
Would be cool to know the differences to Ruby's HexaPDF. One is certainly the license.
Re:Borb – A Python library to read, write, and edit PDF files by senorsmile on Sunday September 19, 2021 @01:24PM (#28586105)
Would you be willing to list all of the command line tools you're using?