News for hackers. Stuff that matters.
Re:Pure Java Typesetting System by whartung on Monday January 23, 2023 @01:01PM (#34492470)
What a fascinating license.

This is a particularly compelling line from it:

  3. You must not distribute the modified file with the filename of the
     original file.
Just in isolation, outside of the other aspects of the license, this is pretty much completely crippling for a Java program.

Simply, if you change a file, you can't use the same name. Well, in Java, the name of the file is the name of the class. You can't have the file name differ from the class inside.

That means you can't change a Java file in this codebase.

Fascinating.

Re:Pure Java Typesetting System by thangalin on Monday January 23, 2023 @01:21PM (#34492780)
I've been looking high-and-low for a way to rename Java classes en masse without breaking the build.

https://stackoverflow.com/questions/75154524/bulk-rename-of-...

A number of classes have been renamed to have the Kt prefix. There are other changes that don't yet have the new prefix applied because changing those classes would ripple into, essentially, all the entire classes. Hence the search for a mass rename procedure that works. Suggestions welcome.

Donald Knuth set that rule up so that "plain.tex" would be the same, world-wide. I don't know why the authors applied the same license to the entire Java application, rather than limit it to Knuth's work.

I've also added a comment "// Originally" that indicates the original name of the class, rather than the name of the file. This is a minor violation of the license, but still keeps the spirit if not the letter of the license. Otherwise there'd be no way to extract an inner class.

Re:Pure Java Typesetting System by counttheforks on Monday January 23, 2023 @02:33PM (#34493760)
If you're fine using intellij then you can probably use its scripting to do this. It has access to most of the same API as plugins and can access intellij features such as its refactor rename.

The (meager) docs on scripting: https://www.jetbrains.com/help/idea/ide-scripting-console.ht.... The plugin docs & intellij source code (especially the unit tests) are a better source of information.

And here's a script that will use intellij's refactoring to rename all classes recursively in a directory to add the prefix Renamed, correctly dealing with references & renaming files if necessary:

  @file:Suppress("NAME_SHADOWING")

  import com.intellij.notification.Notification
  import com.intellij.notification.NotificationType
  import com.intellij.notification.Notifications
  import com.intellij.openapi.actionSystem.*
  import com.intellij.openapi.keymap.KeymapManager
  import com.intellij.openapi.command.WriteCommandAction
  import com.intellij.psi.*
  import com.intellij.psi.search.*
  import com.intellij.refactoring.rename.RenameProcessor
  import com.intellij.util.ThrowableConsumer
  import java.io.PrintWriter
  import java.io.StringWriter
  import javax.swing.KeyStroke

  // Usage: In IDEA: Tools -> IDE Scripting Console -> Kotlin
  // Ctrl+A, Ctrl+Enter to run the script
  // Select folder containing target classes, Ctrl+Shift+A to open action menu, search for Bulk refactor

  //<editor-fold desc="Boilerplate">
  val b = bindings as Map<*, *>
  val IDE = b["IDE"] as com.intellij.ide.script.IDE

  fun registerAction(
    name: String,
    keyBind: String? = null,
    consumer: ThrowableConsumer<AnActionEvent, Throwable>
  ) {
    registerAction(name, keyBind, object : AnAction() {
      override fun actionPerformed(event: AnActionEvent) {
        try {
          consumer.consume(event);
        } catch (t: Throwable) {
          val sw = StringWriter()
          t.printStackTrace(PrintWriter(sw))
          log("Exception in action $name: $t\n\n\n$sw", NotificationType.ERROR)
          throw t
        }
      }
    });
  }

  fun registerAction(name: String, keyBind: String? = null, action: AnAction) {
    action.templatePresentation.text = name;
    action.templatePresentation.description = name;

    KeymapManager.getInstance().activeKeymap.removeAllActionShortcuts(name);
    ActionManager.getInstance().unregisterAction(name);
    ActionManager.getInstance().registerAction(name, action);

    if (keyBind != null) {
      KeymapManager.getInstance().activeKeymap.addShortcut(
        name,
        KeyboardShortcut(KeyStroke.getKeyStroke(keyBind), null)
      );
    }
  }

  fun log(msg: String, notificationType: NotificationType = NotificationType.INFORMATION) {
    log("Scripted Action", msg, notificationType)
  }

  fun log(
    title: String,
    msg: String,
    notificationType: NotificationType = NotificationType.INFORMATION
  ) {
    Notifications.Bus.notify(
      Notification(
        "scriptedAction",
        title,
        msg,
        notificationType
      )
    )
  }
  //</editor-fold>

  registerAction("Bulk refactor") lambda@{ event ->
    val project = event.project ?: return@lambda;
    val psiElement = event.getData(LangDataKeys.PSI_ELEMENT) ?: return@lambda

    log("Bulk refactor for: $psiElement")

    WriteCommandAction.writeCommandAction(event.project).withGlobalUndo().run<Throwable> {
      psiElement.accept(object : PsiRecursiveElementWalkingVisitor() {
        override fun visitElement(element: PsiElement) {
          super.visitElement(element);
          if (element !is PsiClass) {
            return
          }

          if(element.name?.startsWith("Renamed") == false) {
            log("Renaming $element")

            RenameProcessor(project, element, "Renamed" + element.name, false, false).run();
          }
        }
      })
    }
  }
If your project is large this will temporarily freeze intellij since it doesn't properly run on the background thread but just give it a minute.

By the way, this should work on all languages that IntelliJ has a language plugin for, not just Java.

Re:Pure Java Typesetting System by thangalin on Monday January 23, 2023 @03:14PM (#34494308)
This looks rather promising. Doesn't work in my version of IDEA (Build #IC-223.8214.52, built on December 20, 2022, Community Edition), due to the following errors when running the script:

> Argument for @NotNull parameter 'module' of org/jetbrains/.../...GroovyRunner must not be null.

The project doesn't have any references to the NotNull annotation, and there are a number of what appear to be compile errors:

https://i.ibb.co/dfxbV4p/rename-script.png

Running with Ctrl+Enter produces:

    > @file:Suppress("NAME_SHADOWING")
    MultipleCompilationErrorsException: startup failed:
    Script1.groovy: 1: Unexpected input: '@file:' @ line 1, column 6.
       @file:Suppress("NAME_SHADOWING")
            ^

    1 error
I removed the line and re-ran, which resulted in:

    > import com.intellij.notification.Notification
    [22 ms]=> null
Then I selected the entire text and pressed Ctrl+Enter again, which returned:

    MultipleCompilationErrorsException: startup failed:
    Script4.groovy: 20: Unexpected input: '<*' @ line 20, column 25.
       val b = bindings as Map<*, *>
                               ^

    1 error
After changing the asterisks to question marks, the script failed to run, indicating:

    MultipleCompilationErrorsException: startup failed:
    Script5.groovy: 25: Unexpected input: 'registerAction(\n    name: String,\n    keyBind: String? =' @ line 25, column 22.
           keyBind: String? = null,
                            ^

    1 error
I appreciate the effort, though!
Re:Pure Java Typesetting System by counttheforks on Monday January 23, 2023 @03:17PM (#34494355)
Make sure you pick Kotlin for the scripting language, not Groovy.

Then run the script like this:

1. In IDEA: Tools -> IDE Scripting Console -> Kotlin

2. Paste in the script

3. Ctrl+A, Ctrl+Enter to load the script. This should show a green "Loaded!" notification

4. Select a folder containing the classes in the Project window

5. Press Ctrl+Shift+A to open action menu

6. Search for Bulk refactor and select it

Also please note the first version of the script had a bug that would open confirmation dialogues for some refactorings, so see the edited script.

Re:Pure Java Typesetting System by thangalin on Monday January 23, 2023 @04:08PM (#34495047)
> 1. In IDEA: Tools -> IDE Scripting Console -> Kotlin

No such sub-menu of Tools exists, but that's fine because I've mapped Ctrl+Shift+A to bring up the menu. I installed the Kotlin plugin and tried again. There was a missing import. After fixing the missing import:

    [732 ms]=> null
I switched "Rename" to "Kt", of course, and re-ran the script.

> 4. Select a folder containing the classes in the Project window

> 5. Press Ctrl+Shift+A to open action menu

> 6. Search for Bulk refactor and select it

Opening the menu with Ctrl+Shift+A doesn't show the bulk refactor script, but that could be because I remapped Ctrl+Shift+A and am using the NetBeans keyboard mappings, rather than the IntelliJ map.

https://i.ibb.co/5LTsffj/bulk-refactor.png

Re:Pure Java Typesetting System by counttheforks on Monday January 23, 2023 @04:31PM (#34495371)
> There was a missing import

Oops :)

When pressing Ctrl+Enter, did you see the green "Loaded!" notification in the bottom right? If so, then I guess installing the Kotlin plugin worked.

> Opening the menu with Ctrl+Shift+A doesn't show the bulk refactor script

The Ctrl+Shift+A menu from the default keymap that I'm talking about is the "Actions" dialog. You can also get there via "Navigate -> Search Everywhere" and then selecting the "Actions" tab (or just searching in All).

Also you only have to select a single folder/package and it will be processed recursively. I did not test what happens when you select multiple folders. Might be fine, might explode.

edit: Just realized the script I pasted here does not emit the "Loaded!" log message on load. So disregard those comments. If you want to verify it loads correctly, add this as the last line:

    log("Loaded!")
Re:Pure Java Typesetting System by thangalin on Monday January 23, 2023 @07:58PM (#34497563)
There are some minor clean-up issues, but overall, it did the trick.

https://i.ibb.co/MP2JNRM/classes-renamed.png

Thank you!

Re:Pure Java Typesetting System by layer8 on Monday January 23, 2023 @02:56PM (#34494072)
Ignoring the case of the TFA license that actually requires you to rename the files, renaming packages is normally enough, and is rather straightforward, which is probably why there isn’t much existing tooling for batch-renaming classes.
Re:Pure Java Typesetting System by patrec on Monday January 23, 2023 @05:13PM (#34495867)
Wouldn't something like the following shell one-liner rename get you quite close?

   export javas=$(find . -regex '.*\.java$'); for old in $(grep -hrPo 'class (?!Kt)\K[A-Z]\w+'); do echo $old; sed -i "s/\<$old\>/Kt$old/g" $javas;  rename "s/^$old\$/Kt$old/" $javas; done
Untested, unoptimized, needs gnu grep and perl's rename utility (variously packaged as prename or rename-files, as opposed to the crappy linux-utils rename).
Re:Pure Java Typesetting System by patrec on Monday January 23, 2023 @05:54PM (#34496280)
Here you are, "gradle build" works with this cleaned up and faster version (and the diff looks plausible on first sight):

   javas=$(find . -regex '.*\.java$')
   sed -i -E "$(printf 's/\\<(%s)\\>/Kt\\1/g;' $(grep -hrPo '\b(class|interface|record|enum) (?!Kt)(?!List\b)(?!Entry\b)\K[A-Z]\w+'))" $(echo $javas); 
   rename 's;\b(?!Kt)(\w+[.]java)$;Kt$1;' $(echo $javas)
(The $(echo $javas) is so it works in both bash and zsh).
Re:Pure Java Typesetting System by KRAKRISMOTT on Monday January 23, 2023 @01:18PM (#34492722)
Or just use a shading tool (which comes by default with most Maven/Gradle builds). It is not a barrier for any experienced java developer.
Re:Pure Java Typesetting System by ebruchez on Monday January 23, 2023 @01:14PM (#34492654)
As a workaround, you could distribute the modified Foo.java as Foo2.java, and have your build system rename Foo2.java to Foo.java at build time. It might be all it takes (but IANAL).
Re:Pure Java Typesetting System by mqus on Monday January 23, 2023 @02:33PM (#34493772)
WHich would completely cripple IDEs or at the very least makes it difficult to work with
Re:Pure Java Typesetting System by bedatadriven on Monday January 23, 2023 @01:20PM (#34492753)
Looks like it was inherited from an upstream project that has a long history: https://github.com/jamespfennell/new-typesetting-system#lice...
Re:Pure Java Typesetting System by dcminter on Monday January 23, 2023 @01:19PM (#34492745)
Huh? Refactor the class name in virtually any Java supporting IDE and the file name and invocations will be changed along with it.

The license term seems petty, and would be annoying, but the workaround is trivial.

Re:Pure Java Typesetting System by thangalin on Monday January 23, 2023 @01:38PM (#34493025)
> but the workaround is trivial.

If you have a trivial solution to rename hundreds of Java classes and their references en masse without breaking the build and without renaming each file one-at-a-time, do tell. You can find more details about potential solutions I've tried here:

https://stackoverflow.com/questions/75154524/bulk-rename-of-...

Re:Pure Java Typesetting System by playingalong on Monday January 23, 2023 @01:47PM (#34493131)
I bet Open Rewrite can handle that quite easily: https://docs.openrewrite.org/
Re:Pure Java Typesetting System by thangalin on Monday January 23, 2023 @02:03PM (#34493367)
I wasn't able to find a recipe that performs a regular expression rename. That is, there doesn't seem to be anything like:

    ---
    type: specs.openrewrite.org/v1beta/recipe
    name: com.yourorg.ChangeTypeExample
    displayName: Change type example
    recipeList:
      - org.openrewrite.java.ChangeType:
          oldFullyQualifiedTypeName: (.+)
          newFullyQualifiedTypeName: Kt$1
Re:Pure Java Typesetting System by Someone on Monday January 23, 2023 @02:17PM (#34493544)
I would do that the hacker way:

- build the current TeX.jar

- jar tf TeX.jar | grep whatever to get a list of to-be-renamed classes

- use sed or manual editing to generate a large list of recipes

- run those recipes

Instead of jar tf, grep for “^package foo.bar.baz” and “class (complex regex)” in the source code to get a list of package and class names, or use the directory structure. Maybe do all of them, and compare the results for robustness.

That list will be large and it may not go perfect in one go, but it’s a one-off job, and you‘ll get there.

Re:Pure Java Typesetting System by dangerlibrary on Monday January 23, 2023 @01:50PM (#34493176)
This feels like something where the right solution is to drop down to the terminal and use the old standbys - sed/grep/find, etc. Could probably wrap it all up in `make` or `just` if this is something you do regularly.
Re:Pure Java Typesetting System by thangalin on Monday January 23, 2023 @02:05PM (#34493398)
It's not trivial. Some class names are substrings of other classes. I'm pretty sure this requires loading and parsing the source code's AST to accomplish without breaking the build. The SO thread goes into more gnarly details.
Re:Pure Java Typesetting System by dangerlibrary on Tuesday January 24, 2023 @01:03PM (#34507381)
Sorry, I didn't mean to imply it was trivial, by any means. Just that there are powerful tools for "manipulating many line-delimited text files at once" and those tools seem likely to fit your goal.

It's possible you do in fact need to parse the AST for your goals - I haven't spent the requisite time to deeply understand the problem.

Re:Pure Java Typesetting System by dcminter on Monday January 23, 2023 @02:08PM (#34493438)
Edit: Never mind, whartung enlightened me in a sibling comment. Leaving the egg on my face for posterity below.

---

Why do you want to change all the files? The license only says you have to change the names of files that you modify. It's a pretty weird term.

If for some reason you were determined to change the names of all files then you ought to be able to create a plugin that walks the tree of classes and invokes the refactoring capabilities of the IDE to make the change. It's a strange use-case though so I'm not really surprised there isn't a ready tool for it.

Re:Pure Java Typesetting System by thangalin on Monday January 23, 2023 @02:27PM (#34493679)
> Why do you want to change all the files?

Ever pull on a thread to unravel an entire sweater?

Imagine wanting to add an SVG typesetter. To do this, the interface implemented by the DVI typesetter needs to be extracted and generalized so that it works for both SVG and DVI (to maintain existing behaviour). The `Typesetter` interface is used by, for example, `BaseNode`, so any changes to that interface could touch `BaseNode`. Changes to `BaseNode` can affect all `*Node` classes, which then cascade over to `*Noad` and `*Prim` classes, and from there the rest of the source base.

Re:Pure Java Typesetting System by counttheforks on Monday January 23, 2023 @02:43PM (#34493890)
Posted a solution on your other comment: https://news.ycombinator.com/item?id=34493760
Re:Pure Java Typesetting System by whartung on Monday January 23, 2023 @01:33PM (#34492951)
If you do that, you have to rename every class file it impacts as well, because now those files are changed. A lot of refactoring systems don't take into account the ancillary files (like build files, start scripts, etc.).

So, the workaround can be quite impactful in the end. Renaming the entirety of the code base, potentially.

Re:Pure Java Typesetting System by dcminter on Monday January 23, 2023 @02:08PM (#34493448)
Oh, duh, now I get it. That's very annoying.
Re:Pure Java Typesetting System by layer8 on Monday January 23, 2023 @03:08PM (#34494251)
In principle, you could rename the classes on the class-file level after compilation, back to the original names. That wouldn’t violate the license.

Moreover, under the following definition in the license:

  `Modification' of a file
  means any procedure that produces a derivative file under any
  applicable law -- that is, a file containing the original file or
  a significant portion of it, either verbatim or with modifications
  and/or translated into another language.
…compilation would count as modification (“translated into another language”), and compilation also changes the file name (from .java to .class), so distributing the compilation output should be fine. Furthermore, any changes to the sources can be distributed as a diff patch, which also has a different file name.
Re:Pure Java Typesetting System by hbn on Monday January 23, 2023 @03:28PM (#34494521)
Now I want to make a fork, DumbKeenType, where the first thing I do is append "Dumb" to the beginning of every class/filename.

https://www.newyorker.com/tech/annals-of-technology/dumb-sta...

Re:Pure Java Typesetting System by brazzy on Monday January 23, 2023 @03:36PM (#34494626)
> in Java, the name of the file is the name of the class. You can't have the file name differ from the class inside.

Actually, I'm fairly sure you can. You just need a custom classloader.

Re:Pure Java Typesetting System by brabel on Monday January 23, 2023 @04:05PM (#34495014)
Class files don't actually care about file names so ClassLoader has nothing to do with this. The OP problem is the source. The Java compiler won't compile a source file if none of the classes (and the only public one) in the file matches the name of the file.
Re:Pure Java Typesetting System by brazzy on Monday January 23, 2023 @05:00PM (#34495721)
> Class files don't actually care about file names

But URLClassLoader does, which definitely has something to do with this.

True, before you get there, you'd first need it to compile - but modern JDKs have a compiler API that allows customization via the StandardJavaFileManager interface, and that looks like it should be able to do the job to make source file names follow a different convention.

Re:Pure Java Typesetting System by rowls66 on Monday January 23, 2023 @05:08PM (#34495820)
Yes, commonly used ClassLoaders do care about file names and directories. The Java compiler javac does not. The javac compiler compiles compilation units which are generally files, but name does not matter.
Re:Pure Java Typesetting System by teraflop on Monday January 23, 2023 @05:23PM (#34495965)
No, the file names of compilation units do matter to javac. To quote the Java Language Specification, section 7.6:

> If and only if packages are stored in a file system (§7.2), the host system may choose to enforce the restriction that it is a compile-time error if a class or interface is not found in a file under a name composed of the class or interface name plus an extension (such as .java or .jav) if either of the following is true:

> * The class or interface is referred to by code in other ordinary compilation units of the package in which the class or interface is declared.

> * The class or interface is declared public (and therefore is potentially accessible from code in other packages).

and javac does indeed enforce this restriction, at least for public classes. It's an error to declare a public class called Foo in a file that's called anything except Foo.java.

Re:Pure Java Typesetting System by rowls66 on Tuesday January 24, 2023 @10:27AM (#34504744)
I stand corrected. However, I don't believe that the class source file needs to be in a directory (or hierarchy of directories) that matches the package name. Many IDE's enforce this as well, which I believe is not correct.
Re:Pure Java Typesetting System by brabel on Friday January 27, 2023 @06:03AM (#34544739)
Every class file contains the full information about the class name, package and everything needed to instantiate the class... it even includes the source class file name usually and other attributes like statement line numbers.

However, for performance reasons, the JVM loads classes lazily and will only try to find classes in the directory it would be expected to be (either in the directories included in the classpth or in jars). Because you can write your own class loader, you can implement something different (like OSGi does for example - it has much more metadata so it can even load multiple classes with the same name from different jars) but this is the default behaviour.

Re:Pure Java Typesetting System by brabel on Tuesday January 24, 2023 @02:42AM (#34500666)
Such an easy statement to check and quickly prove completely wrong.

    echo "public class Foo {}" > Bar.java
    javac Bar.java

    Bar.java:1: error: class Foo is public, should be declared in a file named Foo.java