If it’s making reviewed boilerplate how is it making a judgement call?

Having human beings write copious tedious boilerplate isn’t good. It takes the judges mind and applies to tedium rather than using judgement. Further under tedium humans make mistakes.

> “Has the jurisprudence of the constitutional court made favorable decisions in similar cases?”

That's a judgment call because "favorable" is subjective. Someone who gets sent to jail for 1 year might tell you the prosecutor got a favorable ruling while the prosecutor considers it unfavorable because they wanted the person sentenced to 5 years.

Note if you read it carefully he only included the responses but wrote parallel arguments and then synthesized the two. So, like I said, it was reviewed.

It’s like complaining engineers use Monte Carlo simulations, which are random, to do analyses of structural integrity instead of doing all the calculations using a human computer[1]. I see these LLM chatbots as being akin to a computational tool like a calculator. It’s great for crunching and provides a force multiplier for a skilled person using it to augment their own abilities, but it’s (not yet) a replacement for the expert.

[1] https://en.m.wikipedia.org/wiki/Computer_(occupation)

It’s not really clear to me from reading the article what it did here, but it sounds like the judge asked the bot if there is precedent. If that’s the case, what if the issue is nuanced and the precedents are conflicting - the bot might provide favourable rulings which check out under review, but omit unfavourable ones.

The judge wrote a parallel analysis and included the chatgpt answer, an analysis comparing them, and a final analysis incorporating both his reasoning and the chatgpt reasoning.