ttkciar

llama.cpp uses a hard cutoff. The agent then does "something" that is specific to the agent's implementation and configuration. It might summarize and then "finish the thought" with a different model, and then resubmit the prompt to the llama.cpp API endpoint with <think>..</think> prefilled. The primary model then infers the remainder of the reply.

Huh. I thought hosting one's own databases was still the norm. Guess I'm just stuck in the past, or don't consume cloud vendor marketing, or something.

Glad my employer is still one of the sane ones.

Whoops, you are right. I misread the article.

Local inference users are all about sampling, but users addicted to commercial inference services are wary of sampling, because they have to pay by the token.

The author speculates that bigger/smarter models interpreting vague directives to utilize general-function tools will outperform more precise and detailed directives to utilize narrow-function tools:

> Granted to use a skill the agent needs to have general purpose access to a computer, but this is the bitter lesson in action. Giving an agent general purpose tools and trusting it to have the ability to use them to accomplish a task might very well be the winning strategy over making specialized tools for every task.

Well, that's just great.

The academic community has been using the term "skill" for years, to refer to classes of tasks at which LLMs exhibit competence.

Now OpenAI has usurped the term to refer to these inference-guiding .md files.

I'm not looking forward to having to pick through a Google hit list for "LLM skills", figuring out which publications are about skills in the traditional sense and which are about the OpenAI feature. Semantic overload sucks.

How do we deal with this? Start using "competencies" (or similar) in academic papers? Or just resign ourselves to suffering the ambiguity?

Or maybe the OpenAI feature will fall flat and nobody will talk about it at all. That would frankly be the best outcome.

Yes, you will need to employ someone with basic system administration competence. That's a given.

Cloud infra is touted as obviating the need to hire system administrators, but that's a marketing fabrication. Trying to manage infrastructure without the necessary in-house skills is a recipe for disaster, whether it's in the cloud or on-prem.

Senior software engineer with 46 years of experience (since I was 7). LLM inference hasn't been too useful for me for writing code, but it has proven very useful for explaining my coworkers' code to me.

Recently I had Gemma3-27B-it explain every Python script and library in a repo with the command:

$ find -name '*.py' -print -exec /home/ttk/bin/g3 "Explain this code in detail:\n\n`cat {}`" \; | tee explain.txt

There were a few files it couldn't figure out without other files, so I ran a second pass with those, giving it the source files it needed to understand source files that used them. Overall, pretty easy, and highly clarifying.

My shell script for wrapping llama.cpp's llama-cli and Gemma3: http://ciar.org/h/g3

That script references this grammar file which forces llama.cpp to infer only ASCII: http://ciar.org/h/ascii.gbnf

Cost: electricity

I've been meaning to check out Aider and GLM-4, but even if it's all it's cracked up to be, I expect to use it sparingly. Skills which aren't exercised are lost, and I'd like to keep my programming skills sharp.

After having worked on accounting software for a living, which did need to precisely take into account the variable nature of time measurements, I greatly appreciate that Numbat has decided not to open that can of worms.

Using averaged constants like this isn't applicable to all use-cases, but has the virtues of being simple and easy to reason about.

IMO they did exactly the right thing by not trying to shoehorn an endless supply of complexity and pain into their language's type system. Users who do need to deal with date math can use a library.

I tried very hard to like Ruby. "On paper" it looks great, but I had a lot of trouble getting used to the syntax.

I ended up going deep on the D programming language for my "new language". As a statically typed language it's not as expressive as the Python/Ruby/Perl contingent, but it's a lot more expressive than most static languages, and its C-like syntax "clicked" with me in a way Ruby never did.

It also can approach C-like run-time performance and memory footprint at times, which I appreciate. As much as I like developing in Python and Perl, I frequently wish they had better run-time performance and a smaller memory footprint. D gives me that, at the cost of a little expressiveness.

Working with Perl, two things spoiled me for other languages: JSON and DBI/DBD.

In Perl, everything serializes to JSON without fuss. It can be "lossy"; objects without an explicit JSON serialization method and coderefs can't be deserialized, but serializations at least have placeholders for them.

Compare to Python's json module, which doesn't try very hard to serialize things and throws exceptions every time it runs across something new. It's very frustrating to use.

Perl's DBI provides a universal API for all databases, with implementation-specific details in an underlying DBD module (which both provides glue between the DBI abstraction and programmer access to features specific to different database systems).

Compare to Python, where you need to import a different module and use a different API for every different kind of database. As I increasingly use Python for a living, I frequently wish they'd follow Perl's example with this.

> Proposing "fall back to Usenet" in 2023 makes about as much sense as "folks who use Slack should just use IRC".

Hey! :-( IRC is still alive and kicking, thank you very much!

> Are you just blanking out on the whole of the 1965-1980?

If you have to dig back half a century to make your point about contemporary events, you have effectively ceded the point.

> Rust had a lot of effort pored (and still going) into the nostd variant to be used in bare metal applications like microcontrollers and kernels.

D has had "dcompute" since 2017, for native execution on GPUs and other accelerators. People have been writing D for microcontrollers since at least 2011, possibly earlier. D is fairly mature in the systems space.

> I’m skeptical D would take.

Maybe, maybe not. Some languages "click" better than others for individual programmers. It's good to have options.

> it’s the first and only non C language in kernel mainline

Do you foresee it being the only non-C language used in the kernel forever?

Rust has a very enthusiastic following, and D is perceived as interloping on "their" territory because it is good at a lot of the same things.

> Doesn’t D require a GC to achieve the safety guarantees that Rust provides without one

No. The paper covers this. The authors used D's fat pointers, scope, slices, @safe, and ownership/borrowing features to improve code safety.

It is trivial to turn off D's GC, and the authors did not use it in the project described in their paper.

> and many third-party libraries require this GC?

Yes, a lot of D libraries depend on GC, which means they wouldn't be available for use for kernel development.

The D compiler will error out if D code marked with the @nogc attribute depends on GC (or depends on code dependent on the GC), which makes going GCless easier.

D has been around for 21 years. It has had a compiler in the gcc project (gdc) for four years. It debugs easily enough under gdb. Several editors support D syntax (including geany and vscode). It has quite a few libraries (though its interoperability with C libraries has muted native D library development somewhat), centralized at https://code.dlang.org/

Are there specific parts of the ecosystem you worry about not being mature?

As for it being a bit esoteric, yes, it's not a very widely known language, though I'm not sure why. I fell in love with it in 2018, and it has thoroughly spoiled me for writing C.

It always gives me a spell of cognitive dissonance when someone points out that Slackware is no longer a "major" distribution.

It used to be the major distribution. Funny how times change.

/me sees FVWM referred to in the past tense from his FVWM desktop, and scowls

My thought exactly. It even incorporates functionality for turning any local git repo into a shared git repo.

Mainly what they need besides that is a place to co-ordinate, collaborate and track tickets. Solutions abound.

The RIAA's actions are annoying but won't kill the project.

I don't disagree with anything you say :-) but it bears pointing out that D already lets you call into C libraries without glue.

This means you can write C and use the entire wide C ecosystem, but use D for the parts of the program which make sense in D, painlessly.

For what it's worth, I'm a long-time C programmer, but have piss-poor C++ skills, and despise C++, and D clicked for me immediately. I'm loving it.

Whether D "clicks" for someone might be more a matter of subjective taste than what languages they're used to.

As for D templates, they puzzled me until I read Andrei's D Programming book, which made everything clear. They're less narrow than C++ templates, more like directives for a syntax/scope-aware C preprocessor.

You can use them for more than just polymorphic functions; they're a way to pass source code around the program at compile-time, for almost any purpose whatsoever.

Google for "dlang" (including the quotes) and you'll get all the D hits you want.

.. and with much better run-time performance than Go.

D comes with a lot of "casual parallelism" baked in at the language level, and in its standard libraries. Actors are just one feature among many.

You -can- explicitly manage threads and processes like in other languages, but much of the time there is a more lightweight, expressive construct available.

For instance, you can turn this loop: foreach(ref i; arr) i = i * i;

Into a parallelized loop by simply: foreach(ref i; parallel(arr)) i = i * i;

Example was shamelessly lifted from: https://tour.dlang.org/tour/en/multithreading/std-parallelis...

We don't have an "NLP lab" here, but when my project required NLP solutions, perl delivered quite nicely.

Its runtime is quite a bit faster and enjoys a smaller memory footprint. It is also more amenable to "bash on steroids" scripting, and has a quarter-million canned solutions in CPAN.

PyPi has caught up with CPAN in terms of sheer number of packages, but there are still a lot of differences in what those packages cover. You might find solutions in CPAN without a counterpart in PyPi.

More narrowly, every time I deal with databases in python, I find myself missing perl's DBI, which provides the same API for all databases with a corresponding DBD. Python should get a similar universal database interface someday.

Since the language-previously-known-as-Perl6 will be known as Raku, there shouldn't be any conflict with Perl v6

I am fascinated by the economy of packing three inaccuracies into that one sentence.

(1) X11 is not a window manager,

(2) There have been several really effective window managers for X11,

(3) MS-Windows has never been good at remote display.

Yes, what Grinnz says. "Bash on steroids" is one of perl's strongest use-cases. It's why a lot of system administrators learned perl.