nickpsecurity
Further, in machine learning, we see lots of papers with math that doesn't lead to anything useful. Most papers claiming advances over prior architectures don't pan out in later testing. So, there will be skepticism when people make performance claims like the authors'. The best response is just independent testing.
1. English specification of the system and its goals
2. Formal, mathematical specification of the system and its goals
3. Mathematical proof the system met its goals
4. Code implementing that system with a proof, often informal, that the code embodied the informal and formal design.
5. Tests of all functionality to prove the design and code worked.
6. Independent analysis of the above to find flaws in them.
7. If the implementation has concurrency or parallelism, more errors can occur due to that. There were techniques to mitigate it.
Each step found problems in systems made by professionals. Mathematical work and coding both have a long history of errors in their on fields. Combining them increases the likelihood of error.
But God gave us beautiful oceans and intellectually-stimulating games. I'm sure some chess players aren't wasting the opportunity to enjoy both. Now, they're at risk of being taken out!
The safest way to do that is a high-assurance, data diode (one-way link). One builds hardware that, physically and by software, only allows information to flow one day. Many are available commercially.
Sometimes you need proof of acknowledgement of each packet but nothing else. Naval Research Laboratory had The [Network] Pump that allowed only Ack's back and with algorithm to throw off covert, timing channels.
Next, one might want to let people send them requests over an API which they can respond to in limited ways. One can use a high-assurance guard with secure, hardware/software architecture. It uses simplified I/O with a message format that's easy to parse securely.
At this point, I'm also thinking it might be better to use the closed-source, proprietary solutions. The AI's are rapidly scanning the open-source code for vulnerabilities which I'm sure they're not always sharing. They'd be more likely to have attacks on it.
So, don't tell the untrusted model how the guard works, use private models + traditional verification to secure its code, and include both HIDS and NIDS monitoring. Also, make the systems as predictable and boring as possible to make anomalies easier to spot.
Most of IT security ignored that work and never used the methods. That's why their syatems can't defend against them. There are hardware and software groups that built countermeasures. They're not mainstream. I don't know if the hardware needed is even commercially available (eg partitioning caches, async & randomized CPU's).
Looking at what's out there, I think GenodeOS should be mentioned for application-specific systems. They can even build some of their stuff on seL4 if you want.
I'll also add there's been unikernel projects that aren't in exotic languages. Even C++ (IncludeOS?). There's probably one in Rust by now. If not, it could probably build on Redox's components.
I'm also using BunnyCDN right now.
So, I think there's still potential to throw together an interface even if it can't do all of it. We do need a placeholder box that can be labeled for the controls we add manually but use its sizing info. Also, a converter that autoscales them in HTML or Qt to maintain the ratios that were in the VB6 app. For the placeholder, I was temporarily using that one control that's like a giant frame or box meant to contain others.
One reason I abandoned the idea is the AI's are so good now that we can probably just generate better GUI's from scratch or a basic, VB6 sketch. I couldn't justify the work until I try an AI with such projects. To avoid hallucinations or garbagr designs, might make sense to do the original program but AI-generate the transpiler.
Then, it was just that it made an exploit, and works really well, and people might use it instead of their products. Tragic for their investors I guess...
So, your program that combines source files or checks dependencies would be fully specified in its success and failure states. Only combinations of functions leading to a provably-secure state are even allowed. If you can't do that, the feature is too complex to allow. Human pentesters review it from design to algorithms to building it to spot ways attacks might happen.
That's what it takes to build software that usually resists subversion. Most software isn't built that way. It can't be because the priorities of developers and customers work against it. So, we'll continue to see clever attacks that exploit systems not designed to high security standards.
For this topic, I recommend David A. Wheeler's page on Software, Configuration Management Security because it covers many issues with it in mostly-centralized systems.
1. Most laws are made about humans. If it's AI, it's often treated like a tool the human is using. So, change "I did this with AI" to "I did this with (other tool here)." The case law on those situations might give hints to what will happen.
2. Intent matters. Did you intend to do damage?
3. If a tool might cause damage, but you didn't prevent that, then someone might claim negligence. There's a lot of legal articles about torts for damages due to negligence. I personally believe a lot of agent use should be considered negligent. By default, I don't connect them to the Internet or my whole filesystem because I know they might do unforeseen damage.
Those are the three that come to mind most in such cases. You'd have to ask a lawyer. There's another risk of even using a lawyer, though.
For using AI agents, you must consider civil and criminal law because its problems are spread across them. Most lawyers in my area do one or the other. You might have to pay two retainers at $5,000-$8000 each or one, expensive firm with combined expertise. Just knowing your legal risk with agents might cost more than they'd make or save you vs just using human-driven AI's.
1. CPU/memory performance on cheap or throwaway systems has a niche benefit. Mostly poor people. I'm still usually on an ancient Thinkpad with a 2nd gen i7. It runs native apps really fast to this day. Almost all GUI cuz I agree with the OP but my apps are TUI by default to keep them lean and fast.
2. Security. While I don't aim for it these days, it was much easier to make textual apps securely than GUI apps. Secure OS's from the 90's already secured console apps. TX, Nitpicker, and EROS made progress on GUI's but there's high complexity still. I'll note the OP's idea of GUI front ends is basically what we did to isolate the GUI part in a dedicated partition with messages it sends checked by the secure component.
Other than those observations, I'm with OP where I'm tired of TUI's if GUI's are that easy now. I considered trying it with some lightweight, cross-platform frameworks. Anybody tried some with cheap AI's?
Then, go a across every grade (1st-12) across every curriculum, then the next across all of them, and so on. Checkpoint it at each grade level. Also, see how many epochs we need per grade to soak up the material. Dedicated fine-tuning for each grade matched to its capabilities. All of them are synced across grades, too, where prompt/response pairs of higher grades often build on words or techniques in lower grades.
Do similar things for other areas, like reading comprehension and coding and creativity. Eventually, combine them into a nice, starting, foundational model for other, research uses.
Beware of copyright issues for API's and patent issues about reimplementations. Wolfram seems serious about his I.P.. After the Oracle case, I'm not reimplementing any language unless it's open with no patent trolling possible.
https://allenai.org/blog/olmo3
That 7B model also worked well in my experiments on a laptop.
If you had a deadly condition, and no diagnosis worked, and a specific model had the answer... past that I wouldn't use it.
If so, I could use a larger model than I have real-time hardware for. The largest, well-trained models can often get the output mostly right in one try. I also would be using AI's as a supplement to, not replacement for, my own brain. So, issues with the outputs wouldn't be a problem because I'm just keeping what's helpful.
If I still need to re-generate it all, it might still save money over time by avoiding cloud costs. Also, hardware that's already paid for is a sunk cost that doesn't inflate over time. Glitches in loading or destroying VM's might blow up into a big bill.
What I thought would be useful is, like Ironsides DNS, a SPARK Ada or other implementation that shows no code injections could ever happen from loading, modifying, or rendering text. That's a useful subset of full verification.
If not that verified, writing things in a memory-safe, concurrecy-safe language covers lots of ground. Rust and Pony put good effort in those areas. In Rust, I think you still had to manually turn on checks for some overflows which hurt performance a lot. So, static analyzers or automated provers for range properties have a performance benefit.
Muen is the largest, production project I know in such a language:
Ironsides was an earlier project: