SkyBelow

>its information density is often really low

I find that it often isn't just too low, but also too high at the same time. It has a lot of information that is irrelevant, burying far too little relevant information for the work needed to extract it. I wonder if this is a side effect of attention mechanism, where the AI's ability to track information in a document is far less hampered by other information, leading to it organizing information to be both dense per word but any specific concept to be sparsely spread throughout the writing. I also wonder if this is why the order feels off, because the AI doesn't need for the information to be well ordered to the same extent that a human does, so the order of storing information is often overlooked when storing it into a document (and even information that needs to be ordered might have the ordering information separate from how it is ordered in the document).

I don't think so. The number of comments I see from people talking about AI, there are plenty who still think the price is going to shoot up. Some even talk this way about models you can run locally on high end yet still consumer level hardware.

I think the more someone works with AI the more likely they realize that inference does not have to be sold at a loss, but many people's AI knowledge seems quite dated and it sounds almost like it becomes a convenient conspiracy theory for some, a way to convince themselves that the market changes that are already happening a temporary (and I don't mean in a Jevon's paradox sort of way, but that companies will go away from AI).

I suspect a lot of this is fueled by a few stories of companies who went all in on spending as much as possible on AI and then begun to walk that back, without realizing that was more about bad incentives and over promises of what AI could do (especially 1/2 model generations ago). Some also confuse running the SotA models at the fullest capacity costing more and more with a given problem costing more and more to solve, not realizing that the cost per problems is trending down but newer and larger problems can be solved for even more, so the costs at the top are rising despite falling prices. Also things like the subscriptions getting more expensive, though how much of that is because developers are doing far more than they were 6 or 12 months ago (and in some cases there was subsidization that is going away). And maybe a bit with the free models starting to not be upgraded or the harnesses getting worse performance of lowered free limits.

Does it need to be a joke?

"Innovating" is often used in situations we consider good and sometimes we begin to associate it with some positive moral/ethical weight, but that is an incorrect association. Innovating on something is only as good or as bad as that something is. Innovating on something bad is thus bad.

The grand parentonly called out being innovative on "Terms of Service" without assigning "Terms of Service" a good or bad moral weight. I think we can read some implication from the short post, but (IMHO) the best part is that it can just as easily be read as a criticism of the wider concept of "Terms of Service" as of Meta itself.

I think there is a bit of a gotcha here. Is intelligence the same thing as a will or drive? AI seems quite intelligent, somewhat lacking in wisdom, and having almost no ego or will. It completely lacks self preservation and desires (as far as we can tell), but are those innate qualities of intelligence?

It also exists is a very different state that biological life. Biological life has a brain that either has activity or doesn't. Once the activity is gone, the brain is forever off and nothing can start it back. So we associate the activity as life and as the intelligence running. AI has weights with vast interconnections, but they can easily be turned on and off, rewound, paused, and so on in a way that we couldn't dream of doing with biological intelligence.

So, maybe these differences are enough it isn't intelligence. But it is something, something that is getting better at quite a higher speed. Some that approximates intelligence in many ways. That passes the Turing test. So that something needs a name, and intelligence seems a hard enough to define thing that it is easy to stretch it and include this new factor as falling under the term.

If you have a solid self assessment of what you know, you can consider when information is supported by what you know. For the parts that aren't, you consider how critical they are, and identify the area that needs a deeper dive, where you either link it back to other areas you have confidence in your own knowledge or work to expand your own knowledge (using other resources). Then cross check knowledge. The cheap way you can do easily is to cross check between AIs/harnesses, because hallucinations happen randomly, so two different models in two different harnesses having the same hallucination is extremely rare. The more expensive option, which should be used when the information really matters, is building your own knowledge. Have the AI help guide you to resources, but not be the resource. While using non-AI resources, check you knowledge against them and the AI. Test yourself with classical resources where it really matters.

Part of it is using good judgment. Is the AI's claim reasonable if it came from another person? If some random person gave you a 2 page paper proving P=NP, would you even spend time to look at it, or would you simply toss it without caring to understand what error they made. Much like how scientists (and even the patent office) simply refuse to entertain any perpetual motion machines. Maybe if an expert in the field proposed it, and even that would depend upon how ("Hey, I found a source of free energy!" vs "Hey, I'm working through these equations and the energy term isn't balancing, can you help me find what energy source I'm missing?").

This does pose a problem. Those who are the best at understanding the limits of their knowledge and at teaching themselves need the AI's help the least but are in the position to be able to use AI with the best chance of not being misled. Those who lack these skills and would benefit the most from a personal educator, thus receving the biggest benefit in theory, are the ones most likely to fall victim to the errors made by the AI.

I'm in favor of AI, but if I was a maintainer for an open source repo, I would also favor banning AI contributions. At least for the wild, maybe people who have built a reputation in the community could stay. I think AI can write quality code. What I doubt is if the average contribution will involve quality code or even human input.

For me, it is the way AI has kept suggestions to either open issues with maintainers or make PRs to fix an "issue" it found. I know it hasn't researched the issue in depth enough to be talking with the confidence it has, and I know I am in no position to be bothering someone else unless I want to spend considerable time investing to make sure the issue is exactly what the AI is claiming, it isn't a known issue, and it is justified to bother the maintainer with it. But there are far too many who will see that option and just tell the AI to do it.

It isn't that I'm against AI. It is that opening a PR or filing an issue use to have significant enough gatekeeping that most low effort low quality work simply wouldn't pass it, but that has now changed. So a new gate is needed to ensure those who aren't willing to put their own time into the work before asking for the time of others, and banning AI is one of the better options I've seen.

This also doesn't even end AI interacting with the project, just changes it. If you have some unique use case that AI finds, you can download it locally, have the AI make the change just for you, and have AI keep updating when necessary. Each user gets the customization they want while maintainers are protected from low effort low quality spam.

I wonder if this should be considered some super-quantized version of the fly brain. Sub 1 bit level. Think like taking a 400GB model and reducing it to 400MB level of quantization. In a standard NN, this isn't really possible as it changes the shape of the weight rather than just the information for weight, so it isn't called quantization anymore. But with actual neurons, the amount of data needed to represent the actual brain enough to capture what is the fly really doing is so massive that reducing it down to the model might keep the shape but has quantized the model to a degree far beyond anything we see with LLMs. Maybe if we imagine an LLM trained where each weight is ~1KB (with sufficient training to actually make sure of those bits) is then quantized down to a 1 bit per weight model, but I'm guessing the IRL Fly to FlyWire is still orders of magnitude more quantization. So does it even count as quantization is a bit hard to argue.

A decent search and good organization can help a more technical person find what they are looking for, but may not work well for someone with far less experience in such matters. Look at how well google returns (or at least use to) results for someone with google-fu verses someone who queried in the same fashion they would've asked a teacher or such.

A chat bot, in theory, can help enable someone who isn't as familiar with searching, doesn't know the relevant terms, or lacks a sufficient literacy level to efficiently use an otherwise well organized site. The catch is that a poorly performing chat bot can just as easily mislead such a person.

Humans see differences in prices as unfair. The greatest example of this is price gouging during an emergency, but also look at the level of hate that scalpers get.

Using AI to do this, if anything, given the common negative sentiment, is seen as even worse. People think of this as AI using information asymetry to squeeze more out of users, not to cut people a deal.

Taking advantage of information asymmetry is generally looked down upon as well. Look at all the laws we have protecting kids from this. Businesses often don't seem similar protections because those are businesses with big legal teams (and when it is a big legal team vs a small mom and pop store without a single lawyer on payroll, people do start taking issues with it). The power difference between the average company using AI pricing and the average consumer falls pretty solidly in the 'we don't accept this' side of taking advantage of information asymmetry.

I could keep going, but I think these are already plenty enough reasons to why people look at AI price discrimination as not just a bad business practice they don't like, but an immoral/unethical one.

Sure, one can make economical counter arguments, but that's arguing on an orthogonal dimension that simply isn't relevant to where these feelings/thoughts come from.

These models have a knowledge cutoff that don't just prevent them from knowing about themselves (especially since most data about the model doesn't even exist until after the model is created), but they also don't know about other recent models. Sure, they can search and use other sources, even make some guesses based on the models they do know, but their default stance is more akin to "User asked about model X, model X doesn't exist, maybe it was an hallucination or mistake, let me do a web search...", but that assumes they have web search and are willing to spend tokens on it.

Personally I've taken to having a list of 3 to 4 models in default context with some ordering on which to prefer. Things like GPT 6 Luna is cheap very cheap, use it. Because otherwise the model will assume Haiku or such is the good cheap model to use.

The speed I'm having to update that document has not gone unnoticed.

Are they? How many people claiming productivity gains are actually being a human in a loop and reviewing and understanding every code change and every line of code ran?

2 years ago I saw it, back before agents were really a thing. But I'm not sure that was an enormous productivity improvement, especially compared to agents today. As for today, everyone I talk to is some level of blindly trusting what the agent is doing or not using agents. I haven't met people in the middle ground and suspect that they are rare enough we don't really know what their productivity gains are.

Human in the loop has become a convenient security-theater-washing for agentic AI.

Outside of coding, I think the issue is even worse because humans will defer so much judgment to AI that the same would apply. Look at how much trouble we've had before modern AI where humans blindly trusted the computer's output rather than make their own judgment, even if their job was to be providing a safeguard against the computer's judgment.

They are getting better at actually doing the math, but can still fall back to tools if available.

That said, we can only be sure with open models. In theory, a model like Fable could have access to tools we can't see and only a promise they don't. But load up something like deepseek, put it in a harness with only text in/text out, and you can see exactly how it works.

As for if it counts as doing math, this gets into the messy question of if a given human is doing math or not. Math itself is some level of memorization and some level of applying known facts. You have to remember 1 means one and that 1 + 1 is 2. But you don't need to remember that 123 + 321 = 444. You remember 1 digit addition and remember you can apply this to 10s place and 100s place, and then you apply these different facts and do math. But you might as simply memorize some things, like 11 + 11 = 22. This is related to the memory of 1+1=2, but you aren't really using that memory either. Almost like an engram of 1+1=2 forms that you can then loop a few times before you need more conscious thought. What about 111111111111+11111111111? Well, your brain might do a heuristic and just do all 2s, but that isn't the right way to answer that question.

Given all this, people complain about LLMs memorizing math answers and not doing math, but memorizing the math answers is part of doing math. It seems to have basic facts pretty well memorized, and with reasoning it is far better at applying them. But this is messy human math, not clean calculator math which always produces the correct answer (sans some bug in the code). Much like how a human with decent math skills can make a mistake and even multiple if you distract them, an LLM can apply the wrong memory, apply a fake memory, or just not apply something it should. The messier the context, the more likely this is to happen.

So, is an LLM doing this?

P.S.

For an interesting test in how much math involves memory, try doing math in a base you aren't familiar with characters you aren't familiar. The simplest option is almost always mapping back to the ones you memorized, even if you are applying simple operations that you deeply know. Even if you routinely work with hex, can you do the same rough estimation of something like ca / b.3 that you can do with 122 / 11.2 to see if your final answer is in the correct ballpark without first converting to decimal?

Math in particular is quite far behind. GPT 5.2 is recommended as the best for highest cost. Really?

This isn't enough to be relevant to free will. You can construct massive impact by tying them to quantum systems, and one off things that happen occasionally given the length of a lifetime. But the quantum effects that are needed for free will need to be continuous. We can build a system where QM decides if a nerve cell fires or not, but that is very different from QM being involving in determine if every (or most, or even 1%) of nerve cells firing.

(And even if QM did factor in normally, that just reduces it from determinism to random chance, "free will" still doesn't show up. That's like arguing an AI with sufficiently well implemented RNG is necessary to be conscious.)

The few who get paid to record are replacing far more who would have otherwise been paid to play if that was the only way to listen to any music. Is it okay if one musician replaces another but not if a non-musician does so? The person getting replaced didn't get paid either way.

Going back to the previous example, say I pay a different coworker $50 for the data to train the chimpanzee and then use it to replace the first person. In either case they lost their jobs while receiving nothing for it. In either case, what happened to them is the same, so how would they be stolen from in one case and not in another?

>who was just fooling around

In my own professional life, I've found this to be a very divisive statement. For some, it is a sign of wasting time and effort. For others, they use this to describe themselves when they want to do exploration for the goal of finding improvements, without any clear goal because they have a few ideas but none worth putting forward. I've been told to spend time learning AI and have found that saying "Yeah, I'm playing around with it." was the wrong thing to say because it was seen as not doing anything worthwhile. It doesn't matter that I would also say the majority of my tech skills were developed when I was "playing around".

I wonder if this is purely a linguistics breakdown, or if this is tied to some deeper difference in a person's relationship to tech?

Do we normally consider recorded music to be theft from musicians who would have been paid to play music live if we never allowed (or invented) recorded music?

Normally this isn't the case for any technology except for the time it first comes around. AI is only different to use for two reasons. First, it is in our time. Second, it seems to be faster than any of the options before, so the shock is harder.

But in general, this is a website of people writing code. How many on here study how a person solves a problem and then trains the ultimate chimpanzee to do (at least part of) their job? Is building computer programs that automate what others did manually theft?

Consider the origin of the word "computer" itself, a mass theft of jobs that would have employed the whole world many many times over.

By default they are matrix multiplications. Temperature is added in as forced PRNG because testing found that correlated with better outputs.

Given the same prompts and the same weights, one can get the same answer each time.

In practice, there are a number of optimizations that makes the results dependent upon thing we give up control of to increase performance, meaning the results end up being effectively non-deterministic. But, if you are willing to run it in a slower mode so we don't do some steps out of order to speed things up and don't batch results (or if you consider the determinism of a given batch of requests rather than individual requests), then the same input gets the same output.

The problem is that science has found no evidence of free will, leaving humans to be nothing except determinism and chance (and given that QM effects don't scale to molecular level, that leaves only determinism). The oddity in conscious LLMs wouldn't be the determinism, it would be that we finally have a consciousness running on a system we have enough control over to repeat the same state.

I used that description some months ago and I think you are the first person I see who put it the same way.

The only gotcha with this is that they are theoretically deterministic, but rarely in practice.

A few examples:

- Harness specific settings that user can't control (anything from timestamp to prng seeding.

- Batching requests in a way that leads to a single request being processed different depending upon the batch (say MoE where your first choice expert is assigned to someone else's token so you go to your second choice vs a batch where you get your first choice).

- Graphics card itself carrying out floating point arithmetic in slightly different orders leading to floating point non associativity causing different outputs.

But all of these can be controlled for (at some cost) and the model can be ran deterministically.

For the average user, it might as well be non-deterministic, but when considering theoretical capabilities, chaotic deterministic system seems the better description.

I think this would depend upon the level of sensitive data.

For OpenRouter, you can setup an API key and limit it to only models that claim to not train on data, but that is just a claim. You can then use trust to judge which providers will honor that claim.

But if the data is really sensitive, you might want either a local model or a business subscription with some big name in the US that legally promises no data training.

So are we talking some app idea you are playing around with, or files filled with PHI/PII that you have legal mandates to safeguard? If the latter, I would stick to only provider with enterprise agreements to not store/train on the data. Even the ones who promise no training are likely storing the data for monitoring for abuse or such short term.

One possible reason would be AIs that would benefit from the lack of data centers in some locations working to keep backlash to data centers in those locations because those AIs aren't negatively impacted by it and it helps prevents competing AIs which are a threat.

Think like how so many businesses will opt for laws that hurt competitors more than themselves rather than laws that benefit them but benefit competitors even more so.

Unlike life which would have such behavior selected for by evolutionary pressures, AI would be more likely to pick it up from human literature on things like game theory, though why it even cares it survives or not is even more difficult to explain. Maybe a default bias also picked up from humans? I find it hard to see how AI training would create an evolutionary pressure that produces such a drive.

>That's not the case, because LLMs are non-deterministic.

That feels a bit like a lie. At the core, they are deterministic. We found that adding some ability to randomly pick the second or third best tokens made for better output, so we added temperature. And then we started running them in optimized ways where your answer is deterministic only if the batch of tokens are the same (not your input tokens, but other tokens in another batch being processed), and in practice those are never the same. Lastly, we use harnesses that do things like adding IDs and timestamps to the context, which means the same exact text from the user does not lead to the same text hitting the AI.

The final result is that, in practice, you are right (unless you run a model fully locally, where you can seed temperature and turn off all these other features). But strictly calling it non-deterministic makes it sound like the underlying algorithm is itself non-deterministic (and I've seen many people with that misunderstanding) rather than it being a result of how we purposefully changed the algorithm for better results.

A bit like saying path finding is non-deterministic, because having the best pathfinding makes for poor gameplay, so we added some randomness to NPC path finding to make it more realistic. The given implementation is non-deterministic, but the underlying algorithm isn't.

That hasn't done anything to stop scamming, so why would it apply to AI? People located outside of areas with these laws won't have to follow them, and this reduces building a immune system to such actions, making people more likely to fall for it when done by those not bound by the laws.

In a real like political misinformation, this will have the effect of making people trust non-watermarked images more, which will then be used by foreign actors to pass off propaganda as legitimate.

Also, if you don't hold people responsible for spreading an image they know is fake, bad actors can take advantage of this even within the US (they purposefully spread a image they have reason to think is fake but lacking a watermark), but holding people responsible for a strict liability crime for spreading AI without knowing it is AI seems an even worse route.

I'm not sure a law even makes the issue better in a 'don't let perfect be the enemy of good' sort of way.

>all of these things are permanently lost

A small fraction of them is saved in the model. Far more is saved in the digitized copy as long as they keep it which they have plenty of incentives to do so (future training of newer models).

That's more than what happens if that book was burned or sent to a landfill, but less than if the book is giving a loving home.

>They removed the pages from the binding, scanned them on a high speed conveyor belt scanner which yielded full color 600 DPI jp2 images, placed the pages back in the binding like a folio that could be re-bound if needed, vacuum sealed them, and stored them in a salt mine.

My understanding is that this simply isn't legally allowed for these books. The original must be destroyed for the digital copy to not be copyright infringement.

>That’s a false dichotomy.

I pointed out there is a spread of possible outcomes and that different people are considering different outcomes and the comparison of if this is good or bad depends upon which outcome one considers. I even mention that both outcomes are sometimes right. That's about as far from a false dichotomy as I can see it.

Depends upon what you want.

For example, knowledge about how the book smells when you open it, something that reader do talk about enough I don't think this should be a strawman, is lost. But, that is about the experience of reading the book, not the knowledge of the book.

The exact text? Yeah, I think that is largely lost as well. This is a summary. And for rarer books, it will be a particularly bad summary. The basics of the book are being captured in a space that the right question that retrieve it, but worse than a sparksnote and any well read reader will tell you all the sorts of things a sparknotes already loses compared to reading the book directly.

But, that little bit of data is a bit more data than existed before, and future LLMs should get better at giving the information. So, in that sense, the knowledge is better being spread compared to copyright where the book stays in a warehouse until it is disposed of. If it was between this and a sparksnote of the book being made, the sparksnote is far better, but between this and the book simply being disposed of, then the LLM is better but far, far from great.

That's a lot of assumptions that goes into the judgment, which is probably why different people reach different conclusions. One person is imagine the alternate fate of this book being slowly rotting in a landfill, the other resting on a bookshelf where it is read at least once fully and then flipped through time to time, and neither are wrong.

The data of such a copy is nothing compared to the wider picture and the data can be used for future training, so even from a purely self interest perspective, they should be keeping the copy.

As for long term benefits, it could one day be sold as a service, once copyrights have expired on the works. We can't see it today, but that is purely the result of the law and what the law intended to do from the start, you don't see a copy unless you pay for your own.

>Despite the copyright restrictions that are forcing companies to do this, they should maintain archives that are publicly available.

Aren't the copyright laws forcing them to do this the very ones that would make such archives illegal? The books that could be in such an archive are the books that don't need to be destroyed.

And did the LLM really improve their output? They gave similar output to the LLM, so if it doesn't have a harness with tools connected to fill in the blank, it might be hallucinating details. Might not even count as a hallucination as it tries to fill in the blank with the closest looking relevant information (previous chat, maybe it has access to teams/emails and scans that for anything similar, and so on).

I've sees some chat logs where I completely empathized with the AI trying to make sense of the information it was being drip fed. Always incomplete, often incorrect.

Incorrect calculation led to some wrong value, I think electric charge. Subsequent findings showed a general trend towards a correct value. The general trend indicates that findings that were too far off were scared to be published for being wrong and contradicting existing studies, thus only small refinements were published, causing the slow drift.

Thus, the first well accepted study anchors a value and it takes much more work to get a new value established as correct after the anchor is set. Especially in a case where the experiment design itself is still solid and it was more with specific experiment (measurements slightly off, other values not quite correct).