The idea is for it to automatically detect the "language" of each word based on the context and its own understanding of the world.
E.g., the following sentence:
"We deployed windows server data centre 2022 into our data center, which has no windows for physical security."
Will be corrected by GPT-4 to the following:
"We deployed Windows Server Datacenter 2022 into our data centre, which has no windows for physical security."
Notice that it combined "data" and "centre" into "Datacenter" and it corrected the second "center" into "centre", which is the British/Australian spelling of the word. It also correctly capitalised only the first use of the word "windows", etc...
That requires a level of understanding that GPT 3.5 just barely has, and no ordinary grammar checker tool has.
General purpose models containing significant overlap between Project Gutenberg and Github are unnecessary and don't scale. Moby Dick has little to do with C++ unless you're creating art for novelty's sake. This is entirely speculative, but I'm convinced ChatGPT is faking the appearance of a single oracle while delegating requests to specialized models under the hood. It scales better and makes sense than trying to serve a 1T model to address everybody's banal questions.
Like, at its core, for people who only want to write literature, give them a model with underweighed programming-related corpora. Writers don't need it, will never use it, and that space could be filled with training content relevant to literature. Anything else results in expensive, unscalable solutions or jack-of-all-trades, master-of-none outcomes.
In recent usage, GPT3.5 helped me hack my way through writing Pester tests for Powershell scripts for the first time, and I mean hack-- there were a lot of assumptions it made and things it got wrong. GPT4 did a much better job, but I couldn't help but think 3.5 probably has a ton of other training data in it that detracts from the specialization I needed from it in that context. For coding help, you don't want to ask some random librarian who occasionally recommends resources that don't exist; you ask someone who specializes in coding and trust they have familiarity with that domain.
--
[0] - Or "something very convincingly pretending to think by parroting stuff back", if you're closer to the "stochastic parrot" view.
[1] - Per my hand-wavy hypothesis that the bulk of what we call thinking boils down to proximity search in extremely high-dimensional space.