Imagine being able to ask your workplace server if it has noticed any unusual traffic, or to write a report on sales with nice graphs. It would be so useful.
But a model that can run on a private cluster is certainly something that there’s going to be demand for. And once that exists there’s no reason it couldn’t be run on site.
You can see why OpenAI doesn’t want to do it though. SaaS is more lucrative.
Except they already do offer private cluster solutions, you just need usage in the hundreds of millions of tokens per day before they want to talk to you (as in they might before that, but that’s the bar they say on the contact us page).
I’m not sure what you mean by this, but it’s incorrect. Sensitive USG information is not processed on Amazon’s commercial offering.
> The Amazon-built cloud will operate behind the IC’s firewall, or more simply: It’s a public cloud built on private premises. [1]
I think this is what you’re referring to.
1 - https://www.theatlantic.com/technology/archive/2014/07/the-d...
So not the greatest analogy. But still I think most doctors, lawyers etc should be okay with their own cluster running in the cloud.
HIPAA data can definitely be stored in the cloud given the right setup. I’ve worked for companies that have done so (the audit is a bit of a pain.)
Absolutely. Virtually every instance of Epic EHR is hosted, for example.
I imagine lawyers knowing about where document data is stored as a bit like software developers being sufficiently aware of licensing. There's plenty who are paying attention, but there's also plenty who are simply unaware.
Microsoft, OTOH, does now offer a HIPAA BAA for its Azure OpenAI service, which includes ChatGPT (which means either they have a bespoke BAA with OpenAI that OpenAI doesn’t publicly offer, or they just are hosting their own ChatGPT instance, a privilege granted based on them being OpenAI’s main sponsor.)
If this proves actually useful I guess such agreements could be arranged quite quickly.
most of the AI offerings on the table right now aren't too dissimilar from that idea in principle.
Google has a contract with the biggest hospital operator in the USA.
Tx also to some certificate they aquires
Of course, since the model is so large and general purpose already, I can’t assume the same fine-tuning techniques are used as for vastly smaller models, so maybe layers aren’t frozen at all.
It is possible to architect things to be fully deterministic with an explicit seed for the pseudorandom aspects (which is mostly how Stable Diffusion works), but I haven't yet seen a Chatbot UI implementation that works that way.
[0] Except on a longer timeframe where the request may be incorporated into future training data.
Also to give it a more natural feel.
Can't find we're I read about it
matrix gets decoded into text on the client side in Javascript, so we receive send and receive from chatGPT only vector of floats (obfuscation?)
I’m probably oversimplifying but it feels doable.
we won’t have that until we come up with a better way to fund these things. “””Open””” AI was founded on that idea, had the most likely chance of anyone in reaching it: even going into things with that intent they failed and switched to lock down the distribution of their models, somehow managed to be bought by MS despite the original non-profit-like structure. you just won’t see what you’re asking for for however long this field is dominated by the profit motive.
https://arstechnica.com/information-technology/2023/03/you-c...
I know nothing about AI, but when DALLE was released, I was under the impression that the leap of tech here is so crazy that no one is going to beat OpenAI at it. We have a bunch now: Stable Diffusion, MidJourney, lots of parallel projects that are similar.
Is it because OpenAI was sharing their secret sauce? Or is it that the sauce isn’t that special?
If it wasn't for patents you'd never get a moat from technology. Google, Facebook, Apple and all have a moat because of two sided markets: advertisers go where the audience is, app makers go where the users are.
(There's another kind of "tech" company that is wrongly lumped in with the others, this is an overcapitalized company that looks like it has a moat because it is overcapitalized and able to lose money to win market share. This includes Amazon, Uber and Netflix.)
About RISC-V: What does you think is different about RISC-V vs ARM? I can only think that ARM has been used in the wild for longer, so there is a meaningful feedback loop. Designers can incorporate this feedback into future designs. Don't give up hope on RISC-V too soon! It might have a place in IoT which needs more diverse compute.
academic performance is a bad predictor for real world performance
Compare this to the AI ecosystem and you get a huge difference. The architecture of these AI systems is pretty well-known despite not being "open," and there is a tremendous amount of competition.
How could I verify this information?
For a concrete example, the bitmanip extensions (which provide significant increases in MIPS/MHz) were used by SiFive in commercial cores before ratification and finalization. No other company could do that because SiFive employees could just change the spec if they did. They're doing the same thing with vector/SIMD instructions now to support their machine learning ambitions.
Most modern tech companies are software companies. To them, the means of production are a commodity server in a rack. It might be an expensive server, but that's actually dependent on scale. It might even be a personal computer on a desk, or a smartphone in a pocket. Further, while creating software is highly technical, duplicating it is probably the most trivial computing operation that exists. Not that distribution is trivial (although it certainly can be) just that if you have one copy of software or data, you have enough software or data for 8 billion people.
Upthread used the term "tech" when the thread is very clearly talking about AI. AI is software, but because they used the term "tech" you cherry-picked non-software tech as a counter example. It doesn't fit because the type of tech that GPT-4 represents doesn't have the manufacturing cost like a chip fab does. It's totally different in kind regardless of the fact that they're both termed "tech".
Chip fabs are literally one of the most expensive facilities ever created. Saying that because they don't need a special moat so therefore nothing in tech ever needs a special moat is so willfully blind that it borders on disingenuity.
The first use of "moat" upthread:
> Curious why even companies at the very edge of innovation are unable to build moats?
Google's Transformer patent isn't relevant to GPT at all. https://patents.google.com/patent/US10452978B2/en
They patented the original Transformer encoder-decoder architecture. But most modern models are built either only out of encoders (the BERT family) or only out of decoders (the GPT family).
Even if they wanted to enforce their patent, they couldn't. It's a classic problem with patenting things that every lawyer warns you about "what if someone could make a change to circumvent your patent".
Now an encoder+decoder is very similar to a decoder-only transformer, but it's certainly an inventive step to make that modification and I'm pretty sure the patent doesn't contain it. It does describe all the other pieces of a decoder/encoder-only transformer though, despite not being covered by any of the claims, and I have no idea what a court would think about that since IANAL.
Once you know that OpenAI gets a certain set of results with roughly technology X, it's much easier to recreate that work than to do it in the first place.
This is true of most technology. Inventing the telephone is something, but if you told a competent engineer the basic idea, they'd be able to do it 50 years earlier no problem.
Same with flight. There are some really tricky problems with counter-intuitive answers (like how stalls work and how turning should work; which still mess up new pilots today). The space of possible answers is huge, and even the questions themselves are very unclear. It took the Wright brothers years of experiments to understand that they were stalling their wing. But once you have the basic questions and their rough answers, any amateur can build a plane today in their shed.
Most likely this.
Right now the magical demo is being paraded around, exploiting the same "worse is better" that toppled previous ivory towers of computing. It's helpful while the real product development happens elsewhere, since it keeps investors hyped about something.
The new verticals seem smaller than all of AI/ML. One company dominating ML is about as likely as a single source owning the living room or the smartphones or the web. That's a platitude for companies to woo their shareholders and for regulators to point at while doing their job. ML dominating the living room or smartphones or the web or education or professional work is equally unrealistic.
But the counter for the high moat would be the atomic bomb -- the soviets were able to build it for a fraction of what it cost the US because the hard parts were leaked to them.
GPT-3 afik is an easier picking because they used a bigger model than necessary, but afterwards there appeared guidelines about model size vs. training data, so GPT-4 probably won't be as easily trimmed down.
The sauce is special, but the recipe is already known. Most of the stuff things like LLMs are based on comes from published research, so in principle coming up with the architecture that can do something very close, is doable to everyone with the skills to understand the research material.
The problems start with a) taking the architecture to a finished and fine tuned model and b) running that model. Because now we are talking about non-trivial amounts of compute, storage and bandwidth, so quite simple resources suddenly become a very real problem.
Isn't this already happening with LLaMA and Dalai etc.? Already now you can run Whisper yourself. And you can run a model almost as powerful as gpt-3.5-turbo. So I can't see why it's out of bounds that we'll be able to host a model as powerful as gpt4.0 on our own (highly specced) Mac Studio M3s, or whatever it may be.
And, presumably you wouldn’t have the model generate the graph directly, but instead have it generate code which generates the graph.
I’m not sure what they had in mind for the “unusual traffic” bit.
It's already been done and discussed:
So the makers proudly say
Will optimize its program
In an almost human way.
And truly, the resemblance
Is uncomfortably strong:
It isn't merely thinking,
It is even thinking wrong.
Piet Hein wrote that in reference to the first operator-free elevators, some 70+ years ago.
What you call hallucination, I call misremembering. Humans do it too. The LLM failure modes are very similar to human failure modes, including making up stuff, being tricked to do something they shouldn't, and even getting mad at their interlocutors. Indeed, they're not merely thinking, they're even thinking wrong.
Given that GPT-4 is a simply large collection of numbers that combine with their inputs via arithmetic manipulation, resulting in a sequence of numbers, I find it hard to understand how they're "thinking".
What exists are voltage levels that cause different stuff to happen. And we can't say much more about what humans do when humans think. You can surely assign abstractions to that too. Interpret neural spiking patters as exotic biological ways to approximate numbers, or whatever.
As it happens I do think our difference from computers matter. But it's not due to our implementation details.
Are you sure? Our senses have gaps that are being constantly filled all day long, it just gets more noticeable when our brain is exhausted and makes errors.
For example, when sleep deprived, people will see things that aren't there but in my own experience they are highly more likely to be things that could be there and make sense in context. I was walking around tired last night and saw a cockroach because I was thinking about cockroaches having killed one earlier but on closer inspection it was a shadow. This has happened for other things in the past like jackets on a chair, people when driving, etc. It seems to me at least when my brain is struggling it fills in the gaps with things it has seen before in similar situations. That sounds a lot like probabilistic extrapolation from possibilities. I could see this capacity extend to novel thought with a few tweaks.
> Given that GPT-4 is a simply large collection of numbers that combine with their inputs via arithmetic manipulation, resulting in a sequence of numbers, I find it hard to understand how they're "thinking".
Reduce a human to atoms and identify which ones cause consciousness or thought. That is the fundamental paradox here and why people think it's a consequence of the system, which could also apply to technology.
So, LLaMA? It's no chat gpt but it can potentially serve this purpose
Tada! Literally runs on a raspberry pi (very slowly).
GPT models are incredible but the future is somehow even more amazing than that.
I suspect this will be the approach for legal / medical uses (if regulation allows).
See
- https://www.zama.ai/post/encrypted-image-filtering-using-hom...
- https://news.ycombinator.com/item?id=31933995
- https://news.ycombinator.com/item?id=34080882
I feel like 30 years is squarely within our generation
Yeah, I think the issues presented will relate to uniquely tricky errors, or entirely new categories of errors we have to understand the nature of. In addition to subtle and rare, I think elaborately hallucinated and justified errors, errors that become justified and reasoned for with increasing sophistication, is going to be a category of error we'll have to deal with. Consider the case of making fake but very plausible sounding citations to research papers, and how much further AI might be able to go to backfill in it's evidence and reasons.
Anyway, I just mean to suggest we will have to contend with a few new genres of errors
OTOH, the more patient info you are putting in, the less likely it is actually legally deidentified.
OpenAI just has to promise they won't store the data. Perhaps they'll add a privacy premium for the extra effort, but so what?
I worked as a lawyer for six years; there are extremely strict ethical and legal restrictions around sharing privileged information.
But Microsoft already got all the needed paperwork done to do these things, it isn't like this is some unsolved problem.
https://aws.amazon.com/compliance/hipaa-eligible-services-re...
As you can see, there is much more than zero of them.
Nevertheless, the development of AI jurisprudence will be interesting.
Most company's confidential information is already in their Gmail, or Office 365.
Offering sealed server boxes with GPT software, to run on premises heavily firewalled or air-gapped could be a viable business model.
OpenAI just simply does not offer the same thing at this time. You’re stuck using Facebook’s model for the moment which is much inferior.
Email is harder, but I do run my own email server. For mostly network related reasons, it is easier to run it as a cloud VM, but there's nothing about the email protocol itself that needs you to use a centralised service or host it in a particular network location.
https://support.microsoft.com/en-us/office/save-documents-on...
https://azure.microsoft.com/en-us/products/cognitive-service...
(disclaimer: I work for Microsoft but not on the Azure team)
I would never send unencrypted PII to such an API, regardless of their privacy policy.
Because your clients do not allow you to share their data with third parties?