News for hackers. Stuff that matters.
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by yobid20 on Tuesday January 28, 2025 @07:43AM (#42851675)
I canceled my OpenAI subscription last night, as did many many others. There were some threads in reddit with everyone chiming in they all just canceled too. imo OpenAI is done, and will go through massive cuts and probably acquired by the end of the year for a very tiny fraction of its current value.
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by Voloskaya on Tuesday January 28, 2025 @08:39AM (#42852146)
You want to bet? The panic around deepseek is getting completely disconnected from reality.

Don’t get me wrong what DS did is great, but anyone thinking this reshape the fundamental trend of scaling laws and make compute irrelevant is dead wrong. I’m sure OpenAI doesn’t really enjoy the PR right now, but guess what OpenAI/Google/Meta/Anthropic can do if you give them a recipe for 11x more efficient training ? They can scale it to their 100k GPUs clusters and still blow everything. This will be textbook Jevons paradox.

Compute is still king and OpenAI has worked on their training platform longer than anyone.

Of course as soon as the next best model is released, we can train on its output and catch up at a fraction of the cost, and thus the infinite bunny hopping will continue.

But OpenAI is very much alive.

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by lolinder on Tuesday January 28, 2025 @09:12AM (#42852486)
> The panic around deepseek is getting completely disconnected from reality.

This entire hype cycle has long been completely disconnected from reality. I've watched a lot of hype waves, and I've never seen one that oscillates so wildly.

I think you're right that OpenAI isn't as hurt by DeepSeek as the mass panic would lead one to believe, but it's also true that DeepSeek exposes how blown out of proportion the initial hype waves were and how inflated the valuations are for this tech.

Meta has been demonstrating for a while that models are a commodity, not a product you can build a business on. DeepSeek proves that conclusively. OpenAI isn't finished, but they need to continue down the path they've already started and give up the idea that "getting to AGI" is a business model that doesn't require them to think about product.

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by regularfry on Tuesday January 28, 2025 @09:28AM (#42852668)
In a sense it doesn't, in that if DeepSeek can do this, making OpenAI-type capabilities available for Llama-type infrastructure costs, then if you apply OpenAI scale infrastructure again to a much more efficient training/evaluation system, everything multiplies back up. I think that's where they'll have to head: using their infrastructure moat (such as it is) to apply these efficiency learnings to allow much more capable models at the top end. Yes, they can't sleep-walk into it, but I don't think that was ever the game.
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by throwup238 on Tuesday January 28, 2025 @10:15AM (#42853249)
> The panic around deepseek is getting completely disconnected from reality.

Couldn’t agree more! Nobody here read the manual. The last paragraph of DeepSeek’s R1 paper:

> Software Engineering Tasks: Due to the long evaluation times, which impact the efficiency of the RL process, large-scale RL has not been applied extensively in software engineering tasks. As a result, DeepSeek-R1 has not demonstrated a huge improvement over DeepSeek-V3 on software engineering benchmarks. Future versions will address this by implementing rejection sampling on software engineering data or incorporating asynchronous evaluations during the RL process to improve efficiency.

Just based on my evaluations so far, R1 is not even an improvement on V3 in terms of real world coding problems because it gets stuck in stupid reasoning loops like whether “write C++ code to …” means it can use a C library or has to find a C++ wrapper which doesn’t exist.

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by miroljub on Tuesday January 28, 2025 @11:48AM (#42854521)
> You want to bet?

Why would anyone bet? They can just short the OpenAI / MS stocks, and see in a few months if they were right or not.

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by IAmGraydon on Tuesday January 28, 2025 @12:08PM (#42854823)
OpenAI isn't publicly traded and MSFT's stake is so minor compared to their other business that it will have a negligible impact on their stock price.
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by icedchai on Tuesday January 28, 2025 @12:08PM (#42854815)
1) OpenAI isn't public, so not possible. 2) MS is one of the most well diversified tech companies, so, if anything, this will be a positive.
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by skeaker on Tuesday January 28, 2025 @05:01PM (#42858628)
How is that any different from a bet?
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by coliveira on Tuesday January 28, 2025 @09:55AM (#42853006)
Computing is not king, DeepSeek just demonstrated otherwise. And yes, OpenAI will have to reinvent itself to copy DS, but this means they'll have to throw away a lot of their investment in existing tech. They might recover but it is not a minor hiccup as you suggest.
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by nlh on Tuesday January 28, 2025 @10:42AM (#42853588)
I just don't see how this is true. OpenAI has a massive cash & hardware pile -- they'll adapt and learn from what DeepSeek has done and be in a position to build and train 10x-50x-100x (or however) faster and better. They are getting a wake-up call for sure but I don't think much is going to be thrown away.
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by dimgl on Tuesday January 28, 2025 @03:25PM (#42857502)
Deepseek is not the only reason. I cancelled my OpenAI subscription because I've replaced it wholesale with Anthropic.
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by joseda-hg on Tuesday January 28, 2025 @10:05PM (#42861024)
I replaced that with kagi, unliminted access to multiple models including Claude, O1 and V3/R1 + you also get Kagi, which was already a good deal
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by hnben on Wednesday January 29, 2025 @08:24AM (#42864623)
oh wow. I have been using kagi premium for months, and never noticed, that their AI assistant now has all the good AIs too. I was using kagi exclusively for search, and perplexity for ai stuff. I guess I can cut down on my subscriptions too. Thanks for your hint. (Also I noticed that kagi has a pwa for their ai assistent, which is also cool)
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by csomar on Wednesday January 29, 2025 @12:53AM (#42861982)
OpenAI issue might be that it is extremely inefficient with money (high salaries, high compute costs, high expenses, etc..). This is fine when you have an absolute monopoly as investors will throw money your way (open ai is burning cash) but once an alternative is clear, you can no longer do that.

OpenAI doesn't have an advantage in compute more than Google, Microsoft or someone with a few billions of $$.

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by diedyesterday on Thursday January 30, 2025 @03:02PM (#42881585)
...until they distill that "11x efficient training" again ...
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by kebaman on Tuesday January 28, 2025 @08:39AM (#42852152)
Doesn't Microsoft own 49% of OpenAI? They'll end up with it all as a division of Microsoft.
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by wqaatwt on Tuesday January 28, 2025 @09:08AM (#42852444)
I think they “own” 49% of OpenAI’s net income until a certain very high amount. Not a share of the actual company.
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by nicce on Tuesday January 28, 2025 @03:34PM (#42857591)
They "own" even 75% of profits until Microsoft has recouped its $13 billion investment. 49% comes after that.
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by generalizations on Tuesday January 28, 2025 @08:55AM (#42852301)
In my experience with deepseek and o1, openai's big talk about (and investment into) hallucination avoidance might save their hides here. Deepseek may be smarter, and understand complex problems better, but it also seems to make mistakes more often. (It's as if it's comprehension is better, but it's worse at memorization/recall.)

Need an LLM to one-shot some complex network scripting? as of last night, o1 is still where its at.

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by flir on Tuesday January 28, 2025 @09:26AM (#42852629)
My experience gels with yours. Given the same code sample, DeepSeek has better, more creative suggestions about how to improve it, but it can't implement them without breaking the code. o1, generally, can implement DeepSeek's suggestions successfully. I think chaining them together might have quite interesting results.
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by wordpad25 on Tuesday January 28, 2025 @10:13AM (#42853216)
Is there a tool that can automate chaining like that?
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by throwup238 on Tuesday January 28, 2025 @10:18AM (#42853297)
Aider has an architect mode where it asks one model to plan out the changes and another to actually write the code.
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by manmal on Tuesday January 28, 2025 @05:33PM (#42858989)
I've used it today, with R1 as architect and Sonnet as editor model. So far, this works great. There's no need to use a reasoning model as editor IMO.

Alex (https://alexcodes.app) also does this now btw.

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by HarHarVeryFunny on Tuesday January 28, 2025 @10:30AM (#42853441)
That's ok if all you want to know is which model should I use today, but a test like that is totally dependent on training data, and there is no reason to expect that either DeepSeek-V3 (the base model for R1) or the additional training data for R1 is that same as what OpenAI used for O1 and whatever base model it was built on.

The benchmark comparisons are perhaps, for now, the best way to compare reasoning prowess of R1 vs O1, since it seems pretty certain they both trained for those cases.

I think the real significance of R1 isn't the released model/weights itself, but more the paper detailing (sans training data) how to replicate it, and how effective "distillation" (i.e. generate synthetic reasoning data for SFT) can be to enhance reasoning even without using RL.

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by wqaatwt on Tuesday January 28, 2025 @09:06AM (#42852425)
IMHO o1 it’s still comparable to a lot better for accomplishing actual stuff than DeepSeek. At least for my use cases.

Of course cost is incomparably higher since plus has a very low limit. Which of course is a huge deal.

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by conradfr on Tuesday January 28, 2025 @09:21AM (#42852578)
Why every time there is a new model all the other competitors are declared immediately dead?
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by HarHarVeryFunny on Tuesday January 28, 2025 @10:36AM (#42853518)
The big deal here isn't that R1 makes any other models obsolete in terms of performance, but how cheap it is $2 vs $60 per million output tokens compared to O1 (which it matches in benchmark performance).

O1 vs R1 performance on specific non-benchmark problems is also not that relevant until people have replicated R1 and/or tried fine-tuning it with additional data. What would be interesting to see is whether (given the different usage of RL) there is any difference in how well R1 vs O1 generalize to reasoning capability over domains they were not specifically trained for. I'd expect that neither do that well, but not knowing details of what they were trained on makes it hard to test.

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by osigurdson on Tuesday January 28, 2025 @09:51AM (#42852944)
Because we like drama.
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by yieldcrv on Tuesday January 28, 2025 @12:28PM (#42855148)
I disagree, I don't really need "conversational chat responses", I need multimodal

ChatGPT is the king of the multimodal experience still. Anthropic is a distant second, only because it lets you upload images from the clipboard and responds to them, but it can't do anything else like generate images - sometimes it will do a flowchat which is kind of cool, GPT won't do that - but will it speak to you, have tones, listen to you? no.

And in the open source side, this area has been stagnant for like 18 months. There is no cohesive multimodal experience yet. Just a couple vision models with chat capabilities and pretty pathetic GUIs to support them. You have to still do everything yourself there.

There is a huge utility for me, and many others that dont know it yet, if we could just load a couple models at once that work together seamlessly in a single seamless GUI like how ChatGPT works.

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by nicce on Tuesday January 28, 2025 @03:32PM (#42857574)
1. You can get all the models by buying Kagi subscription (excluding o1). Includes DeepSeek models. You can also feed the assistant with search data that you can filter.

2. If you have GitHub Copilot, you get o1 chat also there.

I haven't seen much value with OpenAI subscription for ages.

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by sureglymop on Tuesday January 28, 2025 @07:46PM (#42860075)
I have Kagi Ultimate and it is nice for this. But a cheaper suggestion would be to use OpenRouter and then use these models via Fireworks or TogetherAI. It also integrates into much more applications. AFAIK Kagi doesn't document a user facing API for the assistant feature.
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by phaedrix on Tuesday January 28, 2025 @10:46PM (#42861290)
Unfortunately those are both 10-15x the cost of deepseek direct.

Deepinfra is pretty cheap though as a deepseek provider.

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by pants2 on Wednesday January 29, 2025 @01:46AM (#42862254)
That's because DeepSeek is subsidizing their API massively to get more training data.
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by sureglymop on Wednesday January 29, 2025 @11:28AM (#42867103)
Sure. I meant moreso that this would be cheaper than Kagi while providing the same selection of models.

As for deepseek, I couldn't even sign up because my email domain is not on their whitelist. To just try it out for now I don't mind the increased cost.