News for hackers. Stuff that matters.

jumpCastle

Re:OpenAI o3 and o4-mini on Wednesday April 16, 2025 @04:06PM (#43709763)
It was a good benchmark until it entered the training set.
Re:OpenAI o3 and o4-mini on Wednesday April 16, 2025 @04:02PM (#43709734)
But open source like aider
Re:The Llama 4 herd on Saturday April 05, 2025 @11:22PM (#43598729)
Deepseek v3 was FP8
Re:The Llama 4 herd on Saturday April 05, 2025 @11:19PM (#43598708)
It's quite similar to muP

https://github.com/microsoft/mup

Stock prices cannot go up without if the planet is destroyed
Attention was invented because Bengio lab had to be disciplined about a black box (google had more compute)
Aren't services like runpod solve half of these concerns?
Are the updates really slow?
Re:Ilya Sutskever to leave OpenAI on Tuesday May 14, 2024 @09:30PM (#40362132)
Spend more time with my side projects.
Re:2023 ACM Turing Prize awarded to Avi Wigderson on Wednesday April 10, 2024 @04:51PM (#39995573)
You can try his book. https://www.math.ias.edu/avi/book
Re:GPT-4 Turbo with Vision is a step backwards for coding on Wednesday April 10, 2024 @10:59AM (#39991442)
You can use older models with the api.
Also the parameters are optimized also with loss of future tokens in the sequence.
Re:Learning Theory from First Principles [pdf] on Sunday March 03, 2024 @10:12AM (#39581312)
Work pretty well for all classes of problems?
Re:Hi everyone yes, I left OpenAI yesterday on Tuesday February 13, 2024 @10:06PM (#39365915)
Rsu exist before ipo?
Re:Memory and new controls for ChatGPT on Tuesday February 13, 2024 @06:50PM (#39364495)
Use api, embed history and retrieve.
Use model output as training data. For better performance you can get some top log probs and minimize kl divergence.
Without open weights why would anyone care about them? By the time they could compete with gpt4 there's probably be gpt5 already.
Re:GPT-4 generates simple app from Whiteboard photo on Wednesday September 27, 2023 @05:52PM (#37681858)
The future is the AI also writes the high level descriptions of stuff.
Re:70B Llama 2 at 35tokens/second on 4090 on Wednesday September 13, 2023 @11:17AM (#37497900)
But you can fine tune gpt 3.5 turbo, so your comparison is not clear.
I still use infinity for reddit though.
Gzip every query with all training data can get more expensive.
Re:Scaling Transformers to 1B Tokens on Thursday July 06, 2023 @04:25PM (#36622345)
Silly is a charitable interpretation.
Re:Scaling Transformers to 1B Tokens on Thursday July 06, 2023 @03:06PM (#36621172)
Title with a 10 digits number, meaningless first page figure and no experiments related to the main claim. Did a rogue author posted it without permission again?
Re:Threads has passed 2M sign ups in the first 2 hours on Thursday July 06, 2023 @12:24AM (#36611264)
They no longer require sign-in. Got scared of the reaction.
Is Azure as simple to use as OpenAI? Their documentation are much less simple.
For an answer I would expect it to get the same reward for both question orderings. So naively I would expect it to not be affected by the ordering.
Read codebase and implement feature. Read paper and prove conjecture.
For web access there's nat.dev

Older »