weichiang
Re:Cookbook: Finetuning Llama 2 in your own cloud environment, privately
on Wednesday August 02, 2023 @04:10PM (#36976620)
say using A10G ~$1.2/hr and with full utilization on vllm 112 reqs/min => per req ~$0.00018
versus gpt-3.5 turbo $0.002 per 1k token
Re:Cookbook: Finetuning Llama 2 in your own cloud environment, privately
on Wednesday August 02, 2023 @03:59PM (#36976446)
the cost would be depending on GPU type/serving system/traffic pattern. check out some throughput comparison from vllm's blog post https://vllm.ai/
if you serve 7B on cost-optimized GPUs (A10G/L4) and keep it busy, it can be a lot cheaper than gpt3.5 turbo. tho it's not a fair comparison as 3.5's quality is still far better.
Re:Chatbot Arena: a crowd-sourced LLM leaderboard
on Thursday May 11, 2023 @11:19PM (#35911471)
leaderboard link: https://chat.lmsys.org/?leaderboard
Re:Chatbot Arena: Benchmarking LLMs in the Wild with Elo Ratings
on Wednesday May 03, 2023 @04:20PM (#35807236)
Surprised to learn StableLM is worse than plain LLaMA.
link to their leaderboard: leaderboard.lmsys.org
Re:State-of-the-Art Chatbot, Vicuna-7B, now runs on MacBook with GPU a...
on Thursday April 06, 2023 @05:25PM (#35474709)
I asked GPT4All one of Vicuna's benchmark questions:
"What if the Internet had been invented during the Renaissance period?"
Check out their responses: https://imgur.com/a/mPrdZ1W More questions here: https://vicuna.lmsys.org/eval/
Note: not an apple-to-apple comparison but that's the model checkpoint I found on their git repo.
State-of-the-Art Chatbot, Vicuna-7B, now runs on MacBook with GPU acceleration
State-of-the-art open-source chatbot, Vicuna-13B, just released model weights
Re:State-of-the-art open-source chatbot, Vicuna-13B, just released mod...
on Monday April 03, 2023 @04:02PM (#35430433)
See the original Vicuna post:
https://news.ycombinator.com/item?id=35378683
ShareGPT is all you need: matching Google's Bard quality with $300
Re:Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT qu...
on Thursday March 30, 2023 @05:12PM (#35378883)
just tried it worked okay for me tho.
> Can you name all of the soccer players whose names contain a Z
> Here are all the soccer players whose names contain a "Z" that I could find:
Zlatan Ibrahimovic Zinedine Zidane Zlatko Zahovic Zeljko Kalac Zoran Banovic Zlatko Arsovic Zoran Savic Zoran Jovanovic Zoran Djurdjic Zoran Radosavljevic Zoran Zekic Zoran Mirkovic Zoran Sretenovic Zoran Sosic Zoran Luburic Zoran Vulin Zlatan Ljubijankic Zoran Lekovic Zoran Culibrk Zoran Bojovic Please note that this list may not be exhaustive, and there may be other soccer players with names containing "Z" that are not included.