nostrowski
Re:Jamba: Production-grade Mamba-based AI model
on Thursday March 28, 2024 @05:10PM (#39857344)
Two things I'm curious to know:
1. How many tokens can 'traditional' models (e.g. Mistral's 8x7B) fit on a single 80GB GPU? 2. How does quantization affect the single transformer layer in the stack? What are the performance/accuracy trade-offs that happen when so little of the stack depends on this bottleneck?
Re:Mintlify GitHub read/write token leak
on Saturday March 16, 2024 @11:02PM (#39731555)
Unfortunate, but impressed by how the Mintlify team is handling it.
Re:Using GPT-4 Vision with Vimium to browse the web
on Thursday November 09, 2023 @12:49PM (#38208265)
This will be in a future history book under a chapter titled "the beginning of the end"