cherryswitch
Re:M2 Ultra can run 128 streams of Llama 2 7B in parallel
on Wednesday October 11, 2023 @05:37PM (#37850568)
Sometimes the flexibility and time savings is worth the added upfront cost. Similar to how companies like to hire consultants or lease office space. Being able to walk away is better for short term, because companies value profits in the short term.