Re:M2 Ultra can run 128 streams of Llama 2 7B in parallel
by JCharante on Thursday October 12, 2023 @04:41AM
(#37854831)
Really? At my old company each division had its own budget and account and then you’d be an iam member of an account and spin up services under that account, but there’s no central authority to send a request to. There were tools to analyze underused services across all accounts (like EC2 instances constantly under 2% cpu load meaning they were flagged for downsizing if possible).
[ Parent
| on HN ]
There are no comments on this comment yet.