News for hackers. Stuff that matters.
Re:Powering cost-efficient AI inference at scale with Cloud ... by bobbypage on Tuesday November 28, 2023 @09:04PM (#38454530)
Thank you! Regarding performance, TPU v5e which was benchmarked in the MLPerf results and showed impressive performance/dollar, see https://cloud.google.com/blog/products/compute/performance-p... which has more details. Combined with k8s & GKE, TPU v5e workloads can leverage auto-scaling, for example setting up autoscaling based on traffic, so workloads can scale down rapidly when not in use, increasing cost efficiency.
Re:Powering cost-efficient AI inference at scale with Cloud ... by ametrau on Wednesday November 29, 2023 @10:57AM (#38461041)
Can I use LLMs to make more convincing sock puppet questions?