News for hackers. Stuff that matters.

wietsevenema

wietsevenema on Hacker News

Re:Run AI inference apps with self-hosted models on Cloud Run with Nvi... on Wednesday August 21, 2024 @01:47PM (#41312504)
One NVIDIA L4 GPU (24GB vRAM) per Cloud Run instance (many instances per Cloud Run service).

Scale to zero: When there are no incoming requests, Cloud Run stops all remaining instances and you’re not charged.

Fast cold start: When scaling from zero, processes in the container can use the GPU in approximately 5 seconds.

Open large language models up to 13B parameters run great, including: Gemma 2 (9B), Llama 3.1 (8B), Mistral (7B), Qwen2 (7B).

You can get Gemma 2 (2B, Q4_0) to return tokens after 11 seconds from a cold start (best case).

Re:Tell HN: Hacker News BigQuery dataset is updated daily again on Monday June 17, 2024 @06:53AM (#40704279)
Closing in on 41 million items
Re:Tell HN: Hacker News BigQuery dataset is updated daily again on Wednesday June 12, 2024 @02:17PM (#40661185)
Yes, that’s how it works. Same data as in the API.

Tell HN: Hacker News BigQuery dataset is updated daily again

Posted by wietsevenema on Tuesday June 11, 2024 @06:26AM
from the self-post dept.

As you might know, there's a Hacker News BigQuery dataset [1].I recently noticed that the export was broken (since Nov 2022) and I worked with the team to help fix it. It'll now again run daily.SELECT timestamp FROM `bigquery-public-data.hacker_news.full` ORDER BY id DESC LIMIT 1 # 2024-06-09 09:12:55 UTC[1]: https://console.cloud.google.com/marketplace/product/y-combinator/hacker-news

( Read More... | 11 comments | 2 points | on HN )
Re:Building a high-scale chat server on Cloud Run on Wednesday March 10, 2021 @02:48PM (#26415322)
You can use an HTTP push subscription to an IAM protected Cloud Run service.