News for hackers. Stuff that matters.
Re:Localaik – Run OpenAI and Gemini APIs Locally for CI and ... by jeremyfelps on Tuesday May 19, 2026 @12:44PM (#48195766)
Missing piece in the agent-testing chain. Testing agentic workflows that span dozens of LLM calls, real API costs blow up fast, and mock libraries always lag actual API behavior by months. A local shim that speaks both Gemini and OpenAI on the same port is the right shape for CI.

Curious about the function-calling determinism. Even at temp 0, small-model JSON schema adherence drifts, especially on rare function shapes. Are you stabilizing it somehow for CI assertions, or just 'rerun on flake'?

The PDF-to-images path is interesting. Multimodal agent tests are where I keep hitting walls. Out of scope for v1, or queued?

GHA service container pattern is the right call. Stealing this idea.

There are no comments on this comment yet.