kflansburg

At no point do we claim that it is a bug with the model. We specifically outline how it was a mis-configuration on Cloudflare's part and a bug in SGLang. We name the model in the title because our customers need to know which of our endpoints was affected. In fact, we say "the GLM-5.3 Workers AI model", not "GLM-5.3".

> Our post-training utilizes label-smoothed cross-entropy for valid schema outputs paired with a Brier loss to refine probability calibration.

Some users on X [1] were discussing the use of DiffusionGemma for Jev-like use cases. Similar to Jev, diffusion models generate output fully in parallel. We achieve similar estimation of relative confidence by looking at token log probabilities. Finally, by disabling reasoning we achieve low output tokens and similar latencies. This demo lets you compare the output to Jev for some canned prompts, or see the DiffusionGemma output for your own prompts.

[1] Original X post with vLLM PR https://x.com/mmastrac/status/2100626193943052784

My point was not about our ranking specifically, but the methodology of taking a point-in-time sample.

I would recommend tracking this data over time. I work on Cloudflare's KV cache for Kimi K2.6, and while there are periods where our cache rate is low, we are frequently in the 80-90% range. OpenRouter shows us at 87.3% at the time of this post. We observe cache rates change quite a bit from hour to hour.

If you aren't already aware, Karpathy has several videos that could get you there in a few hours https://www.youtube.com/@AndrejKarpathy

Cloudflare | Systems or ML Engineer, Workers AI | Austin, TX or London, UK or San Francisco, CA (Hybrid) | Full-Time | https://cloudflare.com

Cloudflare is building across the entire AI stack. Here are some exciting things that we have launched recently:

- Kimi k2.5: https://blog.cloudflare.com/workers-ai-large-models/

- Kimi Performance Improvements: https://blog.cloudflare.com/high-performance-llms/

- Kimi k2.6 Speculative Decoding and Shared KV Cache: https://x.com/kevin_flansburg/status/2050238819065299110

- Unweight Tensor Compression: https://blog.cloudflare.com/unweight-tensor-compression/

- Code Mode MCP Servers: https://blog.cloudflare.com/code-mode/

- Agents SDK: https://blog.cloudflare.com/building-agents-with-openai-and-...

- Agent Memory: https://blog.cloudflare.com/introducing-agent-memory/

- Internal AI Tooling: https://blog.cloudflare.com/internal-ai-engineering-stack/

- Release Cog v0.19: https://github.com/replicate/cog/releases/tag/v0.19.0

- Dynamic Worker Sandboxes: https://blog.cloudflare.com/dynamic-workers/

- Dynamic Workflows: https://blog.cloudflare.com/dynamic-workflows/

We are looking for systems and ML engineers to help build our edge inference platform:

- Senior Systems Engineer, Workers AI - https://job-boards.greenhouse.io/cloudflare/jobs/7764827?gh_...

- Senior / Principal Machine Learning Engineer, Workers AI- https://job-boards.greenhouse.io/cloudflare/jobs/6297179?gh_...

Cloudflare | Systems or ML Engineer, Workers AI | Austin, TX or London, UK or San Francisco, CA (Hybrid) | Full-Time | https://cloudflare.com

Cloudflare is building across the entire AI stack. Here are some exciting things that we have launched recently:

- Release Cog v0.17: https://github.com/replicate/cog/releases/tag/v0.17.0

- Dynamic Worker Sandboxes: https://blog.cloudflare.com/dynamic-workers/

- Kimi K2.5: https://blog.cloudflare.com/workers-ai-large-models/

- Code Mode MCP Servers: https://blog.cloudflare.com/code-mode/

- Agents SDK: https://blog.cloudflare.com/building-agents-with-openai-and-...

We are looking for systems and ML engineers to help build our edge inference platform:

- Senior Systems Engineer, Workers AI - https://job-boards.greenhouse.io/cloudflare/jobs/7764827?gh_...

- Senior / Principal Machine Learning Engineer, Workers AI- https://job-boards.greenhouse.io/cloudflare/jobs/6297179?gh_...

I see now that this incident happened in September 2024 as well.

Yes, I've already performed the upgrade for my projects, but since they hit this bug, I'm guessing they haven't.

> an if let expression over an RWLock assumed (reasonably, but incorrectly) in its else branch that the lock had been released. Instant and virulently contagious deadlock.

I believe this behavior is changing in the 2024 edition: https://doc.rust-lang.org/edition-guide/rust-2024/temporary-...

I'd love to see this comparison between Garmin and Oura. I ditched Whoop because it was too easy to get 100 sleep scores, but my Garmin watch is much harder to please. I do think it offers better signal, but I've never scored over a 90, so maybe it is too critical?

All of the JS features are available in Rust, but some don’t have a first-class SDK API yet and you must use wasm-bindgen.

I was responding to your point about isolates and cold starts. Snapshots are unique to Python, but V8 does not seem relevant here, all this is doing is initializing the linear buffer that backs Wasm memory for a particular instance. We have a lot of ideas here, some of which are mentioned in the blog post.

We discussed a separate configuration field for Python version. It’s not technically challenging, this was a design choice we made to simplify configuration for users and encourage more efficiencies in terms of shared dependencies.

Your concerns about V8 would impact JavaScript Workers as well and do not match what we see in production. It is also definitely possible to invoke C++ host functions directly from Wasm with V8.

I believe that your summary misunderstands how we will handle versioning. The pyodide /package versions will be controlled by the compatibility date, and we will be able to support multiple in production at once. For packages like langchain (or numpy as you mentioned) the plan is to update quite frequently.

Could you expand on why you believe V8 will be a limiting factor? It is quite a powerful Wasm runtime, and most of the optimizations we have planned don’t really depend on the underlying engine.

Run:AI https://news.ycombinator.com/item?id=39738342

You can find an IV that makes sense for a single option with invalid other parameters, but things will break down when you go to price other expirations / strikes.

When trading, you don't want to wait to see an "updated" IV, you would want to respond directly to changes in important and well understood parameters like underlying price.

It's not really implied volatility, it's the actual volatility between now and expiration. Everyone can only estimate at what that will be.

IV is essentially using prevailing prices to understand what everyone else has estimated that forward volatility to be.

Beyond that, you will also find that IV differs across strikes [1]. Still, being able to fit a vol smile from incomplete market data (and some other adjustments if you are very sophisticated) and then price an arbitrary option is pretty useful.

[1] https://en.wikipedia.org/wiki/Volatility_smile

Cloudflare | Multiple Roles | Full-Time | Austin, Lisbon, London, or REMOTE | cloudflare.com

Cloudflare Workers is hiring for multiple roles to help build the leading serverless platform, powered by Cloudflare’s edge network. Our platform is growing rapidly, with industry leading cold-start times and advanced features like Hyperdrive and Workers for Platforms. Cloudflare builds a large portion of our software with open source, such as workerd (runtime), Wrangler, workers-rs, Pingora, and Foundations.

We are looking for experienced C++ developers to work on the Workers Runtime or Durable Objects:

Workers Runtime - https://boards.greenhouse.io/cloudflare/jobs/5446161?gh_jid=...

Workers Durable Objects - https://boards.greenhouse.io/cloudflare/jobs/5390243?gh_jid=...

There are also a few other roles open related to Workers:

Workers Core API - https://boards.greenhouse.io/cloudflare/jobs/5718743?gh_jid=...

I believe that what you are looking for is Hyperdrive. Global connection pooling and query caching.

https://developers.cloudflare.com/hyperdrive/configuration/h...

Nice! I always end up implementing some portion of this on every major project I do. After using something like Rust's Serde crate, you really miss this functionality in Python.

Location: Chicago Remote: Yes Willing to relocate: No Technologies: AWS, Google Cloud, Python, Kubernetes, Spark, Kafka, Docker, Terraform, Rust Resume: http://bit.ly/2uEtV6B Email: kevin.flansburg [at] gmail [dot] com

SEEKING WORK | Remote | Chicago, United States

Experienced developer with a focus on real time data pipelines and dev ops.

- Extensive work with AWS and Google Cloud - Python data science stack (numpy, pandas, scipy, scikit-learn, etc.) - Kafka and Spark (Scala) - Kubernetes and Docker Swarm - Terraform - Rust

kevin.flansburg [at] gmail [dot] com