cpnwaugha

Formatting the code for easy copy-paste

client = OpenAI( base_url="https://gateway.vlm.run/v1/openai", api_key="<VLMRUN_API_KEY>" )

response = client.chat.completions.create( model="rednote-hilab/dots.mocr", messages=[{ "role": "user", "content": [{ "type": "document_url", "document_url": {"url": "https://.../invoice.pdf"}, }], }], extra_body={"document_dpi": 72} )

via the CLI:

pip install vlmrun

vlmrun gw models

vlmrun config set --api-key 'vlmrun' # anon-user, rate-limited

vlmrun gw chat <doc>.pdf -m zai-org/glm-ocr

vlmrun gw chat <doc>.pdf -m zai-org/glm-ocr --json-mode

vlmrun gw chat <doc>.pdf -m deepseek-ai/deepseek-ocr-2

vlmrun gw chat <doc>.pdf -m rednote-hilab/dots.mocr

vlmrun gw chat <doc>.pdf -m paddleocr/pp-ocrv6

Yeah, GPT 5.6 Sol is very good. Generally, Gemini models remain SOTA for VLM tasks with 3.7-flash at the top. ---

That said, considering variables like cost (say, over 100k PDF pages) and accuracy requirements (e.g., construction documents with dense images & tables), Gemini and other SOTA VLMs are expensive and inaccurate, and therefore unsuitable. This is where niche, open-weight, and task-specific OCR/VL models come in.

With a one-line change, you can switch between DeepSeek OCR 2, GLM-OCR, dots.mocr, Paddle OCR VL, PP-OCRv6, etc., and process 100K+ pages for under $60 on VLM Run Gateway

This is why we built VLM Run Gateway, one OpenAI-compatible endpoint for open-weight OCR and VLM models.

Try it out quickly via OpenAI SDK:

``` client = OpenAI( base_url="https://gateway.vlm.run/v1/openai", api_key="<VLMRUN_API_KEY>", )

response = client.chat.completions.create( model="rednote-hilab/dots.mocr", messages=[{ "role": "user", "content": [{ "type": "document_url", "document_url": {"url": "https://.../invoice.pdf"}, }], }], extra_body={"document_dpi": 72}, ) ```

or via our CLI:

``` pip install vlmrun vlmrun gw models vlmrun config set --api-key 'vlmrun' # anon-user, rate-limited vlmrun gw chat <doc>.pdf -m zai-org/glm-ocr vlmrun gw chat <doc>.pdf -m zai-org/glm-ocr --json-mode vlmrun gw chat <doc>.pdf -m deepseek-ai/deepseek-ocr-2 vlmrun gw chat <doc>.pdf -m rednote-hilab/dots.mocr vlmrun gw chat <doc>.pdf -m paddleocr/pp-ocrv6 ```

Docs: https://docs.vlm.run/gateway

Catalog: https://docs.vlm.run/gateway/models

MCP: https://docs.vlm.run/gateway/mcp-server

Colab Quickstart: https://colab.research.google.com/drive/1RkuVIyuc5Po-UlcSlFy...

Read the full post here: https://huggingface.co/blog/vlm-run/intro-to-vlmrun-gateway

With a one-line change, you can switch between open-weight OCR VLMs (DeepSeek OCR 2, GLM-OCR, dots.mocr, Paddle OCR VL, PP-OCRv6, etc.) and process 100K+ pages for under $60 on VLM Run Gateway

> one OpenAI-compatible endpoint for open-weight OCR and VLM models

Try it out quickly via OpenAI SDK:

``` client = OpenAI( base_url="https://gateway.vlm.run/v1/openai", api_key="<VLMRUN_API_KEY>", )

response = client.chat.completions.create( model="rednote-hilab/dots.mocr", messages=[{ "role": "user", "content": [{ "type": "document_url", "document_url": {"url": "https://.../invoice.pdf"}, }], }], extra_body={"document_dpi": 72}, ) ```

or via our CLI:

``` pip install vlmrun vlmrun gw models vlmrun config set --api-key 'vlmrun' # anon-user, rate-limited vlmrun gw chat <doc>.pdf -m zai-org/glm-ocr vlmrun gw chat <doc>.pdf -m zai-org/glm-ocr --json-mode vlmrun gw chat <doc>.pdf -m deepseek-ai/deepseek-ocr-2 vlmrun gw chat <doc>.pdf -m rednote-hilab/dots.mocr vlmrun gw chat <doc>.pdf -m paddleocr/pp-ocrv6 ```

Docs: https://docs.vlm.run/gateway

Catalog: https://docs.vlm.run/gateway/models

MCP: https://docs.vlm.run/gateway/mcp-server

Colab Quickstart: https://colab.research.google.com/drive/1RkuVIyuc5Po-UlcSlFy...

Read the full post here: https://huggingface.co/blog/vlm-run/intro-to-vlmrun-gateway

Here's a Colab link to explore the gateway: https://colab.research.google.com/drive/1RkuVIyuc5Po-UlcSlFy...

mm can spot charts on a screen. did you try it?

Awesome!

Sirus (Qax) is 90% built with LLM coding agents (mix).

I should say that I'm deeply involved in the process - architecting and reviewing (and sometimes revising) to ensure the code is clean and aligned with my preferences.

one approach to ensuring purity of work is staging good changes before asking the agent to make new ones. that way, stashing/deleting poor quality work isn't too expensive and doesn't cost you the current progress.

- https://sirus.veedo.ai

The comments in this post strongly validate the need for reliable video processing and understanding with VLMs.

While you can use Gemini or other local VLMs, the real challenge is token efficiency, accuracy, and coverage. For example, how do you make a VLM “watch” a 2-hour or 4GB video without losing context or meaning?

Video transcript alone can be sufficient for basic workflows needing no visual context. But when deep contextual understanding is required, e.g., self-driving, security analysis, warehouse tracking, etc., you’ll need more advanced methods like keyframe sampling, clipping, chunking, and shots+transcript.

You can explore the different encoding strategies we designed for efficient video processing and understanding here: https://vlm-run.github.io/mm/encoders/#video.

FYI, the repo is now public, and contributions are welcome.

Two approaches for passing binary files, say a 2-hour lecture video, to a VLM: 1. Send it whole: accurate, but slow to encode and process. 2. Keyframe extraction: fast, but could miss what matters.

There's no right or wrong approach here. It all depends on your use case and acceptable trade-offs.

Find useful strategies for encoding binary files in the post below. Enjoy!

We're helping litigation attorneys and insurance claims adjusters turn video into insights in seconds

AI-generated, with summaries and key moments

Tired of sending 40-minute videos that no one watches to the end?

We found that converting long videos into short-form slide (and PDF) content dramatically improves engagement and retention.

With Video Decks, you can:

- Present key moments that matter to your business or lecture

- Auto-distill transcripts and visuals into concise slide summaries

- Navigate with arrows or quick-play previews

- Deliver an experience that enhances engagement and retention

How it works

Upload a video → AI slices it into segments → we craft titles, pull visuals, and generate summaries → you share. Done.

Why we think this is useful

- Faster consumption drives higher engagement

- Perfect for webinars, demos, tutorials, and investor updates where insights get buried in lengthy recordings

- Transforms passive viewing into active learning with clear progression and visual anchors

Try it today - https://veedo.ai/decks

À très bientôt, let's make every second of your video count.