yoeven

DefaultModel is a new inference service that hosts one model: the single best open source model, picked by the community.

Open source models are catching up to their closed source counterparts, but inference for them is scattered across general GPU providers. Those providers constantly swap models, and pricing, reliability and support differ from one to the next. You don't get the consistency of a provider like OpenAI.

Large labs get around this by aggregating their own GPU resources, pointing most of the capacity at their latest model, and migrating users there when a new one ships. We want the same for open source models.

So we're taking a different approach. We host one model, the best one, voted by you, the community. We host it for a quarter, and every quarter you vote to keep it or replace it based on the latest releases.

- All GPU capacity goes into one deployment instead of a catalog

- Serving is tuned for that one model, not averaged across fifty

- Performance stays consistent across every request, with no degradation in quality

- When a better model wins the vote, you migrate by changing one model ID instead of rewriting an integration

thanks!

1. yeah, we've optimized significantly for latency, give the API a try and you'll see it's pretty fast.

2. more than you think, we use a wide range of providers internally depending on the context needed. Exa could be great in some for example people search but is pretty bad at getting the latest content of the page, Parallel is better there. Apify is great for getting google ranked pages. Perplexity is great at financial data which Exa is pretty bad at. Context7 (coming soon) great at code, way better than exa for code. Depending on the use case, different indexes do better. we're also seeing companies with access to prosperity data like Valyu.

Really cool! LLMs tho, it's hard to measure accuracy and attribution in real time like confidence scores and bounding boxes which is great for KYC apps. Try Interfaze for LLM OCR, get's you the metadata and output you want.

Yup does really depend on the use case.

We see two types: workflows & agents.

Workflows are the most common, there's a pipeline like processing loan documents before data gets loaded to the next step or translating user comments before being stored in the database.

Agents are where you have a chat based system or a brain of sorts that calls many tools to achieve a user goal. The model doing this is a lot better at non deterministic task which then delegates to Interfaze for specific deterministic actions like OCR, Web extract then consumes that data. That's the article you referenced :)

Try it using run task mode when comparing to pure STT models: https://interfaze.ai/docs/audio/speech-to-text#long-audio-tr...

Excited to see the results

Thanks for the feedback!

We're working a lot more on speed in the coming few weeks :) More GPUs and more optimizations.

Our has been focus on quality of output first and we'll make optimizations as we grow :)

The lite models are great for simple use cases but won't don well in more complex OCR use cases.

yeah it would treat it like an OCR task and extract it, you could prompt it to format it better with the code alignment.

We serve it though an API. Check out the docs: https://interfaze.ai/docs

It's free to gets started.

It can, you could try prompting the model to use object detection vision and text extraction, we realized when we purely extract text it does amazing at word/sentence level bounds since the text acts as the anchor. However, when you treat it as a object detection problem, it sees that chunk of text as a segment allowing you the extract it as one column bound. Give that a try.

Not directly, LAMs tend to be focused a lot on tool calling or trained for a set of specific action for example in the robotics field. Good tool calling might be a good by product of Interfaze but wasn't specifically trained for that use case.

The focus has been for deterministic outputs that require high accuracy. In situations where there is "one right answer"

It wasn't designed to do well on MMMLU, it's a general model designed for deterministic task like OCR, object detection, STT and more and a by product of that is great language abilities. It still has a transformer backbone giving great language skills while being good at other stuff.

See the full benchmark: https://interfaze.ai/leaderboards

For sure there a tons of OCR bounding models and tons of other models like SAM 3 for segmentation.

Interfaze is a more powerful version of them combined into a single model, you can run multi turn tasks like extract all the text and object from this document then translate or generate a report.

It's like getting the best of both worlds from pure DNN/CNN models like Paddle and the flexibility and nuace of an LLM while outperforming both in accuracy.

Use it run task mode if you're doing a one to one comparison to whisper, it's going to be a lot faster too.

Here's a good example: https://interfaze.ai/docs/audio/speech-to-text#long-audio-tr...

The other way round, task specific DNNs adapted to share the same vector space as omni-transformers with generalized vision, audio encoders.

E.g. For an OCR task, the first pass will be handled by the CNN, converted to shared tokens which the transformer can consume, correct any issues if needed and a decoder that can handle both the DNN and transformer output.

It's a service API but we do have on prem deployment in certain regions for enterprises

Google Cloud Vision AI is a specialized model built on CNNs frameworks which is part of the Interfaze architecture which is an hybrid so you get best of both worlds. Google cloud vision was pretty far behind other specalized models like PaddleOCR etc anyways so if you're looking for a pure CNN, check them out.

You can find the explanation and the comparison in the article, which we benchmarked pure CNN models, pure LLM models and a hybrid architecture like ours.

Yup run task mode runs a much smaller part of the model when can drop quality of scans. The issue with run task we have to figure out is how much of the model is needed just for OCR and how to activate the right parts. A lot more improvements coming here with the same cost reduction.

I'd be happy to test it against your sample and see how we can get good results at a lower per page cost. Feel free to email me [email protected]

It isn't on our roadmap right now since in most cases it should work out of the box and if it doesn't we'll work with you to train that into the model generally.

However, if we see enough people who has something super niche that our model can't handle, we might start considering a fine tuning service

Code extraction maybe, not something we have tested or built for but you could give it a try.

Code manipulation probably not since it's a lot smaller of a model compared to a Claude Opus which is SOTA for code generation/manipulation.

Generally code generation is a non-deterministic task by nature and general LLMs tend to be better at them.

We have a full benchmark breakdown specifically on structured output that you can take a look at https://interfaze.ai/leaderboards/structured-output-benchmar...

This is awesome, I've been following time series forecasting a ton so build a zero-shot time series model that allows you to pass in historical data and provide a 98% accurate forecast compared to custom training your own model with custom dataset. https://jigsawstack.com/ai-prediction

if you find it interesting, you could add it to your articles too as a comparison :)

How do you solve the access control part? Did you build the layer ontop or more like a auth redirect?

I think you can try the model