johnshahbazian

Financial institutions are incredibly risk-averse when it comes to technology, so its no surprise to me. Local AI might soon become an answer for them, though.

I got very good results from JevK5: https://github.com/allebee/jevk5

It's what I use locally. For my workload, it was right up there with Jev.

I think its offering a compromise between accuracy and tokens. You don't have to use as many tokens and you get pretty good accuracy, but not the absolute best. I think, however, in many situations people aren't super concerned about the tokens as they are with the accuracy.

Are the battery compartments water proof? We got hit by a hurricane a few years ago, and a few nearby teslas caught on fire from the floodwaters, torching people's houses.

I'm not affiliated with them at all. yeah I like them, and yeah I like to compare decision models at the moment.

Don't believe me? Google my name.

For my own purposes, I found Mercury Decide to be excellent. Better than Jev at what I threw at it. Inception did a really great job with it.

If you know the exact domain you're working in, then a trained model could potentially do better, but its not a guarantee. For example, I fine-tuned Muse Glimmer on a specific domain and then tested it against Mercury Decide - which I've found does better than Jev - and they were even (and no fine-tuning was done on Mercury). But Mercury Decide was was faster and cheaper. The P95 for latency was about 5.6x lower. Decision models are very, very fast.

People like me who are excited about Jev-style models aren't going to use them to diagnose cancer. We are going to use them to make fairly straightforward decisions very quickly (300 ms or less) and very cheaply ($0.042 per M input tokens - output tokens aren't billed). Even the lower end frontier models will take much longer and be much more expensive, so these decision models are perfect for certain things.