joelhuman

Great question!

Our gating model(the heart) in Humiris' system functions as an intelligent gating mechanism that dynamically selects and orchestrates multiple large language models (LLMs) based on predefined parameters such as cost, performance, privacy, and speed. While the exact implementation specifics can vary, here’s how it might work under the hood:

The gating model evaluates each query and the available models' characteristics to determine the optimal routing. This involves dynamically assigning weights to multiple parameters and scoring models based on the query's requirements. The architecture combines elements of both machine learning and decision-making algorithms rather than relying solely on traditional decision trees or reinforcement learning.

Neural Networks with Softmax Activation:

A neural network trained to route queries based on encoded query features and user priorities. The softmax function outputs probabilities for each model, and the model with the highest probability is selected (or multiple models in collaborative tasks).

Reinforcement Learning (RL):

In advanced systems, reinforcement learning may be employed. The routing model learns optimal routing strategies by maximizing rewards (e.g., high response quality, low latency, reduced cost). RL can also adapt to new models or parameters over time, improving efficiency through trial and feedback loops.

The routing model in Humiris likely uses a hybrid approach, combining machine learning (neural networks) for dynamic decision-making with principles of multi-criteria optimization. The advanced mode incorporate reinforcement learning for adaptability in complex and evolving environments

Yes, that's correct! Humiris breaks down tasks into subtasks when appropriate and routes each to the most suitable model. This process ensures that every part of the task is handled by an LLM optimized for factors specific tasks or domain and others parameters like quality, speed, cost, energy efficiency, or privacy. It’s all about using the right tool for the right job to maximize efficiency and performance.

Our technology is typically more cost-effective than deploying single LLMs on custom infrastructure due to several key factors:

Optimized Resource Use: Humiris dynamically routes each task to the most suitable model, ensuring you only use the necessary resources and avoid paying for unused capacity.

Multi-Model Efficiency: By leveraging multiple specialized LLMs, Humiris handles diverse tasks more efficiently than a one-size-fits-all model, improving performance without the need for overprovisioning.

Lower Operational Costs: As a managed platform, Humiris reduces the expenses and complexities associated with maintaining your own infrastructure, including updates, scaling, and monitoring.

Scalability: Humiris seamlessly scales with your business needs, allowing you to handle increased workloads without the significant costs tied to scaling single-model setups.

Long-Term Savings: Intelligent model selection and resource optimization lead to predictable and reduced costs over time compared to the high initial and ongoing expenses of single LLM deployments.

Yes, Humiris is exploring built-in tools for model drift detection as part of future updates. Currently, we recommend integrating tools like Alibi Detect for monitoring drift in your workflows. Stay tuned for updates as we enhance our platform's capabilities!