Providers & Models

AI Providers and Models are the foundation of Huf agents. Providers give you access to AI services, and Models are the specific AI “brains” your agents use.

Overview

Providers and models are two distinct layers you configure separately: the provider handles authentication, while the model determines the intelligence your agent will use. Understanding the difference helps you set up credentials once and then experiment with different models without changing your integration.

Your agent needs both: a provider for authentication, and a model for intelligence.

AI Providers

Providers are the first thing you configure because they supply the authenticated connection to an AI service.

What is a Provider?

An AI Provider stores the credentials (API key) needed to access an AI service. Huf uses LiteLLM to provide unified access to 100+ AI providers through a single interface.

Supported Providers

Huf supports all providers via LiteLLM, including:

And 100+ more providers supported via LiteLLM.

Adding a Provider

You add providers in the Frappe Desk so Huf can authenticate with the AI service on your behalf. Navigate to Desk → Huf → AI Provider, click New, and complete the form.

Get API keys from the provider sites below:

Provider Configuration

Providers are global—once added, they can be used by any agent on your site (subject to Frappe permissions). Follow these practices to keep your setup secure and cost-effective:

• Use separate API keys for development and production — isolates traffic, limits blast radius, and makes cost attribution easier.
• Set spending limits on your provider dashboard — prevents runaway usage from impacting your budget.
• Monitor usage through provider dashboards and Huf’s Agent Run logs — lets you spot unusual patterns and optimize model choices.
• For teams, use organization/team API keys rather than personal ones — simplifies rotation and offboarding.

AI Models

Once a provider is configured, you create models to represent the specific LLMs your agents can call.

What is a Model?

An AI Model defines which specific LLM (large language model) your agent will use. Models vary in capability, speed, cost, and specialization.

Model Name Formats

Huf automatically normalizes model names to LiteLLM format, so you can use the simpler user-friendly names in most cases.

User-Friendly Format (recommended):

gpt-4-turbo
claude-3-opus
gemini-pro

LiteLLM Format (also supported):

openai/gpt-4-turbo
anthropic/claude-3-opus
google/gemini-pro

Use the user-friendly format—Huf adds the provider prefix automatically based on your provider selection.

Adding a Model

Models are created separately from providers so you can reuse a single provider across many models. Navigate to Desk → Huf → AI Model, click New, and complete the form.

Popular Models Comparison

The table below compares commonly used models so you can balance capability, speed, and cost for each agent.

The cost symbols in the table above map to approximate per-token price ranges:

Choosing the Right Model

Match the model to the task. A smaller, faster model keeps costs low for simple work, while larger models justify their price when reasoning quality matters.

How Providers and Models Work Together

Understanding how a model request travels from an agent to the provider API makes it easier to debug issues and tune performance.

The Flow

When an agent runs, Huf resolves the model and provider into a single authenticated request to LiteLLM. The sequence is:

LiteLLM Integration

Huf uses LiteLLM as a unified interface to all providers. This gives you several operational advantages:

• Consistent API — the same calling code works across every supported provider, so switching models rarely requires code changes.
• Automatic retries — transient failures are retried with built-in exponential backoff.
• Cost tracking — LiteLLM counts tokens and calculates cost automatically for each run.
• Fallbacks — you can configure backup models that take over if the primary model fails.
• 100+ providers — nearly every major LLM provider is available through one integration.

Model Settings in Agents

When you assign a model to an agent, you can tune its behavior with generation parameters.

• Temperature (0.0 - 2.0) — controls how random or creative the output is.

  • 0.0 = Deterministic and focused
  • 1.0 = Balanced (default)
  • 2.0 = Creative and varied

• Top P (0.0 - 1.0) — an alternative way to restrict output diversity.

  • 1.0 = Full vocabulary available (default)
  • 0.1 = Very focused sampling

Multiple Providers & Models

You can (and should) add multiple providers and models. A small portfolio gives you flexibility when models are down, over budget, or simply not the right fit for a specific agent.

• Fallback options — if one provider has an outage, you can point agents at another provider without redeploying code.
• Cost optimization — route simple tasks to cheaper models and reserve expensive models for complex work.
• Specialization — pair different models with different agent types based on their strengths.
• Testing — compare model performance across providers using the same prompts.

Example setup:

Monitoring and Costs

Running agents consumes tokens, so monitoring usage and optimizing costs is part of operating Huf at scale.

Tracking Usage

Every agent run records token usage and cost so you can see exactly what each execution consumed. The logged values are:

View in: Desk → Huf → Agent Run

Cost Optimization Tips

Use the strategies below to keep inference costs predictable without sacrificing agent quality:

1. Choose the right model for the task — avoid GPT-4 for simple work and use Claude Haiku for high-volume, simple interactions.
2. Manage conversation history — cap history length to reduce tokens in context, and summarize old conversations instead of sending full history.
3. Optimize tool definitions — only assign necessary tools to agents and keep tool descriptions concise but clear.
4. Monitor and iterate — check Agent Run logs regularly to identify expensive runs and optimize them.
5. Set provider spending limits — configure limits and alerts in your provider dashboard to catch unusual usage early.

Rate Limits

AI providers enforce rate limits on requests per minute and tokens per minute. If you hit a limit, try these approaches:

• Spread out requests — avoid running many agents simultaneously so you stay under per-minute caps.
• Upgrade your provider tier — most providers offer paid tiers with higher limits.
• Use LiteLLM retries — built-in exponential backoff handles transient limit errors automatically.
• Batch operations — group similar requests to make fewer API calls overall.

Troubleshooting

Most provider and model errors fall into a few common categories. Use the table below to diagnose the symptom and apply the fix.

Best Practices

A few consistent practices help you keep provider credentials secure, models well-organized, and inference costs under control.

Security

Treat API keys like production secrets. A few habits keep your provider credentials safe:

• Never share API keys in code, logs, or documentation — exposure can lead to unauthorized usage.
• Use environment-specific keys for development and production — limits the impact of a leak.
• Rotate keys periodically — regular rotation reduces the window of exposure if a key is compromised.
• Set spending alerts on provider dashboards — unexpected cost spikes can be the first sign of misuse.
• Use team/org keys rather than personal keys — makes access control and offboarding easier.

Organization

A well-organized provider and model setup is easier to maintain and audit:

• Use descriptive provider names — clear names prevent confusion when multiple accounts or regions are involved.
• Document model purposes — add notes explaining what each model is intended for.
• Test before production — always test new models with draft agents before releasing them.
• Monitor costs — review Agent Run costs regularly to catch drift in usage or pricing.

Performance

Small tuning decisions can significantly improve latency and cost:

• Match model to task — do not over-engineer; use the smallest model that reliably produces the result.
• Cache common responses — consider caching for FAQ-style agents to avoid repeated inference.
• Optimize prompts — shorter, clearer prompts reduce token counts and improve accuracy.
• Batch when possible — group similar requests to amortize overhead.

What’s Next?

Now that you understand Providers & Models, explore the rest of the Huf concepts and guides:

• Tools — give agents capabilities.
• Agents — configure agents with models.
• Quick Start — create your first agent.
• Advanced Settings — configure LiteLLM options.

Further Reading

For deeper detail on providers and models, see the official documentation below:

• LiteLLM Documentation 
• OpenAI Models 
• Anthropic Models 
• Google Gemini 

Questions? Open an issue or discussion on GitHub .