Use Case Guide

Ship Your Own AI API
in a Day

Use Smartflow to package OpenAI, Anthropic, Google Gemini — or any mix — under your own branded endpoint, your own model names, your own compliance rules, and your own audit trail. Zero cloud lock-in.

Category
AI Packaging & White-Labelling
Introduced
Smartflow 1.7
Published
April 2026
Target Audience
ISVs · MSPs · Enterprise Platform Teams
 Executive Summary

Smartflow 1.7 introduces the AI Packaging Platform — a new operating mode that lets any organisation take one or more LLM providers and re-expose them as a fully governed, branded AI API under their own domain and their own rules.

Where previously Smartflow was purchased as an AI governance layer for an organisation's own AI use, it can now power a product that an ISV, MSP, or enterprise platform team ships to their customers — complete with custom model names, virtual API keys, per-tenant compliance policies, semantic caching, and an immutable audit trail.

< 1 day
From provider key to live branded API
8+
Supported provider types out of the box
100%
On-prem deployable — no cloud dependency
∞
Virtual keys, tenants, and model aliases

What Is AI Packaging?

AI Packaging is the practice of taking one or more foundation model providers — OpenAI, Anthropic, Google, Ollama, or any OpenAI-compatible endpoint — and re-exposing them through a controlled, branded intermediary layer that your customers or internal users call directly.

Your customers never know which underlying provider is serving their request. They see:

  • Your API endpoint: https://ai.acme.com/v1/chat/completions
  • Your model names: acme-standard, acme-premium, acme-code
  • Your API keys: sk-acme-...
  • Your usage dashboard, your rate limits, your pricing

Smartflow sits between your customers and the providers, handling everything in the middle: routing, caching, compliance, identity, metering, and the audit log.

Think of it as Stripe for AI — you're not a bank, but you process payments under your own brand backed by a banking network. You're not OpenAI, but you serve AI under your own brand backed by the models your clients trust.

How It Works

Smartflow's AI Packaging mode uses three existing capabilities together, wired through the new Provider Onboarding Wizard introduced in v1.7:

Your Customer
Client App
Any OpenAI SDK
Python · JS · Go · REST
→
Virtual Key
sk-acme-xxxx
Your branding
→
Your Domain
ai.acme.com
/v1/chat/completions
Smartflow Gateway (Your Infrastructure)
Identity & Auth
Key Vault
Per-tenant limits
→
Compliance
Policy Engine
Guardrails · PII · RBAC
→
Semantic Cache
L1–L4 Cache
Exact + vector + KV
→
Routing
Provider Router
Failover · cost · perf
Upstream Providers (Hidden from Your Customer)
OpenAI
GPT-4o · o1 · o3
|
Anthropic
Claude 3.5 · 3.7
|
Ollama
On-prem open models
|
Any Provider
OpenAI-compatible

Your customers integrate once against your API. You control everything behind it — swapping providers, adjusting compliance rules, rerouting traffic, or adding new models — without any change to client code.

Use Cases

🏢
ISV / SaaS AI Product
You're building an AI-powered SaaS product. Rather than integrating OpenAI directly and exposing your API keys to your backend, Smartflow becomes your AI layer — you ship ai.yourproduct.com backed by whichever models work best, with caching that cuts your token costs by 30–60%.
Ship faster
🤝
MSP / AI Reseller
You're a managed service provider adding AI services to your portfolio. Smartflow lets you provision isolated AI environments per client — their own keys, their own usage quotas, their own compliance policy, their own audit trail — all managed centrally by your team.
New revenue line
🏦
Regulated Enterprise Platform
Your enterprise wants to expose AI to internal teams and partners under tight governance controls. Every request is audited in an immutable VAS log, PII is caught before it leaves the perimeter, and each business unit gets its own policy set — all behind your SSO identity provider.
Compliance-first
🔬
Research / Multi-Tenant Platform
You're running an AI research platform where different teams or institutions need access to different models with different cost budgets and data-handling rules. Smartflow's virtual key system and per-policy routing handles the isolation without per-tenant infrastructure.
Multi-tenancy
⚖️
Legal / Finance AI Tooling
You need to prove to auditors exactly which AI model processed which document, with what prompt, at what time, with what response — and that the data never left your jurisdiction. The immutable VAS audit stream and on-prem deployment make this possible without custom logging infrastructure.
Audit-ready
🤖
AI Agent Infrastructure
Your platform exposes MCP servers and A2A agent APIs. Smartflow's MCP Gateway lets you wrap those as provider-served endpoints, meaning your agents call your branded API and Smartflow routes and audits every tool call — including cost tracking per agent session.
Agent-native

What You Control at Every Layer

Every layer of the Smartflow stack is configurable per-provider, per-tenant, or globally — giving you surgical control without building bespoke infrastructure.

1
Brand & Model Names
Expose any model under any name. acme-premium can route to Claude Sonnet today and GPT-4.1 tomorrow — your customers never change their code.
2
Virtual Key Vault
Issue sk-your-prefix-xxx keys per tenant, user, or department. Set rate limits, token budgets, and expiry per key — all managed through the dashboard.
3
Compliance & Guardrails
Apply different policy sets per tenant or model alias. PII detection, content filtering, data classification, regulatory framework tagging (HIPAA, SOC2, GDPR) — enforced at the proxy before any data touches an LLM.
4
Semantic Cache
L1 exact match, L2 semantic vector, L3 approximate KNN, L4 TTL — all transparent to your customers. They see faster responses; you see lower provider costs. Cache is shared across tenants (configurable) for maximum efficiency.
5
Intelligent Routing
Route by performance, cost, latency, or custom rules. Automatic failover to a backup provider if primary degrades. Weight-based load distribution across providers. All configurable without code changes.
6
Immutable VAS Audit Log
Every request, response, provider used, compliance result, and cache outcome is written to an append-only audit stream backed by Redis (live) and MongoDB (archive). Never deletable, always queryable. Per-tenant views available.
7
Analytics & Observability
Real-time dashboard showing tokens used, cost by tenant, cache hit rate, latency by model, compliance violations — broken down per provider, per model, per virtual key. Prometheus metrics and Splunk integration available.

Smartflow vs. Azure AI Gateway vs. AWS Bedrock

Both Azure AI Gateway and AWS Bedrock offer some form of AI packaging capability, but both require you to stay inside their cloud ecosystem and give up control at the infrastructure layer.

Capability Smartflow Azure AI Gateway AWS Bedrock
Deploy on-prem / private cloud✓ Full support✗ Azure only✗ AWS only
Custom branded endpoint domain✓ Any domain◐ Under Azure domains✗ Not supported
Custom model name aliases✓ Full mapping◐ Via deployment names◐ Limited
Virtual key issuance per tenant✓ Unlimited◐ Via APIM subscriptions✗ IAM only
Mix OpenAI + Anthropic + Ollama✓ Any combination◐ OpenAI-family + limited◐ Bedrock models only
Semantic cache (L1–L4)✓ Built-in◐ Semantic cache only✗ Not available
Immutable per-request audit log✓ VAS stream◐ Via Azure Monitor◐ CloudTrail only
Per-tenant compliance policies✓ MAESTRO engine◐ APIM policies◐ Guardrails (basic)
MCP server exposure✓ Full MCP Gateway✗ Preview only✗ Not available
No vendor lock-in✓ Open deployment✗ Azure dependent✗ AWS dependent
For a deeper technical comparison with Azure AI Gateway specifically — including a complementary adoption path for organisations running APIM today — see Smartflow vs Azure AI Gateway.

Quickstart — Onboard a Provider

In Smartflow 1.7, the Provider Onboarding Wizard in the Routing dashboard reduces provider setup from hours of config file editing to a 5-step guided workflow.

  1. Open the Routing dashboard → click the green Onboard Provider tab.
  2. Select your provider — OpenAI, Anthropic, Google, Azure OpenAI, Ollama, AWS Bedrock, Cohere, or any custom OpenAI-compatible endpoint.
  3. Enter credentials — API key, base URL (pre-filled), optional org ID or deployment name. Keys are stored in the enterprise key vault and never logged.
  4. Test the connection — Smartflow validates the credentials and discovers available models.
  5. Set routing policy — choose default vs. fallback, configure strategy weights (cost / latency / success rate), and add model name mappings (e.g. my-model → gpt-4o).
  6. Activate — one click applies the routing strategy, default provider, and model mappings via the Smartflow API. Done.
After activating a provider, your existing OpenAI SDK clients can call your Smartflow endpoint with zero code changes by setting base_url to your Smartflow URL and using a virtual key. Model names can be the real names or your own aliases — both work.

Environment Variable Checklist

After using the wizard, add the provider credential to your proxy environment. The wizard shows you the exact variable name.

  • OpenAI: OPENAI_API_KEY=sk-...
  • Anthropic: ANTHROPIC_API_KEY=sk-ant-...
  • Google: GOOGLE_API_KEY=AIza...
  • Azure OpenAI: AZURE_OPENAI_API_KEY=... + AZURE_OPENAI_ENDPOINT=...
  • Ollama (self-hosted): No key required — just set OLLAMA_BASE_URL=http://your-ollama-host:11434

Then restart the proxy container: docker compose restart smartflow-proxy (or kubectl rollout restart deployment/smartflow-proxy -n smartflow).

Ready to Package Your AI?

Smartflow 1.7 is available now. The Provider Onboarding Wizard ships with every deployment — on-prem, Kubernetes, or Docker Compose.

See What's in 1.7 Deployment Guide