Skip to main content

Model Providers

📖 Best for: AI product managers / developers / platform admins — people who want to unify access to 60+ LLM vendors, configure load balancing, routing policies, cost tracking, and security compliance

📖 Reading time: 4 minutes

📖 In one sentence: YingCore's model router — abstracts 60+ LLM vendors (OpenAI, Anthropic, Zhipu GLM, Tongyi Qianwen, DeepSeek, SenseTime, MiniMax, Ollama, vLLM, ...) into a single consistent interface. Every AI capability on the platform (chat, retrieval, recognition, speech, knowledge base) goes through models registered here. Accessible from the top navigation "模型供应商 / Model Providers" (route /model-provider) for all roles. Model Providers handle "how to connect", Digital Employees handle "how to use".

I. Core Value​

YingCore compresses the path from "onboarding a new model" (modify code + integration test + deploy) to "fill in an API Key + batch-check models". On top of that, it provides production-grade scheduling and governance:

  • Unified Access — callers only care about capability (chat / embedding / rerank / image / speech / OCR), not the specific vendor. New models come online with zero code change.
  • Reliability — configure multiple models for the same capability as backup; the primary model's failure automatically switches to the backup, with zero user-facing impact.
  • Cost Control — route by capability, cost, and latency. Use small models for simple tasks (save cost), use large models for complex tasks (keep quality).
  • Security & Compliance — API Keys stored encrypted; VPC-internal calls and full audit trail are supported.

Think of it as the "signal tower" for your phone — no matter which carrier you use, the numbers you dial and the data you transmit all feel the same. YingCore brings that same experience to AI models.

II. Core Capabilities (8 items)​

#CapabilityDescription
160+ Vendors Out-of-the-BoxBuilt-in support for OpenAI, Azure OpenAI, Anthropic Claude, Google Gemini, Zhipu GLM, Tongyi Qianwen, DeepSeek, SenseTime, Baidu, ByteDance, MiniMax, Moonshot, etc. No development needed to onboard.
2Unified Interface & 7 Model TypesAbstracts 7 model capabilities — chat / embedding / rerank / image2text / speech2text / tts / ocr. The upper layer calls capability, never a specific vendor. Model cards are color-coded by type and labeled with capability tags (vision, tool calling, context window, etc.).
3Private & Self-Hosted DeploymentConnects to local inference services like Ollama, vLLM, TGI, LM-Studio, Xinference, GPUStack — data never leaves your network.
4OpenAI-Compatible CustomAny service compatible with the OpenAI protocol can be onboarded by filling in a base_url — no development required.
5Batch Add & Catalog RefreshOne-click fetch of vendor's /v1/models catalog, group-check by type, and auto-skip already-added models. "Refresh Catalog" forces sync of the vendor's latest model list.
6Load Balancing & FailoverConfigure multiple models as backup for the same capability. The primary model's timeout/failure automatically switches to backup, with zero business impact.
7Smart Routing & Cost StatisticsRoute by capability, cost, latency, and compliance. Track token consumption and fees by model, caller, and time window. Budget alerts supported.
8Security & ComplianceAPI Keys stored encrypted; VPC-internal calls supported; full audit trail.

III. Typical Scenarios (5 items)​

Scenario 1: Multi-Model A/B Comparison & Cost Optimization​

Call GPT-4o and Claude 3.5 in parallel for the same task to compare quality and cost. In a customer-service scenario, route 80% of simple questions to GLM-4-Flash, and only escalate complex ones to GPT-4 — overall cost drops by 70%.

Scenario 2: Failover & High Availability​

When the primary vendor (OpenAI) is down, automatically switch to the backup vendor (DeepSeek / Claude). Business is completely unaffected, SLA is more guaranteed — don't put all your eggs in one basket.

Scenario 3: Onboard New Models in 5 Minutes​

When DeepSeek releases a new model, you can enable it on the platform within 5 minutes — just add the API Key + batch-add models, no code change needed. The new model is immediately available to all upper-layer features.

Scenario 4: Domestication / Private Deployment​

For financial and government customers requiring private deployment, switch the model provider to a local vLLM inference cluster — data never leaves the network. Cloud models can still be retained as capability backup, satisfying both compliance and flexibility.

Scenario 5: Multimodal Capability Extension​

To add vision/speech capabilities to a digital employee, simply add GPT-4o / Qwen-VL / Whisper as a model provider — the assistant immediately gains new capabilities without modifying the agent orchestration logic.

IV. Usage Guide​

1. Interface Layout (/model-provider)​

  • Left Sidebar (resizable): split into two groups — "Added" and "Available" vendor lists, with search support.
  • Right Main Area: selecting a vendor shows the vendor info card and model card list for that vendor.
  • Top Actions: Search, Batch Add, Refresh Model Catalog.

2. Add Vendor API Key​

  1. Find the target vendor in the "Available" list on the left.
  2. Click the "API-Key" button on the vendor card and enter the API Key in the popup.
    • Some vendors accept an optional base_url; vendors like MiniMax have additional config fields (e.g. group_id).
    • Special vendors (Azure, Bedrock, Volcengine, Hunyuan, Spark, YiYan, FishAudio, Tencent Cloud, Google Cloud, MinerU, PaddleOCR, OpenDataLoader, etc.) use their own independent config popups.
  3. After saving, the vendor moves to the "Added" list and its model list displays on the right.
  4. (Optional) Click "Refresh Catalog" to force-fetch the vendor's latest models.

3. Batch Add Models​

  1. Click "Batch Add".
  2. The system fetches the vendor's model catalog (/v1/models) in real time.
  3. Group-check by type, add in batch (already-added models are auto-skipped).

4. Onboard Custom / OpenAI-Compatible Services​

  1. Select the OpenAI-API-Compatible vendor.
  2. Fill in base_url; API Key is optional (leave empty for keyless endpoints).
  3. Optionally customize the display name (e.g., "My Kimi").
  4. After saving, the system automatically requests /models to sync all available models — no need to type model names manually.

5. Delete / Replace​

  • Delete a vendor or a single model from the "Added" list to remove it.
  • If a model fails, re-fill the Key or refresh the catalog and try again.

6. Default Model Configuration Recommendations​

Many features (Knowledge Base retrieval test, Wanxiang Graph AI assist, Digital Employees, Marketing AI assistant, etc.) use the chat model you configure in Model Providers. Recommendations:

  1. Configure at least one vendor's chat model as primary — all AI features on the platform depend on it.
  2. For semantic retrieval, configure an embedding model (knowledge base default is fine if available).
  3. For higher retrieval precision, configure a rerank model.

V. Best Practices​

  1. Always have at least two vendors for primary/backup — primary on GPT-4o, backup on Claude 3.5 or DeepSeek. Auto-switch on primary failure, SLA is more reliable.
  2. Tier routing by task difficulty — route simple tasks (classification, extraction, short replies) to GLM-4-Flash / GPT-4o-mini. Only escalate complex tasks (reasoning, long-form writing) to GPT-4 / Claude 3.5 Opus. Cost drops 50%+.
  3. Pick by capability, not brand — when choosing a model, check capability tags first (tool calling / vision / reasoning / context window). Whether it can do the job matters more than which brand it is.
  4. Use local models for privacy — sensitive data (finance, healthcare, government) prefers Ollama / vLLM local models — data never leaves the network. Use cloud models only for general tasks.
  5. Configure Embedding & Rerank separately — chat models and embedding models are different capabilities. For RAG scenarios, always configure a dedicated embedding (e.g. bge / bce). Never use a chat model for vectorization (10x worse quality).
  6. Key rotation & isolation — use dedicated keys for different business lines (production / staging / testing), and rotate regularly. YingCore encrypts Keys at rest by default, but still avoid leaking them to code repos.
  7. Refresh first, then batch-add for new models — when onboarding a new vendor, refresh the catalog first, then batch-add. This syncs all available models in one shot, 10x faster than adding one by one.
  8. Switch back fast when API Key expires — when a single model fails, "Refresh Catalog" first, then "Re-fill Key" — 90% of "model is down" cases are due to expired Key or revoked permissions; a Key re-fill usually restores service.