Deploy certified inference runtimes across distributed nodes.
UmamiEdge v6 introduces the marketplace framing: standardized runtime packs, compatibility tiers, health checks, model metadata, and node placement policies.
| Runtime pack | Type | Status | Models | Best for |
|---|---|---|---|---|
| vLLM Runtime Pack | LLM serving | ready | Llama, Qwen, Mistral, DeepSeek distilled | high-throughput chat completions |
| Ollama Edge Pack | Local model serving | ready | small/medium local models | pilot nodes and developer demos |
| TGI Enterprise Pack | Text generation | pilot | Hugging Face models | enterprise-hosted inference |
| Embedding Gateway | Vector embedding | ready | BGE, E5, OpenAI-compatible | RAG and search workloads |
| Vision Inference Pack | Vision AI | planned | classification, OCR, detection | agriculture, field ops, security review |