AI Server
A self-hosted Python AI server that runs local LLMs behind OpenAI- and Anthropic-compatible APIs, with agents, RAG, multi-modal services and MCP management.
What it is
AI Server is a Python 3.11+ Flask application that serves large language models through llama.cpp, vLLM and MLX backends. It exposes OpenAI-compatible endpoints for chat, embeddings, audio, images, responses and fine-tuning, and an Anthropic-compatible messages endpoint. Its agent layer runs on LangGraph, DeepAgents or a native runtime and ships 54 specialty agents in four packs, alongside connector agents for 38 enterprise systems. Retrieval-augmented generation uses FAISS or ChromaDB with clustering, graph extraction and grounding checks, next to speech, vision, image generation, MCP server management and enterprise sign-in.
What you get
OpenAI-compatible API for chat, embeddings, audio, images, responses and fine-tuning, plus an Anthropic-compatible messages API
Local inference with llama.cpp, vLLM and MLX, plus Hugging Face and Ollama model discovery
54 specialty agents (coding, business analysis, office work, oil and gas) on LangGraph, DeepAgents or a native runtime
Connector agents for 38 enterprise systems, including SAP, Salesforce, ServiceNow, Kafka and SharePoint
RAG with FAISS or ChromaDB, distributed clusters, graph extraction and grounding validation
Speech-to-text, text-to-speech, voice-to-voice, text-to-image, vision and object detection
MCP server lifecycle management, tool catalog and tooling bridge
JWT, API tokens, OAuth2/OIDC, LDAP, Azure AD, MFA and RBAC; SDKs for 14 languages plus Godot and Unity
At a glance
- Category
- AI
- Price
- free
- Built for
- For teams running and training AI on their own machines.
More in AI
A self-hosted community platform for sharing machine learning datasets, notebooks and models, and running leaderboard competitions.
A self-hosted research workspace for documents, qualitative coding, references and evidence synthesis, with its AI provided by AI Server.
A guided, web-based machine learning workspace for projects, visual workflows, experiments and model serving.