A self-hosted model-serving stack: an OpenAI-compatible API, repeatable capability benchmarks for the locally served models, and a metrics poller.
- ROLEauthor
- STACKPython · FastAPI · llama-swap · uv
- STATUSself-hosted
- OpenAI-compatible endpoint in front of local models
- Benchmarks on your own tasks instead of leaderboard scores
- Several models on one GPU, loaded on demand