Open source · LLM gateways & observability
Open-source llm gateways & observability
There are 5 open-source llm gateways & observability worth knowing about, led by LiteLLM and Langfuse. They are most often used to replace OpenAI API, LangChain. All can be self-hosted, which removes per-seat pricing entirely — but only pays off if someone on your side will own updates, backups and security patches.
Reviewed 2026-07-28 How we compare Full catalog
The projects
| Project | What it is | Commonly replaces | Source |
|---|---|---|---|
| LiteLLM | Self-hosted proxy giving every provider an OpenAI-shaped API. | OpenAI API | repo ↗ |
| Langfuse | Open-source tracing, evals and prompt management. | LangChain | repo ↗ |
| Helicone | Observability and caching that sits in front of your calls. | LangChain | repo ↗ |
| Portkey | AI gateway with routing, fallbacks and guardrails. | OpenAI API | repo ↗ |
| Phoenix | Open-source tracing and evaluation for LLM apps. | LangChain | repo ↗ |
OpenAI API 
LangChain 
Don't self-host if…
- Nobody has agreed to own patching — an unmaintained service facing the internet is a liability
- You have never tested restoring from a backup of anything
- The tool would hold data whose loss would genuinely hurt, with no second copy
- Your team's time is worth more than the subscription you are trying to avoid
We list these because they are often the right answer. We say this because they are not always the right answer.
Questions people actually ask
What is the best open-source llm gateway?
LiteLLM is the most widely deployed of the 5 projects here — self-hosted proxy giving every provider an openai-shaped api. The honest answer depends less on features than on operations: the best one is whichever your team will actually keep patched and backed up.
Are open-source llm gateways & observability really free?
The software is free; running it is not. Budget roughly $5–20 a month for a small server, plus your own time for updates, backups and security patching. You are swapping a subscription for maintenance work, which is a good trade only if someone will actually do the work.
Can I self-host without a sysadmin?
Often yes, with Docker and a managed VPS. Several of these projects also sell a hosted version, which keeps the open-source licence and data-ownership benefits without the maintenance burden — at that point you are comparing their price against the commercial incumbent's.
Is open source safe for business use?
Widely deployed open-source software is not inherently less secure — the code is inspectable, which is an advantage. The genuine risk is operational: an unpatched self-hosted service facing the internet is far more dangerous than a maintained SaaS product. Security depends on your discipline, not the licence.