Monthly Archives: September 2026

AI: LLM Spend Control with LiteLLM Budgets and OpenAI Limits
0 (0)

24 September 2026

The main goal we had when introducing LiteLLM AI Gateway on the project (see LiteLLM: AI Gateway for LLMs – features overview) was access and cost control, because right now we can’t enable auto-recharge for our OpenAI or OpenRouter account: our startup is still in the development and experimentation stage, developers actively use AI agents… Read More: AI: LLM Spend Control with LiteLLM Budgets and OpenAI Limits0… »

Loading

LiteLLM: Debugging AI Cost Monitoring with VictoriaMetrics
5 (1)

16 September 2026

Had a pretty interesting case with monitoring LLM costs through LiteLLM when using multiple providers. What we have: LiteLLM: AI Gateway, all our services work through it, it proxies requests to providers, generates metrics and traces, controls access, spending, etc. OpenAI and OpenRouter: currently the two main providers LiteLLM sends requests to some clients (our… Read More: LiteLLM: Debugging AI Cost Monitoring with VictoriaMetrics5 (1) »

Loading

LiteLLM: Custom Callbacks and LLM Evaluations with Judge LLM
0 (0)

11 September 2026

A quick recap of what we are doing on the project right now: we have a separate hardware server (hostname == “Matrix”), where we run our own self-hosted models with llama.cpp. In our Kubernetes cluster we have a LiteLLM AI Gateway for our clients – Backend API and other project services. Clients send their OpenAI/Anthropic/OpenRouter… Read More: LiteLLM: Custom Callbacks and LLM Evaluations with Judge LLM0 (0) »

Loading

LiteLLM: Custom Callback for Traffic Mirroring and OTel Tracing to VictoriaTraces
0 (0)

9 September 2026

We’re currently setting up our own hardware server where we want to run self-hosted models. But we can’t just switch client traffic to them right away – first we need to see how these self-hosted LLMs will actually perform. So the general idea for now is to keep sending traffic to the primary provider, OpenAI… Read More: LiteLLM: Custom Callback for Traffic Mirroring and OTel Tracing to… »

Loading