Tag Archives: VictoriaMetrics

LiteLLM: Debugging AI Cost Monitoring with VictoriaMetrics
0 (0)

16 September 2026

Had a pretty interesting case with monitoring LLM costs through LiteLLM when using multiple providers. What we have: LiteLLM: AI Gateway, all our services work through it, it proxies requests to providers, generates metrics and traces, controls access, spending, etc. OpenAI and OpenRouter: currently the two main providers LiteLLM sends requests to some clients (our… Read More: LiteLLM: Debugging AI Cost Monitoring with VictoriaMetrics0 (0) »

Loading

LiteLLM: Custom Callbacks and LLM Evaluations with Judge LLM
0 (0)

11 September 2026

A quick recap of what we are doing on the project right now: we have a separate hardware server (hostname == “Matrix”), where we run our own self-hosted models with llama.cpp. In our Kubernetes cluster we have a LiteLLM AI Gateway for our clients – Backend API and other project services. Clients send their OpenAI/Anthropic/OpenRouter… Read More: LiteLLM: Custom Callbacks and LLM Evaluations with Judge LLM0 (0) »

Loading

llama.cpp: Metrics and Monitoring with VictoriaMetrics
0 (0)

21 August 2026

We have a server where we’re going to run self-hosted LLMs. We spent quite a while choosing what exactly to use for running the models – vLLM, SGLang, or llama.cpp, and eventually settled on llama.cpp – at least for now. In the post NixOS: getting started, package installation, and system configuration I described installing Node… Read More: llama.cpp: Metrics and Monitoring with VictoriaMetrics0 (0) »

Loading

LiteLLM: Metrics, Traces, and Debugging exception_class=”ValueError”
0 (0)

12 August 2026

A few days ago, I ran into an interesting situation with LiteLLM: on the one hand, the metrics showed a lot of errors “from the provider”, while on the other hand, the traces and alerts showed only a single error. I had to dig into it a bit and figure out some nuances of how… Read More: LiteLLM: Metrics, Traces, and Debugging exception_class=”ValueError”0 (0) »

Loading

LiteLLM: Monitoring with VictoriaMetrics – Alerts and Grafana
0 (0)

17 July 2026

This is the second part of the LiteLLM monitoring series – in the previous one, we covered the general integration with VictoriaStack and looked at the metrics and traces we get from LiteLLM (see LiteLLM: metrics, traces, and integration with the VictoriaMetrics Stack). Now let’s move on to the practical part – what to monitor… Read More: LiteLLM: Monitoring with VictoriaMetrics – Alerts and Grafana0 (0) »

Loading

LiteLLM: Metrics, Traces, and VictoriaMetrics Stack Integration
0 (0)

9 July 2026

Third part on running LiteLLM – AI Gateway or LLM Proxy, and finally we’re getting to monitoring. In the first part we got familiar with LiteLLM in general (see LiteLLM: AI Gateway for LLMs – overview of features), and in the second one we deployed it in Kubernetes and hooked up VictoriaTraces and VictoriaMetrics to… Read More: LiteLLM: Metrics, Traces, and VictoriaMetrics Stack Integration0 (0) »

Loading

LiteLLM: AI Gateway on Kubernetes and Metrics in VictoriaMetrics
0 (0)

26 June 2026

In the first part – LiteLLM: AI Gateway for LLMs – features overview we got familiar with what LiteLLM can do in general – now we can run it in Kubernetes and connect clients. At the same time we’ll check the integration with our existing monitoring stack – for now just metrics to VictoriaMetrics. Logs… Read More: LiteLLM: AI Gateway on Kubernetes and Metrics in VictoriaMetrics0 (0) »

Loading

Claude Code: Monitoring with OpenTelemetry and VictoriaMetrics
0 (0)

18 June 2026

While working on LiteLLM (see LiteLLM: AI Gateway for LLMs – features overview), I had an idea: besides services like our Backend API, why not also monitor the Claude Code developers? Just out of curiosity – to see what’s going on there in general and how everyone uses our Anthropic Organization, because a lot of… Read More: Claude Code: Monitoring with OpenTelemetry and VictoriaMetrics0 (0) »

Loading

VictoriaTraces: Recording Rules, Metrics, and Alerts from Trace Spans
0 (0)

22 May 2026

VictoriaTraces – just like VictoriaLogs – supports Recording Rules (see VictoriaMetrics: Recording Rules for AWS Load Balancer logs) for traces, because traces are essentially the same logs, just structured differently. And since we have Recording Rules – we can build metrics out of logs for alerts and Grafana dashboards. Although this actually isn’t the best… Read More: VictoriaTraces: Recording Rules, Metrics, and Alerts from Trace Spans0 (0) »

Loading

VictoriaTraces: Tracing, Observability, and OpenTelemetry
0 (0)

19 May 2026

On the project, we’ve gradually grown to the point where it’s time to have proper tracing – to build real observability, not just monitoring. A long time ago I did something similar with Jaeger – a monster, and it kind of stayed in my drafts from 2019 or 2020. Since right now our entire stack… Read More: VictoriaTraces: Tracing, Observability, and OpenTelemetry0 (0) »

Loading