Category Archives: Monitoring

Hardware, services and network monitoring systems

LiteLLM: Traffic Mirroring, Batch Completions, and Traffic to Two Providers
0 (0)

21 August 2026

We’re getting ready to launch a self-hosted LLM, and at the testing stage the general idea is to send client requests simultaneously both to the “default production model” like GPT-5.6 and to the model running on our own server. And after getting the responses, we’ll compare them with Phoenix or Opik, and gradually tune our… Read More: LiteLLM: Traffic Mirroring, Batch Completions, and Traffic to Two Providers0… »

Loading

llama.cpp: Metrics and Monitoring with VictoriaMetrics
0 (0)

21 August 2026

We have a server where we’re going to run self-hosted LLMs. We spent quite a while choosing what exactly to use for running the models – vLLM, SGLang, or llama.cpp, and eventually settled on llama.cpp – at least for now. In the post NixOS: getting started, package installation, and system configuration I described installing Node… Read More: llama.cpp: Metrics and Monitoring with VictoriaMetrics0 (0) »

Loading

LiteLLM: Metrics, Traces, and Debugging exception_class=”ValueError”
0 (0)

12 August 2026

A few days ago, I ran into an interesting situation with LiteLLM: on the one hand, the metrics showed a lot of errors “from the provider”, while on the other hand, the traces and alerts showed only a single error. I had to dig into it a bit and figure out some nuances of how… Read More: LiteLLM: Metrics, Traces, and Debugging exception_class=”ValueError”0 (0) »

Loading

NixOS: Getting Started, Installing Packages, and Configuring the System
0 (0)

5 August 2026

We got a new instance, a hardware server that will run our self-hosted LLMs. The server will run NixOS – not my choice, but the system looks interesting. I’ve been hearing about it for a long time, and now I have a great opportunity to get familiar with it. For now, my part is only… Read More: NixOS: Getting Started, Installing Packages, and Configuring the System0 (0) »

Loading

LiteLLM: Monitoring with VictoriaMetrics – Alerts and Grafana
0 (0)

17 July 2026

This is the second part of the LiteLLM monitoring series – in the previous one, we covered the general integration with VictoriaStack and looked at the metrics and traces we get from LiteLLM (see LiteLLM: metrics, traces, and integration with the VictoriaMetrics Stack). Now let’s move on to the practical part – what to monitor… Read More: LiteLLM: Monitoring with VictoriaMetrics – Alerts and Grafana0 (0) »

Loading

LiteLLM: Metrics, Traces, and VictoriaMetrics Stack Integration
0 (0)

9 July 2026

Third part on running LiteLLM – AI Gateway or LLM Proxy, and finally we’re getting to monitoring. In the first part we got familiar with LiteLLM in general (see LiteLLM: AI Gateway for LLMs – overview of features), and in the second one we deployed it in Kubernetes and hooked up VictoriaTraces and VictoriaMetrics to… Read More: LiteLLM: Metrics, Traces, and VictoriaMetrics Stack Integration0 (0) »

Loading

LiteLLM: AI Gateway on Kubernetes and Metrics in VictoriaMetrics
0 (0)

26 June 2026

In the first part – LiteLLM: AI Gateway for LLMs – features overview we got familiar with what LiteLLM can do in general – now we can run it in Kubernetes and connect clients. At the same time we’ll check the integration with our existing monitoring stack – for now just metrics to VictoriaMetrics. Logs… Read More: LiteLLM: AI Gateway on Kubernetes and Metrics in VictoriaMetrics0 (0) »

Loading

Claude Code: Monitoring with OpenTelemetry and VictoriaMetrics
0 (0)

18 June 2026

While working on LiteLLM (see LiteLLM: AI Gateway for LLMs – features overview), I had an idea: besides services like our Backend API, why not also monitor the Claude Code developers? Just out of curiosity – to see what’s going on there in general and how everyone uses our Anthropic Organization, because a lot of… Read More: Claude Code: Monitoring with OpenTelemetry and VictoriaMetrics0 (0) »

Loading

LiteLLM: AI Gateway for LLMs – Features Overview
0 (0)

4 June 2026

In the previous posts on OpenTelemetry and VictoriaTraces (see OpenTelemetry: OTel Collectors in Kubernetes and integration with the VictoriaMetrics stack and VictoriaTraces: Tracing, Observability and OpenTelemetry) we covered the general concepts of what observability is and how to work with traces. But this topic actually came up on the project when we realized that using… Read More: LiteLLM: AI Gateway for LLMs – Features Overview0 (0) »

Loading

VictoriaTraces: Recording Rules, Metrics, and Alerts from Trace Spans
0 (0)

22 May 2026

VictoriaTraces – just like VictoriaLogs – supports Recording Rules (see VictoriaMetrics: Recording Rules for AWS Load Balancer logs) for traces, because traces are essentially the same logs, just structured differently. And since we have Recording Rules – we can build metrics out of logs for alerts and Grafana dashboards. Although this actually isn’t the best… Read More: VictoriaTraces: Recording Rules, Metrics, and Alerts from Trace Spans0 (0) »

Loading