Tag Archives: VictoriaTraces

LiteLLM: Debugging AI Cost Monitoring with VictoriaMetrics
0 (0)

16 September 2026

Had a pretty interesting case with monitoring LLM costs through LiteLLM when using multiple providers. What we have: LiteLLM: AI Gateway, all our services work through it, it proxies requests to providers, generates metrics and traces, controls access, spending, etc. OpenAI and OpenRouter: currently the two main providers LiteLLM sends requests to some clients (our… Read More: LiteLLM: Debugging AI Cost Monitoring with VictoriaMetrics0 (0) »

Loading

LiteLLM: Custom Callbacks and LLM Evaluations with Judge LLM
0 (0)

11 September 2026

A quick recap of what we are doing on the project right now: we have a separate hardware server (hostname == “Matrix”), where we run our own self-hosted models with llama.cpp. In our Kubernetes cluster we have a LiteLLM AI Gateway for our clients – Backend API and other project services. Clients send their OpenAI/Anthropic/OpenRouter… Read More: LiteLLM: Custom Callbacks and LLM Evaluations with Judge LLM0 (0) »

Loading

LiteLLM: Custom Callback for Traffic Mirroring and OTel Tracing to VictoriaTraces
0 (0)

9 September 2026

We’re currently setting up our own hardware server where we want to run self-hosted models. But we can’t just switch client traffic to them right away – first we need to see how these self-hosted LLMs will actually perform. So the general idea for now is to keep sending traffic to the primary provider, OpenAI… Read More: LiteLLM: Custom Callback for Traffic Mirroring and OTel Tracing to… »

Loading

LiteLLM: Traffic Mirroring, Batch Completions, and Traffic to Two Providers
0 (0)

21 August 2026

We’re getting ready to launch a self-hosted LLM, and at the testing stage the general idea is to send client requests simultaneously both to the “default production model” like GPT-5.6 and to the model running on our own server. And after getting the responses, we’ll compare them with Phoenix or Opik, and gradually tune our… Read More: LiteLLM: Traffic Mirroring, Batch Completions, and Traffic to Two Providers0… »

Loading

LiteLLM: Metrics, Traces, and VictoriaMetrics Stack Integration
0 (0)

9 July 2026

Third part on running LiteLLM – AI Gateway or LLM Proxy, and finally we’re getting to monitoring. In the first part we got familiar with LiteLLM in general (see LiteLLM: AI Gateway for LLMs – overview of features), and in the second one we deployed it in Kubernetes and hooked up VictoriaTraces and VictoriaMetrics to… Read More: LiteLLM: Metrics, Traces, and VictoriaMetrics Stack Integration0 (0) »

Loading

LiteLLM: AI Gateway for LLMs – Features Overview
0 (0)

4 June 2026

In the previous posts on OpenTelemetry and VictoriaTraces (see OpenTelemetry: OTel Collectors in Kubernetes and integration with the VictoriaMetrics stack and VictoriaTraces: Tracing, Observability and OpenTelemetry) we covered the general concepts of what observability is and how to work with traces. But this topic actually came up on the project when we realized that using… Read More: LiteLLM: AI Gateway for LLMs – Features Overview0 (0) »

Loading

VictoriaTraces: Recording Rules, Metrics, and Alerts from Trace Spans
0 (0)

22 May 2026

VictoriaTraces – just like VictoriaLogs – supports Recording Rules (see VictoriaMetrics: Recording Rules for AWS Load Balancer logs) for traces, because traces are essentially the same logs, just structured differently. And since we have Recording Rules – we can build metrics out of logs for alerts and Grafana dashboards. Although this actually isn’t the best… Read More: VictoriaTraces: Recording Rules, Metrics, and Alerts from Trace Spans0 (0) »

Loading