Tag Archives: LLM

LiteLLM: Custom Callbacks and LLM Evaluations with Judge LLM
0 (0)

11 September 2026

A quick recap of what we are doing on the project right now: we have a separate hardware server (hostname == “Matrix”), where we run our own self-hosted models with llama.cpp. In our Kubernetes cluster we have a LiteLLM AI Gateway for our clients – Backend API and other project services. Clients send their OpenAI/Anthropic/OpenRouter… Read More: LiteLLM: Custom Callbacks and LLM Evaluations with Judge LLM0 (0) »

Loading

LiteLLM: Custom Callback for Traffic Mirroring and OTel Tracing to VictoriaTraces
0 (0)

9 September 2026

We’re currently setting up our own hardware server where we want to run self-hosted models. But we can’t just switch client traffic to them right away – first we need to see how these self-hosted LLMs will actually perform. So the general idea for now is to keep sending traffic to the primary provider, OpenAI… Read More: LiteLLM: Custom Callback for Traffic Mirroring and OTel Tracing to… »

Loading

LiteLLM: AI Gateway for LLMs – Features Overview
0 (0)

4 June 2026

In the previous posts on OpenTelemetry and VictoriaTraces (see OpenTelemetry: OTel Collectors in Kubernetes and integration with the VictoriaMetrics stack and VictoriaTraces: Tracing, Observability and OpenTelemetry) we covered the general concepts of what observability is and how to work with traces. But this topic actually came up on the project when we realized that using… Read More: LiteLLM: AI Gateway for LLMs – Features Overview0 (0) »

Loading