The RTFM migration from DigitalOcean to AWS went smoothly, and I’m gradually settling in. New infrastructure, everything new – so for the first while I want to keep a close eye on the server and blog state, which means setting up basic monitoring for WordPress: NGINX, PHP-FPM, the database, and the infrastructure running it all.… Read More »
Just a quick note, because I’ve needed to do something like this fairly often – and only today discovered how elegantly it’s done with vmalert. So, sometimes in an alert you want to display multiple $value entries, for example: – alert: OpenAI Budget Usage expr: | openai_budget_used_usd / openai_budget_total_usd * 100 > 80 … annotations:… Read More »
A continuation of the home NAS setup series. Monitoring in general has already been configured in previous parts, but log management still needs to be set up – because doing it in the console with tail -f /var/log/messages is certainly possible – but there are more convenient tools available. We’ll use VictoriaLogs – especially since… Read More »
I think all Opsgenie users are aware that Atlassian is killing is shutting down the project. I’ve been using Opsgenie since 2018, got used to it, and overall it had everything I needed from an alerting system – a bit rough around the edges in places, but the necessary integrations worked and were easy enough… Read More »
In the previous post FreeBSD: Home NAS, part 10 – monitoring with VictoriaMetrics and Grafana, we configured VictoriaMetrics, node_exporter, Grafana and created a basic dashboard and basic alerts. Now, I want to add a bit more monitoring – to see process CPU/RAM data, SMART information, and ZFS details. Everything written here has been added to… Read More »
Finally got around to monitoring. I was interested in running a standard stack with VictoriaMetrics + Grafana + Alertmanager not in the usual Kubernetes with a Helm chart, but simply on the host. However, the approach is the same as monitoring services in AWS/Kubernetes – on FreeBSD, we will have VictoriaMetrics for metrics, Grafana for… Read More »
Continuing the topic of AWS Load Balancer logs: in the previous post, Golang: recording AWS Load Balancer logs to VictoriaLogs, we collected logs using our own logs collector in Golang. Now we need to get something useful out of these logs. Previously, when I had Loki on my project, we used its RecordingRules to create… Read More »
The next task I want to do with Golang is building a custom log collector that retrieves AWS Application Load Balancer logs from S3 and sends them to VictoriaLogs. Sure, we could simply use Vector.dev, like what I did for AWS VPC Flow Logs (see: Vector.dev introduction, logs from AWS S3, and VictoriaLogs integration). But… Read More »
Got a new task to monitor costs on OpenAI – to see how much each project spends per day, and send alerts to Slack if costs exceed a set threshold. I tried several existing exporters for the OpenAI API, but none provided cost metrics per project – so we’ll build one ourselves. To create the… Read More »
One day I received one of the default VictoriaMetrics alerts that are generated during the deployment of the Helm chart victoria-metrics-k8s-stack: I thought about writing a short post like “What is Churn Rate and how to fix it,” but in the end, I ended up diving deep into how VictoriaMetrics works with data in general… Read More »