I actually started to write about creating my own Kubernetes Operator, but decided to make a separate topic about what a Kubernetes CustomResourceDefinition is, and how creating a CRD works at the level of the Kubernetes API and the etcd. That is, to start with how Kubernetes actually works with resources, and what happens when… Read More »
Perhaps everyone has used operators in Kubernetes, for example, PostgreSQL operator, VictoriaMetrics Operator. But what’s going on under the hood? How and to what are CustomResourceDefinition (CRD) applied, and what is an “operator”? And finally, what is the difference between a Kubernetes Operator and a Kubernetes Controller? In the previous part – Kubernetes: Kubernetes APIs,… Read More »
How exactly do resources.requests and resources.limits in a Kubernetes manifest works “under the hood”, and how exactly will Linux allocate and limit resources for containers? So, in Kubernetes for Pods, we can set two main parameters for CPU and Memory – the spec.containers.resources.requests and spec.containers.resources.limits fields: resources.requests: affects how and where a Pod will be… Read More »
It’s not the first time I’ve encountered similar errors when upgrading VictoriaMetrics, so it’s time to note a solution. So, after upgrading the victoria-metrics-k8s-stack 0.55.0 => 0.56.0, errors appeared in the Operator logs: … {“logger”:”controller-runtime.source.EventHandler”,”msg”:”if kind is a CRD, it should be installed before calling Start”,”kind”:”VMAnomaly.operator.victoriametrics.com”,”error”:”no matches for kind \”VMAnomaly\” in version \”operator.victoriametrics.com/v1\””} …… Read More »
After migration to a new EKS cluster, we started getting alerts about 503 errors sometimes. The errors were happened in three cases: sometimes without any deployment, when all Pods were Running && Ready sometimes during deployment – but only on Dev, because there is only one Pod for API and sometimes during Karpenter Consolidation.… Read More »
We have VictoriaMetrics and VictoriaLogs running on an AWS Elastic Kubernetes Service cluster. We do major upgrades to EKS by creating a new cluster, and therefore we have to transfer monitoring data from the old VMSingle instance to the new one. For VictoriaMetrics, there is the vmctl tool which can migrate data through the APIs… Read More »
We have an AWS Elastic Kubernetes Service with the VictoriaMetrics stack deployed (see VictoriaMetrics: deploying a Kubernetes monitoring stack). I need to migrate the data from the old VMSingle Pod to the new one on the new cluster, and to do this, I need to find VMSingle’s data on an EC2. Note: regarding the migration… Read More »
I wrote about launching Nexus in the Nexus: launch in Kubernetes, and PyPI caching repository configuration post, now I want to add Docker image caching to PyPI, especially since Docker Hub introduces new limits from April 1, 2025 – see Docker Hub usage and limits. We’ll do it as usual: first run manually locally on… Read More »
In Kubernetes, we run GitHub Runner for the build and deploy of our Backend API, see GitHub Actions: Running Actions Runner Controller in Kubernetes. But over time, we noticed that there was too much traffic on the NAT Gateway – see VictoriaLogs: a Grafana dashboard for AWS VPC Flow Logs – migrating from Grafana Loki.… Read More »
One day, we looked at our AWS costs on AWS Load Balancers, and understood that we needed to decrease the spends a bit. What we wanted was to have one LoadBalancer, and through it to route requests to different Kubernetes Ingresses and Services in different Namespaces. The first thing that came to mind was… Read More »