Category Archives: Troubleshooting

Various problems solutions

Terraform: “no available releases match the given constraints
0 (0)

6 August 2025

A fairly common error when upgrading module versions, when there are restrictions on module or provider versions, and they do not match each other. The Issue In this case, I merged Pull Requests from Renovate and didn’t notice that terraform-aws-modules/terraform-aws-lambda needed hashicorp/aws provider version 6: And first I upgraded Lambda to version 8. After that… Read More »

Loading

Kubernetes: PVC in StatefulSet, and the “Forbidden updates to statefulset spec” error
0 (0)

22 July 2025

We have a VictoriaLogs Helm chart with a PVC size of 30 GB, which is no longer enough for us, and we need to increase it. But the problem is that .spec.volumeClaimTemplates[*].spec.resources.requests.storage in STS is immutable, that is, we can’t just change the size through values.yaml file, because it will lead to the error“Forbidden: updates… Read More »

Loading

VictoriaMetrics: fixing the “no matches for kind VMAnomaly” error
0 (0)

10 July 2025

It’s not the first time I’ve encountered similar errors when upgrading VictoriaMetrics, so it’s time to note a solution. So, after upgrading the victoria-metrics-k8s-stack 0.55.0 => 0.56.0, errors appeared in the Operator logs: … {“logger”:”controller-runtime.source.EventHandler”,”msg”:”if kind is a CRD, it should be installed before calling Start”,”kind”:”VMAnomaly.operator.victoriametrics.com”,”error”:”no matches for kind \”VMAnomaly\” in version \”operator.victoriametrics.com/v1\””} … {“logger”:”setup”,”msg”:”cannot… Read More »

Loading

Arch Linux: linux-firmware-nvidia: /usr/lib/firmware/nvidia/ exists in filesystem
0 (0)

9 July 2025

Even though the changes were announced in emails from Arch Linux, for some reason, many people have had problems with the latest update: in the Arch Linux subreddits on Reddit, people are complaining like “Everything is broken, what should I do now?” Let’s see how to complete the upgrade and what exactly has changed. The… Read More »

Loading

Kubernetes: 503 errors with AWS ALB possible causes and solutions
0 (0)

9 July 2025

After migration to a new EKS cluster, we started getting alerts about 503 errors sometimes. The errors were happened in three cases: sometimes without any deployment, when all Pods were Running && Ready sometimes during deployment – but only on Dev, because there is only one Pod for API and sometimes during Karpenter Consolidation. Let’s… Read More »

Loading

VictoriaMetrics: migrating VMSingle and VictoriaLogs data between Kubernetes cluster
0 (0)

5 July 2025

We have VictoriaMetrics and VictoriaLogs running on an AWS Elastic Kubernetes Service cluster. We do major upgrades to EKS by creating a new cluster, and therefore we have to transfer monitoring data from the old VMSingle instance to the new one. For VictoriaMetrics, there is the vmctl tool which can migrate data through the APIs… Read More »

Loading

Terraform: using import, and some hidden pitfalls
0 (0)

15 June 2025

Terraform has two ways to bring existing resources under Terraform management – using the Terraform CLI and the terraform import command, or using the import resource. Why might we need to import resources? if we already have a manually configured (the “clickops”) service that we want to bring under Terraform management (for example, the common… Read More »

Loading

Python: introduction to the Celery, and its monitoring configurations
0 (0)

20 May 2025

To put it very simply, Celery is something we can use to perform tasks outside of our main service. For example, there is a Backend API that has some kind of endpoint to which mobile devices send information that the user has created a new whatever in the application. The task of the Backend is… Read More »

Loading

Nexus: Configuring Docker proxy repository, and ContainerD in Kubernetes
0 (0)

17 May 2025

I wrote about launching Nexus in the Nexus: launch in Kubernetes, and PyPI caching repository configuration post, now I want to add Docker image caching to PyPI, especially since Docker Hub introduces new limits from April 1, 2025 – see Docker Hub usage and limits. We’ll do it as usual: first run manually locally on… Read More »

Loading