In our Deployments, we are using the NodeLocal DNS as a local DNS cache to reduce requests number to the AWS VPC DNS, see the Kubernetes: load-testing and high-load tuning – problems and solutions for details. Currently, a manifest looks like the next: … dnsPolicy: “None” dnsConfig: nameservers: – 169.254.20.10 … The problem is, that… Read More »
We have an ExternalDNS service running, see the Kubernetes: update AWS Route53 DNS from an Ingress post, which started sending a lot of messages like: msg=”failed to list resource records sets for zone /hostedzone/Z2VM3W5SRY4I9J: Throttling: \n\tstatus code: 400 And even AWS Console in the Route53 says “Throttling error that was caused because API rate was… Read More »
We have a Kubernetes Cluster Autoscaler for AWS EC2 WorkerNode groups scaling. On our Dev cluster sometimes it stop working with the following message in its logs: [simterm] … E0331 08:57:52.264549 1 leaderelection.go:320] error retrieving resource lock kube-system/cluster-autoscaler: Get https://172.20.0.1:443/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cluster-autoscaler: context deadline exceeded I0331 08:58:14.468096 1 leaderelection.go:277] failed to renew lease kube-system/cluster-autoscaler: timed out waiting… Read More »
Faced with a very interesting thing during removal of a Kubernetes Namespace. After a kubectl delete namespace NAMESPACE is executed, the namespace hangs in the Terminating state, and any attempt to forcibly remove it didn’t help. First, let’s see how such a force-removal can be done, and then will check the real cause and a… Read More »
One day our developers said that they can not create new AWS LoadBalancers via Ingress on our Kubernetes Dev cluster (AWS EKS). Balancers are created using AWS ALB Controller, see AWS Elastic Kubernetes Service: running ALB Ingress controller. Errors in the controller’s logs were: E0304 07:12:38.595113 1 controller.go:217] kubebuilder/controller “msg”=”Reconciler error” “error”=”no object matching key… Read More »
We have a PHP application running with Kubernetes in pods with two dedicated containers – NGINX и PHP-FPM. The problem is that during downscaling clients get 502 errors. E.g. when a pod is stopping, its containers can not correctly close existing connections. So, in this post, we will take a closer look at the pods’… Read More »
Locales is a set of environment variables that are used to determine how to display data and time (for example, first of the week), symbols encoding (for example, how to display cyrillic symbols), default files order when one executing the ls command, and so on. Those variables are: LANG: Determines the default locale in the… Read More »
ArgoCD helps to deliver applications to Kubernetes by using the GitOps approach, i.e. when a Git-repository is used as a source of trust, thus all manifest, configs and other data are stored in a repository. It can b used with Kubernetes manifest, kustomize, ksonnet, jsonnet, and what we are using in our project – Helm-charts.… Read More »
Finally got time to migrate the RTFM.CO.UA blog to a new server with Debian 10. This time manually, without any automation will set up a LEMP stack Wrote a similar at 2016 – Debian: установка LEMP — NGINX + PHP-FPM + MariaDB (Rus), but in time the post is more complete of the process and… Read More »
We have a project running in Kubernetes that needs to run SQL migrations during deployment. To run migrations need to clone a Github repository and run actually migrations stored in it. Currently, this is done with Kubernetes initContainers , and there are two of them – the first one with git clones the repository with… Read More »