We have an AWS EKS cluster with WorkerNodes/EC2 created with Karpenter. The process of creating the infrastructure, cluster, and launching Karpenter is described in previous posts: Terraform: Building EKS, part 1 – VPC, Subnets and Endpoints Terraform: Building EKS, part 2 – an EKS cluster, WorkerNodes, and IAM Terraform: Building EKS, part 3 –… Read More »
Sometimes, when deploying Helm charts, the error “UPGRADE FAILED: another operation (install/upgrade/rollback) is in progress” may appear: It can occur because the previous deployment failed due to errors in the chart, or the connection between the build machine and the Kubernetes cluster was lost. Check the release status with ls –all: $ helm -n dev-backend-api-ns… Read More »
We have a relatively large spending on AWS NAT Gateway Processed Bytes, and it became interesting to know what exactly is processed through it. It would seem that everything is simple – just turn on VPC Flow Logs and see what’s what. But when it comes to AWS Elastic Kubernetes Service and NAT Gateways, things… Read More »
Tracing allows you to track requests between components, that is, for example, when using AWS and Kubernetes we can trace the entire path of a request from AWS Load Balancer to Kubernetes Pod and to DynamoDB or RDS. This helps us both to track performance issues – where and which requests are taking a long… Read More »
Each WorkerNode in a Kubernetes cluster can have a limited number of Pods running, and this limit is determined by three parameters: CPU: the total number of requests.cpu cannot be more than the number of CPUs on the Node Memory: the total number of requests.memory cannot be more than the Memory on the Node IP:… Read More »
An example of creating a Terraform module to automate log collection from AWS Load Balancers in Grafana Loki. See how the scheme works in the Grafana Loki: collecting AWS LoadBalancer logs from S3 with Promtail Lambda blog. In short, ALB writes logs to an S3 bucket, from where they are picked up by a Lambda… Read More »
We have an AWS Elastic Kubernetes Service cluster with Karpenter which is responsible for EC2 auto-scaling, see AWS: Getting started with Karpenter for autoscaling in EKS, and its installation with Helm. In general, there are no problems with it so far, but in any case we need to monitor it. For its monitoring, Karpenter provides… Read More »
Another very interesting new feature from the latest re:Invent is the EKS Pod Identities: a new ability to manage Pod access to AWS resources. The current state: IAM Roles for Service Accounts Before that, we used the IAM Roles for Service Accounts (IRSA) model, where in order to give a Pod access to, for example,… Read More »
We have two new projects in the Elastic Kubernetes Service (см. AWS Elastic Kubernetes Service: a cluster creation automation, part 1 – CloudFormation), each project lives in its own separate Namespace. In addition, there are two users, developers, who need to be given access to these two Namespaces, but only to Pods in them and… Read More »
Currently, we are able to collect our API Gateway logs from the CloudWatch Logs to Grafana Loki, see. Loki: collecting logs from CloudWatch Logs using Lambda Promtail. But in the process of migrating to Kubernetes, we have Application Load Balancers that can only write logs to S3, and we need to learn how to collect… Read More »