Continuing the topic of AWS Load Balancer logs: in the previous post, Golang: recording AWS Load Balancer logs to VictoriaLogs, we collected logs using our own logs collector in Golang. Now we need to get something useful out of these logs. Previously, when I had Loki on my project, we used its RecordingRules to create… Read More »
In the previous part, AWS: Getting Started with OpenSearch Service as a Vector Store, we looked at AWS OpenSearch Service in general, figured out how data is organized in it, what shards and nodes are, and what types of instances we actually need for data nodes. The next step is to create a cluster and… Read More »
AWS EKS Terraform module version v21.0.0 added support for the AWS Provider Version 6. Documentation – here>>>. The main changes in the AWS EKS module are the replacement of IRSA with EKS Pod Identity for the Karpenter sub-module: Native support for IAM roles for service accounts (IRSA) has been removed; EKS Pod Identity is now… Read More »
After migration to a new EKS cluster, we started getting alerts about 503 errors sometimes. The errors were happened in three cases: sometimes without any deployment, when all Pods were Running && Ready sometimes during deployment – but only on Dev, because there is only one Pod for API and sometimes during Karpenter Consolidation. Let’s… Read More »
To put it very simply, Celery is something we can use to perform tasks outside of our main service. For example, there is a Backend API that has some kind of endpoint to which mobile devices send information that the user has created a new whatever in the application. The task of the Backend is… Read More »
In Kubernetes, we run GitHub Runner for the build and deploy of our Backend API, see GitHub Actions: Running Actions Runner Controller in Kubernetes. But over time, we noticed that there was too much traffic on the NAT Gateway – see VictoriaLogs: a Grafana dashboard for AWS VPC Flow Logs – migrating from Grafana Loki.… Read More »
One day, we looked at our AWS costs on AWS Load Balancers, and understood that we needed to decrease the spends a bit. What we wanted was to have one LoadBalancer, and through it to route requests to different Kubernetes Ingresses and Services in different Namespaces. The first thing that came to mind was either… Read More »
We run GitHub Runners in Kubernetes to build and deploy our Backend API, see GitHub Actions: running the Actions Runner Controller in Kubernetes. But over time, we noticed that there was too much traffic on the NAT Gateway – see VictoriaLogs: a Grafana dashboard for AWS VPC Flow Logs – migrating from Grafana Loki. The… Read More »
So, we’re back to the topic of AWS VPC Flow Logs, VictoriaLogs, and the Grafana dashboard. In the post VictoriaLogs: a Grafana dashboard for AWS VPC Flow Logs – migrating from Grafana Loki, we created a cool dashboard to display various statistics on AWS NAT Gateway traffic. But there is a small drawback: all the… Read More »
In the previous post – AWS: VPC Flow Logs – logs to S3 and Grafana dashboard with Loki, we created a Grafana dashboard that displays NAT Gateway traffic usage statistics. What we were interested in there was which Kubernetes Pods use the most bytes, because it directly affects our AWS Costs. And everything appears to… Read More »