Category Archives: Monitoring

Hardware, services and network monitoring systems

AWS: Lambda – copy EC2 tags to its EBS, part 2 – create a Lambda function

13 October 2021

 let’s proceed in our journey of the AWS Lambda function, which will copy an EC2’s AWS Tags to all EBS volumes, attached to it. In the first part, AWS: Lambda — copy EC2 tags to its EBS, part 1 – Python and boto3, we wrote a Python script that can get all EC2 instances in… Read More »

AWS: WAF WebACL logging and integration

22 July 2021

 In the first post – AWS: Web Application Firewall overview, configuration, and its monitoring – we spoke about its main components, created a WebACL and Rules for it, and did basic monitoring. Also, we’ve configured WebACL’s logs collection with AWS Kinesis, but now it’s time to see them, as CloudWatch Logs isn’t available for… Read More »

AWS: Web Application Firewall overview, configuration, and its monitoring

19 July 2021

  AWS WAF (Web Application Firewall) is an AWS service for monitoring incoming traffic to secure a web application for suspicious activity like SQL injections. Can be attached to an AWS Application LoadBalancer, AWS CloudFront distribution, Amazon API Gateway, and AWS AppSync GraphQL API. In case of finding any request that sits WAF’s rules, it… Read More »

AWS: CloudTrail overview and integration with CloudWatch and Opsgenie

15 July 2021

 AWS CloudTrail is a service for auditing AWS accounts events and is enabled by default. It saves all actions that were done by a user, IAM role, or an AWS service via AWS Console, AWS CLI, or AWS SDK. CloudTrail will write information about every API call, log in to the system, services events, and… Read More »

AWS: Simple Email Service Bounce rate and monitoring with and Prometheus

14 July 2021

 Recently, AWS blocked our AWS Simple Email Service because of its low bounce rate. This can be checked in the AWS SES > Reputation Dashboard, our account currently has Under review status: After we’ve connected AWS Tech Support, they enabled it back, but we must solve the issue asap, and have to monitor AWS SES… Read More »

Kubernetes: namespace hangs in Terminating and metrics-server non-obviousness

1 April 2021

 Faced with a very interesting thing during removal of a Kubernetes Namespace. After a kubectl delete namespace NAMESPACE is executed, the namespace hangs in the Terminating state, and any attempt to forcibly remove it didn’t help. First, let’s see how such a force-removal can be done, and then will check the real cause and a… Read More »

Opsgenie: integration with AWS RDS and alerting

18 March 2021

 Let’s configure Opsgenie with AWS RDS. The idea is to get notifications from RDS about events and send them to Opsgenie which will send them to our Slack. To do so, we need to configure AWS Simple Notification Service and AWS RDS Event subscriptions. The official documentation is here>>>. Opsgenie confiuration Go to the Integrations… Read More » collection logs from Kubernetes – fluentd vs filebeat

1 February 2021

 We are using to collect our Kubernetes cluster logs (also, there is a local Loki instance). Logs are collected and processed by a Fluentd pod on every WorkerNode which are deployed from a DaemonSet in its default configuration, see the documentation here – logzio-k8s. The problem we faced is that those pods are consuming… Read More »

Prometheus: Alertmanager Web UI alerts Silence

26 January 2021

 Active alerts sending frequency via Alertmanager is configured via the repeat_interval in the /etc/alertmanager/config.yml file. We have this interval set to 15 minutes, and as result, we have notifications about alerts in our Slack each fifteen minutes. Still, some alerts are such a “known issues”, when we already started the investigation or fixing it, but… Read More »