Category Archives: Loki

Loki – Prometheus-inspired logging for cloud natives

Grafana: values ​​from records in Loki logs, and dual-Y-axes panels in Grafana
0 (0)

19 August 2023

We have a function in AWS Lambda, that is writing logs to CloudWatch Logs, from where with the lambda-promtail we are getting them to a Grafana Loki instance to use them in Grafana graphs. What the task is: in the logs, we have records about “Init duration” and “Max Memory Used” by Lambdas. There are no… Read More »

Loading

Grafana Loki: performance optimization with Recording Rules, caching, and parallel queries
0 (0)

19 August 2023

So, we have Loki installed from the chart in simple-scale mode, see Grafana Loki: architecture and running in Kubernetes with AWS S3 storage and boltdb-shipper. Loki is runnings on an AWS Elastic Kubernetes Service cluster, installed with Loki Helm chart, AWS S3 is used as a long-term store, and BoltDB Shipper is used to work… Read More »

Loading

AWS: Grafana Loki, InterZone traffic in AWS, and Kubernetes nodeAffinity
0 (0)

19 August 2023

Traffic in AWS is generally quite an interesting and sometimes complicated thing, I once wrote about it in the AWS: Cost optimization – services expenses overview and traffic costs in AWS. Now, it’s time to return to this topic again. So, what’s the problem: in AWS Cost Explorer, I’ve noticed that we have an increase… Read More »

Loading

VictoriaMetrics: deploying a Kubernetes monitoring stack
0 (0)

23 July 2023

Now we have VictoriaMetrics + Grafana on a regular EC2 instance, launched with Docker Compose, see the VictoriaMetrics: an overview and its use instead of Prometheus. It was kind of a Proof of Concept, and it’s time to launch it “in an adult way” – in Kubernetes and all the configurations stored in a GitHub… Read More »

Loading

Loki: collecting logs from CloudWatch Logs using Lambda Promtail
0 (0)

20 May 2023

Collecting logs in Grafana Loki with Kubernetes is very simple – we just launch Promtail in DaemonSet, configure it to read all data from /var/logs – and that’s it (in fact, we don’t specify anything at all – everything works out of the box from the Helm chart). But what about CloudWatch Logs? On my new project,… Read More »

Loading

Grafana Loki: alerts from the Loki Ruler and labels from logs
0 (0)

12 March 2023

For general information on Grafana Loki, see the Grafana Loki: architecture and running in Kubernetes with AWS S3 storage and boltdb-shipper. Among other services that make up Loki, there is a separate service called ruler that is responsible for working with alerts that can be generated directly from logs. The idea is very simple: create… Read More »

Loading

Grafana Loki: alerts from Ruler and labels from logs
0 (0)

7 January 2023

For general information about Grafana Loki, see the Grafana Loki: architecture and running in Kubernetes with AWS S3 storage and boltdb-shipper post. Among other components of the Loki, there is a separate service called ruler that is responsible for working with alerts that can be generated directly from logs. The idea is very simple: create… Read More »

Loading

Grafana Loki: architecture and running in Kubernetes with AWS S3 storage and boltdb-shipper
0 (0)

7 January 2023

The last time I worked with Loki when it was still in Beta, and it looked much simpler then than it does now. In the new project, there is no logging system at all, and since we all love the Grafana stack, we also decided to use Loki for logging. Although to be honest, I… Read More »

Loading

Grafana: Loki – the LogQL’s Prometheus-like counters, aggregation functions and dnsmasq’s requests graphs
0 (0)

17 November 2019

The last time I configured Loki for logs collecting and monitoring was in February 2019 – almost a year ago, see the Grafana Labs: Loki – logs collecting and monitoring system post, when Loki was in its Beta state. Now we faced with outgoing traffic issues in our Production environments and can’t find who guilty for… Read More »

Loading