Category Archives: Troubleshooting

Various problems solutions

HTTP: redirects, POST and GET requests, and the “lost” data
0 (0)

23 November 2019

We have a web application which has to accept POST-requests from clients. In front of this application, there is some proxy service, no matter which – initially, we faced the issues on an AWS’s Application Load Balancer, then I reproduced them with NGINX, and it will “work” for any other proxying system. Besides proxying –… Read More »

Loading

Grafana: Loki – the LogQL’s Prometheus-like counters, aggregation functions and dnsmasq’s requests graphs
0 (0)

17 November 2019

The last time I configured Loki for logs collecting and monitoring was in February 2019 – almost a year ago, see the Grafana Labs: Loki – logs collecting and monitoring system post, when Loki was in its Beta state. Now we faced with outgoing traffic issues in our Production environments and can’t find who guilty for… Read More »

Loading

dnsmasq: AWS – “Temporary failure in name resolution”, logs, debug and dnsmasq cache size
0 (0)

28 October 2019

We are using AWS VPC DNS and sometimes facing with errors like “php_network_getaddresses: getaddrinfo failed: Temporary failure in name resolution“. The only advice from AWS tech. support was to configure a local dnsmasq service to act as a local DNS cache, but I did this already year ago and this issue happens once in 1-2-3… Read More »

Loading

Arch Linux: package-query: error while loading shared libraries: libalpm.so.11
0 (0)

26 October 2019

Didn’t install upgrades for a couple of weeks, today packages to be upgraded became over 100, so I run it. Upgrades on my Arch Linux are installed with a simple alias in the.bashrc: … alias osupgrade=”yaourt -Syua –noconfirm” … (seems eventually it’s time to change it to the alias osupgrade=”yay -Syua –noconfirm”) The error Usually,… Read More »

Loading

OpenVPN: the No route to host and ERR_ADDRESS_UNREACHABLE errors – the cause and a solution
0 (0)

14 October 2019

We have an OpenVPN Access Server running, see its setup in the OpenVPN: OpenVPN Access Server set up and AWS VPC peering configuration post. The VPN server is hosted in a first AWS VPC, while a Bitwarden service – in another one VPC. Between those VPCs we have a VPC peering configured, and the OpenVPN… Read More »

Loading

Debian: logrotate won’t rotate logs with an “unknown group ‘syslog'” error
0 (0)

9 October 2019

We have an AWS EC2 with Debian and logrotate. One day its root partition was exhausted and when I started investigating it – found, that we have a bunch of files like /var/log/syslog.N.gz. At the same time by default logrotate creates a config file to rotate syslog log files: [simterm] root@monitoring-dev:~# cat /etc/logrotate.d/syslog # Ansible… Read More »

Loading

PostgreSQL: FATAL: Peer authentication failed for user “name”
0 (0)

11 September 2019

A quick note mainly for myself on how to enable remote access in PostgreSQL and create its database dump. An EC2-server moved to our AWS account with one website. During an attempt to connect to the PostgreSQL running on this host – the “FATAL: Peer authentication failed for user “name”” appears. And the same error… Read More »

Loading

Kubernetes: part 4 – AWS EKS authentification, aws-iam-authenticator and AWS IAM
0 (0)

3 September 2019

Let’s proceed with our AWS Elastic Kubernetes Service, EKS. Previous parts: Kubernetes: part 1 – architecture and main components overview Kubernetes: part 2 – a cluster set up on AWS with AWS cloud-provider and AWS LoadBalancer Kubernetes: part 3 – AWS EKS overview and manual EKS cluster set up. In the previous – Kubernetes: part… Read More »

Loading

SonarQube: solving “Unrecoverable indexation failures” and Elasticsearch “Disk watermark exceeded” errors
0 (0)

29 August 2019

We are using SonarQube started from a Jenkins jobs. See more at SonarQube: running tests from Jenkins Pipeline in Docker. So, the SonarQube is started from the Jenkins – all good here, but inside SonarQube we can see errors: java.lang.IllegalStateException: Unrecoverable indexation failures: 1 errors among 1 requests at org.sonar.server.es.IndexingListener$1.onFinish(IndexingListener.java:39) at org.sonar.server.es.BulkIndexer.stop(BulkIndexer.java:122) … And host… Read More »

Loading

Redis: fork – Cannot allocate memory, Linux, virtual memory and vm.overcommit_memory
0 (0)

28 August 2019

Currently, I’m configuring a Redis as a caching service for our application and during that, I faced with the question: do I need to set vm.overcommit_memory to the value 1, i.e. disable it – or not? The question is quite old for me, see The story, but only now I found time to get to… Read More »

Loading