Helm: UPGRADE FAILED: another operation (install/upgrade/rollback) is in progress

By | 05/24/2024
 

Sometimes, when deploying Helm charts, the error “UPGRADE FAILED: another operation (install/upgrade/rollback) is in progress” may appear:

It can occur because the previous deployment failed due to errors in the chart, or the connection between the build machine and the Kubernetes cluster was lost.

Check the release status with ls --all:

$ helm -n dev-backend-api-ns ls --all
NAME            NAMESPACE               REVISION        UPDATED                                 STATUS          CHART           APP VERSION
dev-backend-api dev-backend-api-ns      590             2024-05-23 09:11:51.332096671 +0000 UTC pending-upgrade kraken-0.1.0    1.16.0     

And we see that it is the “pending-upgrade” instead of “deployed“.

You can also see from helm history what happened there before and what the status is now:

And again we see the same status “Preparing upgrade” instead of “Upgrade complete“.

Okay, let’s fix it.

The first option – the hard way – is to simply uninstall the release with helm uninstall, and redeploy with helm upgrade --install, but this will delete all the resources that were created by this chart.

Another option is to do a helm rollback to the previous stable deploy.

In this case, it was 587 – Upgrade complete.

Run it:

$ helm -n dev-backend-api-ns rollback dev-backend-api 587
Rollback was a success! Happy Helming!

Let’s check the status now:

$ helm -n dev-backend-api-ns ls --all
NAME            NAMESPACE               REVISION        UPDATED                                         STATUS          CHART           APP VERSION
dev-backend-api dev-backend-api-ns      591             2024-05-23 13:01:36.905101349 +0300 EEST        deployed        kraken-0.1.0    1.16.0

Or:

$ helm -n dev-backend-api-ns status dev-backend-api
NAME: dev-backend-api
LAST DEPLOYED: Thu May 23 13:01:36 2024
NAMESPACE: dev-backend-api-ns
STATUS: deployed
REVISION: 591
TEST SUITE: None

Restart the job in GitHub Actions and everything works now.