Gorush is a Go-written application which we are planning to use to send push notifications to our mobile clients.
The project’s home – https://github.com/appleboy/gorush
The service will be running in our Kubernetes cluster in a dedicated namespace and must be accessible within the cluster’s VPC only, so we will use an Internal ALB from AWS.
Contents
Run Gorush service
Namespace
Clone a repository:
[simterm]
$ git clone https://github.com/appleboy/gorush $ cd gorush/k8s/
[/simterm]
Create a namespace and a confgiMap which will be used to configure access to a local Redis service:
[simterm]
$ kubectl apply -f gorush-namespace.yaml namespace/gorush created $ kubectl apply -f gorush-configmap.yaml configmap/gorush-config created
[/simterm]
We also can use this configMap later to add our own config-file for the Gorush service.
Check resources:
[simterm]
$ kubectl -n gorush get cm NAME DATA AGE gorush-config 2 20s
[/simterm]
Redis
Spin up a Redis service:
[simterm]
$ kubectl apply -f gorush-redis-deployment.yaml deployment.extensions/redis created $ kubectl apply -f gorush-redis-service.yaml service/redis created
[/simterm]
Gorush
I’d like to add an additional pod with Debian to run tests after creating the Gorush to make sure it’s working, so let’s update the deployment file gorush-deployment.yaml with a new container named Bastion:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: gorush
namespace: gorush
spec:
replicas: 3
template:
metadata:
labels:
app: gorush
tier: frontend
spec:
containers:
- image: appleboy/gorush
name: gorush
imagePullPolicy: Always
ports:
- containerPort: 8088
livenessProbe:
httpGet:
path: /healthz
port: 8088
initialDelaySeconds: 3
periodSeconds: 3
env:
- name: GORUSH_STAT_ENGINE
valueFrom:
configMapKeyRef:
name: gorush-config
key: stat.engine
- name: GORUSH_STAT_REDIS_ADDR
valueFrom:
configMapKeyRef:
name: gorush-config
key: stat.redis.host
- image: debian
name: bastion
command: ["sleep"]
args: ["600"]
And now we can start the Gorush itself:
[simterm]
$ kubectl apply -f gorush-deployment.yaml deployment.extensions/gorush created
[/simterm]
Check pods:
[simterm]
$ kubectl -n gorush get po NAME READY STATUS RESTARTS AGE gorush-59bd9dd4fc-dzm47 2/2 Running 0 8s gorush-59bd9dd4fc-fkrhw 2/2 Running 0 8s gorush-59bd9dd4fc-klsbz 2/2 Running 0 8s redis-7d5844c58d-7j5jp 1/1 Running 0 3m1s
[/simterm]
AWS Internal Application Load Balancer
The next things to create are a Service with the NodePort and an Ingress service which will trigger an alb-ingress-controller to create a new load balancer on an AWS account.
Update the gorush-service.yaml and uncomment the NodePort and comment out the LoadBalancer lines:
... #type: LoadBalancer type: NodePort ...
So now it looks like:
apiVersion: v1
kind: Service
metadata:
name: gorush
namespace: gorush
labels:
app: gorush
tier: frontend
spec:
selector:
app: gorush
tier: frontend
# if your cluster supports it, uncomment the following to automatically create
# an external load-balanced IP for the frontend service.
#type: LoadBalancer
type: NodePort
ports:
- protocol: TCP
port: 80
targetPort: 8088
Create the service:
[simterm]
$ kubectl apply -f gorush-service.yaml service/gorush created
[/simterm]
And let’s configure the ALB. Update the gorush-aws-alb-ingress.yaml, add the kubernetes.io/ingress.class: alb to trigger the alb-ingress-controller, update scheme to the internal value, and set values for the subnets and a security-group.
Also, notice that servicePort was changed here from the original 8088 to the 80, as we are using an ALB and a dedicated NodePort Service:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: gorush
namespace: gorush
annotations:
# Kubernetes Ingress Controller for AWS ALB
# https://github.com/coreos/alb-ingress-controller
#alb.ingress.kubernetes.io/scheme: internet-facing
kubernetes.io/ingress.class: alb
alb.ingress.kubernetes.io/scheme: internal
alb.ingress.kubernetes.io/subnets: subnet-010f9918532f52c6d, subnet-0f6dbde36b6669f48
alb.ingress.kubernetes.io/security-groups: sg-0f1df776a767a2589
spec:
rules:
- http:
paths:
- path: /*
backend:
serviceName: gorush
servicePort: 80
Create an ALB:
[simterm]
$ kubectl apply -f gorush-aws-alb-ingress.yaml ingress.extensions/gorush created
[/simterm]
Check alb-ingress-controller logs:
[simterm]
$ kubectl logs -f -n kube-system $(kubectl get po -n kube-system | egrep -o 'alb-ingress[a-zA-Z0-9-]+')
...
E0206 11:08:52.664554 1 controller.go:217] kubebuilder/controller "msg"="Reconciler error" "error"="no object matching key \"gorush/gorush\" in local store" "controller"="alb-ingress-controller" "request"={"Namespace":"gorush","Name":"gorush"}
I0206 11:08:53.731420 1 loadbalancer.go:191] gorush/gorush: creating LoadBalancer 3407a88c-gorush-gorush-f66a
I0206 11:08:54.372743 1 loadbalancer.go:208] gorush/gorush: LoadBalancer 3407a88c-gorush-gorush-f66a created, ARN: arn:aws:elasticloadbalancing:us-east-2:534***385:loadbalancer/app/3407a88c-gorush-gorush-f66a/d31b461c65a278f0
I0206 11:08:54.516572 1 targetgroup.go:119] gorush/gorush: creating target group 3407a88c-e50bdf2f1d39b9db54c
I0206 11:08:54.627177 1 targetgroup.go:138] gorush/gorush: target group 3407a88c-e50bdf2f1d39b9db54c created: arn:aws:elasticloadbalancing:us-east-2:534***385:targetgroup/3407a88c-e50bdf2f1d39b9db54c/e1fdcb752a01b891
I0206 11:08:54.643263 1 tags.go:43] gorush/gorush: modifying tags { kubernetes.io/service-name: "gorush", kubernetes.io/service-port: "80", ingress.k8s.aws/stack: "gorush/gorush", kubernetes.io/cluster/eksctl-bttrm-eks-production-1: "owned", kubernetes.io/namespace: "gorush", kubernetes.io/ingress-name: "gorush", ingress.k8s.aws/cluster: "eksctl-bttrm-eks-production-1", ingress.k8s.aws/resource: "gorush/gorush-gorush:80"} on arn:aws:elasticloadbalancing:us-east-2:534***385:targetgroup/3407a88c-e50bdf2f1d39b9db54c/e1fdcb752a01b891
I0206 11:08:54.978243 1 targets.go:80] gorush/gorush: Adding targets to arn:aws:elasticloadbalancing:us-east-2:534***385:targetgroup/3407a88c-e50bdf2f1d39b9db54c/e1fdcb752a01b891: i-05fbfa35e00ade9b3:31472, i-08ecf579e0dd8b5a0:31472, i-098d672bbc222d424:31472, i-0712f941381a624b8:31472
I0206 11:08:55.196629 1 listener.go:110] gorush/gorush: creating listener 80
E0206 11:08:55.247492 1 controller.go:217] kubebuilder/controller "msg"="Reconciler error" "error"="failed to reconcile listeners due to failed to reconcile rules due to ListenerNotFound: One or more listeners not found\n\tstatus code: 400, request id: 9adc08a0-d25d-499a-b347-2170489429d7" "controller"="alb-ingress-controller" "request"={"Namespace":"gorush","Name":"gorush"}
I0206 11:08:56.610223 1 rules.go:60] gorush/gorush: creating rule 1 on arn:aws:elasticloadbalancing:us-east-2:534***385:listener/app/3407a88c-gorush-gorush-f66a/d31b461c65a278f0/6248c53e3077174d
I0206 11:08:56.648485 1 rules.go:77] gorush/gorush: rule 1 created with conditions [{ Field: "path-pattern", Values: ["/*"] }]
I0206 11:08:57.174774 1 rules.go:82] gorush/gorush: modifying rule 1 on arn:aws:elasticloadbalancing:us-east-2:534***385:listener/app/3407a88c-gorush-gorush-f66a/d31b461c65a278f0/6248c53e3077174d
I0206 11:08:57.215907 1 rules.go:98] gorush/gorush: rule 1 modified with conditions [{ Field: "path-pattern", Values: ["/*"] }]
[/simterm]
Check the Ingress service:
[simterm]
$ kubectl -n gorush get ingress NAME HOSTS ADDRESS PORTS AGE gorush * internal-3407a88c-gorush-gorush-f66a-***.us-east-2.elb.amazonaws.com 80 3m57s
[/simterm]
Testing the API
Let’s connect to our Bastion hos, install dnsutils, dnsping and the curl packages.
To get a Pod we can use the following command:
[simterm]
$ kubectl -n gorush get pod | grep gorush | cut -d" " -f 1 | tail -1 gorush-59bd9dd4fc-klsbz
[/simterm]
And to connect to the Bastion:
[simterm]
$ kubectl -n gorush exec -ti $(kubectl -n gorush get pod | grep gorush | cut -d" " -f 1 | tail -1) -c bastion bash root@gorush-59bd9dd4fc-klsbz:/#
[/simterm]
Install necessary utils, and check if our push-service is working:
[simterm]
root@gorush-59bd9dd4fc-klsbz:/# apt update && apt -y install dnsutils curl dnsdiag
root@gorush-59bd9dd4fc-klsbz:/# curl dualstack.internal-3407a88c-gorush-gorush-f66a-***.us-east-2.elb.amazonaws.com
{"text":"Welcome to notification server."}
[/simterm]
Looks good so far.
Route53
Let’s check if our Internal ALB is resolved to a Private IP, as it’s type was set to the Internal:
[simterm]
root@gorush-59bd9dd4fc-klsbz:/# dig +short dualstack.internal-3407a88c-gorush-gorush-f66a-***.us-east-2.elb.amazonaws.com 10.0.15.167 10.0.23.138
[/simterm]
Good.
We’d like to use an own URL to access the ALB created above, so let’s go to the AWS Route53, and add a new record pointed to the ALB as usual – via an ALIAS:
And make a GET request from the Bastion to make it running inside of the VPS using its own DNS to resolve the domain to the ALB’s Private IPs:
[simterm]
root@gorush-59bd9dd4fc-klsbz:/# dig push.example.com +short 10.0.23.138 10.0.15.167
[/simterm]
The service:
[simterm]
root@gorush-59bd9dd4fc-klsbz:/# curl push.example.com
{"text":"Welcome to notification server."}
[/simterm]
Done.
