I’ve spent too much time on this problem, and I hope you don’t have to as well.
When you create an Ingress configuration on GKE, it automatically launches a Layer 7 load balancer with all the bells and whistles. It’s also easy to enable HTTPS and even use their own managed certificate service. All is well until you understand that it only supports external load balancers and, setting the allowed source addresses is not trivial (and not supported at the time of writing).
One recommendation that came up a few times is setting the serivce type to LoadBalancer, with an “Internal” annotation, istead of Ingress. There is a lot of conflicting information, and the official GKE documentation is not very helpful. Are these the only options?
- HTTPS with an External Load Balancer
- Internal Load Balancer with no SSL
The answer is, of course, no. There is a way to achieve it, and in hindsight it’s pretty straight forward.
My initial confusion started when I messed around with a few Helm Charts that all had an Ingress option built in them. Thanks for the great abstraction, I didn’t think about the Ingress controller itself and assumed that the all-mighty GKE service would take care of it. Once I stopped to think, the obvious answer was to run “my own” controller.
A few minutes later, and I had my service published to my internal network with a valid SSL certificate.
Using an Nginx Ingress Controller with an Internal Load Balancer
To save some time, for this post, I used the stable/nginx-ingress Chart. All you have to add to the values.yaml file is the load-balancer-type annotation:
controller: service: annotations: cloud.google.com/load-balancer-type: "Internal"
Create a static internal IP address in gcp using the command (where your-internal-lb-address is a nickname of your choosing):
gcloud compute addresses create your-internal-lb-address \ --region us-east2 --subnet your-subnet
Then, get the reserved address:
gcloud compute addresses describe your-internal-lb-address
And use it in the loadBalancerIP attribute in values.yaml
rbac.create: true controller.publishService.enabled: true controller: service: loadBalancerIP: 172.16.2.100 loadBalancerSourceRanges: - 10.0.30.0/24 annotations: cloud.google.com/load-balancer-type: "Internal"
Installing the chart
helm install --name nginx-ingress stable/nginx-ingress -f values.yaml
The next thing that you have to do is make sure you are using the correct ingress controller in your application’s Ingress config.
annotations: kubernetes.io/ingress.class: nginx
You will also have to create the secret that will contain the SSL certificate:
kubectl create secret tls my-certificate-secret --cert mycert.crt --key mycert.key
Then specify it in the Ingress settings of the app:
apiVersion: extensions/v1beta1 kind: Ingress metadata: namespace: default labels: "app.kubernetes.io/name": 'myapp' annotations: kubernetes.io/ingress.class: nginx name: myapp spec: rules: - http: paths: - backend: serviceName: myapp servicePort: 8080 host: "myapp.mydomain.com" tls: - hosts: - myapp.mydomain.com secretName: my-certificate-secret
Once everything is set up, you should be able to access the app through the internal load balancer with https enabled.
➜ myapp git:(master) ✗ kubectl get ingress NAME HOSTS ADDRESS PORTS AGE myapp myapp.mydomain.com 80, 443 3d18h ➜ myapp git:(master) ✗ kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.12.0.1 <none> 443/TCP 5d23h nginx-ingress-controller LoadBalancer 10.12.7.29 172:16.2.100 80:32103/TCP,443:30799/TCP 3d19h nginx-ingress-default-backend ClusterIP 10.12.2.22 <none> 80/TCP 3d19h