Using Envoy Proxy to load-balance gRPC services on GKE with header value based Session Affinity

Using Envoy Proxy to load-balance gRPC services on GKE with header value based Session Affinity


This repository contains the code used in the tutorial Using Envoy Proxy to load-balance gRPC services on GKE. This tutorial demonstrates how to perform session affinity based on request header value to gRPC service instances deployed on Google Kubernetes Engine (GKE) via a single external IP address using Network Load Balancing and Envoy Proxy. We use Envoy Proxy in this tutorial to highlight some of the advanced features it provides for gRPC and hash based load balancing.

Links

Display the current project ID

gcloud config list --format 'value(core.project)'

Set project id and region

gcloud config set project <project-id>

REGION=us-central1
ZONE=$REGION-c

GOOGLE_CLOUD_PROJECT=$(gcloud config list --format 'value(core.project)')
CLUSTER_NAME=grpc-cluster-dc

Create Cluster

gcloud container clusters create $CLUSTER_NAME \
--zone $ZONE \
--workload-pool=$GOOGLE_CLOUD_PROJECT.svc.id.goog

# Check nodes
kubectl get nodes -o name
# ...
# node/gke-grpc-cluster-default-pool-c9a3c791-1kpt
# node/gke-grpc-cluster-default-pool-c9a3c791-qn92
# node/gke-grpc-cluster-default-pool-c9a3c791-wf2h



# Create K8S service account
KS_NAMESPACE=default
KSA_NAME=prometheus-to-sd-sa
kubectl create serviceaccount --namespace $KS_NAMESPACE $KSA_NAME

# Attach existing GCP service account to it
gcloud iam service-accounts add-iam-policy-binding --role \
  roles/iam.workloadIdentityUser --member \
  "serviceAccount:$GOOGLE_CLOUD_PROJECT.svc.id.goog[$KS_NAMESPACE/$KSA_NAME]" \
  $GOOGLE_CLOUD_PROJECT@$GOOGLE_CLOUD_PROJECT.iam.gserviceaccount.com

# Annotate the Custom Metrics - Stackdriver Adapter service account:
kubectl annotate serviceaccount --namespace=$KS_NAMESPACE \
  $KSA_NAME \
  iam.gke.io/gcp-service-account=$GOOGLE_CLOUD_PROJECT@$GOOGLE_CLOUD_PROJECT.iam.gserviceaccount.com

# Test workload identity
kubectl run -it \
  --image google/cloud-sdk:slim \
  --serviceaccount $KSA_NAME \
  --namespace $KS_NAMESPACE \
  workload-identity-test

# When prompt appears run
gcloud auth list

# Credentialed Accounts
# ACTIVE  ACCOUNT
# *       [email protected]

Deploy the gRPC services

# Build grpc service image
# You may also use docker locally and push to GCP registry

gcloud builds submit -t gcr.io/$GOOGLE_CLOUD_PROJECT/echo-grpc echo-grpc

gcloud container images list --repository gcr.io/$GOOGLE_CLOUD_PROJECT
# ...
# NAME
# gcr.io/GOOGLE_CLOUD_PROJECT/echo-grpc

# Deploy GRPC service deployment
sed s/GOOGLE_CLOUD_PROJECT/$GOOGLE_CLOUD_PROJECT/ \
    k8s/echo-deployment.yaml | kubectl apply -f -

kubectl get deployments
# ...
# NAME           DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
# echo-grpc      2         2         2            2           1m

# Deploy GRPC k8s service
kubectl apply -f k8s/echo-service.yaml

kubectl get services
# ...
# NAME           TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
# echo-grpc      ClusterIP   None         <none>        8081/TCP   35s

Deploy Custom metrics adapter

kubectl apply -f k8s/metrics-adapter-deployment.yaml

gcloud iam service-accounts add-iam-policy-binding --role \
  roles/iam.workloadIdentityUser --member \
  "serviceAccount:$GOOGLE_CLOUD_PROJECT.svc.id.goog[custom-metrics/custom-metrics-stackdriver-adapter]" \
  $GOOGLE_CLOUD_PROJECT@$GOOGLE_CLOUD_PROJECT.iam.gserviceaccount.com

kubectl annotate serviceaccount --namespace custom-metrics \
  custom-metrics-stackdriver-adapter \
  iam.gke.io/gcp-service-account=$GOOGLE_CLOUD_PROJECT@$GOOGLE_CLOUD_PROJECT.iam.gserviceaccount.com

Set up Network Load Balancing

kubectl apply -f k8s/envoy-service.yaml

# You may watch service deployment - ensure EXTERNAL-IP for the envoy service changes from <pending> to a public IP address:
kubectl get services envoy --watch
# Press Control+C to stop waiting.

Create a self-signed SSL/TLS certificate

# Store Envoy external ip
EXTERNAL_IP=$(kubectl get service envoy -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

# Generate public and private keys
openssl req -x509 -nodes -newkey rsa:2048 -days 365 \
    -keyout privkey.pem -out cert.pem -subj "/CN=$EXTERNAL_IP"

# Upload keys and creates secrets in k8s
kubectl create secret tls envoy-certs \
    --key privkey.pem --cert cert.pem \
    --dry-run -o yaml | kubectl apply -f -

Deploy Envoy

# Deploy Envoy configuration
kubectl apply -f k8s/envoy-configmap.yaml

# Deploy Envoy deployment
kubectl apply -f k8s/envoy-deployment.yaml

kubectl get deployment envoy
# ...
# NAME      DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
# envoy     2         2         2            2           1m

Test the gRPC services

# Install grpcurl
go get github.com/fullstorydev/grpcurl
go install github.com/fullstorydev/grpcurl/cmd/grpcurl

# Make GRPC request to echo-rpc via Envoy
grpcurl.exe \
    -H "x-session-hash: test-header-1" \
    -d '{"content": "With a given header I will always hit the same Pod"}' \
    -proto echo-grpc/api/echo.proto \
    -insecure \
    -v $EXTERNAL_IP:443 api.Echo/Echo
# ...
# Resolved method descriptor:
# rpc Echo ( .api.EchoRequest ) returns ( .api.EchoResponse );

# Request metadata to send:
# x-session-hash: test-header-1

# Response headers received:
# content-type: application/grpc
# date: Fri, 11 Sep 2020 16:08:20 GMT
# hostname: echo-grpc-67458bf84f-87tf8
# server: envoy
# x-envoy-upstream-service-time: 0

# Response contents:
# {
#   "content": "With a given header I will always hit the same Pod"
# }

# Response trailers received:
# (empty)
# Sent 1 request and received 1 response

Envoy Admin

kubectl port-forward \
    $(kubectl get pods -o name | grep envoy | head -n1) 8080:9901
# ...
# Forwarding from 127.0.0.1:8080 -> 8090
# Browse to 127.0.0.1:8080 to see the admin dashboard

Session affinity test

# Ensure grpcurl.exe is in your PATH
./grpc_xxx.sh

# You then see for a given header value, all  calls are routed to the same hostname (Pod)

# ...
# FOR: test---nTfr6ftL
# hostname: echo-grpc-67458bf84f-vbzj5
# hostname: echo-grpc-67458bf84f-vbzj5
# hostname: echo-grpc-67458bf84f-vbzj5
# hostname: echo-grpc-67458bf84f-vbzj5
# hostname: echo-grpc-67458bf84f-vbzj5
# FOR: test---NJpVNN9x
# hostname: echo-grpc-67458bf84f-87tf8
# hostname: echo-grpc-67458bf84f-87tf8
# hostname: echo-grpc-67458bf84f-87tf8
# hostname: echo-grpc-67458bf84f-87tf8
# hostname: echo-grpc-67458bf84f-87tf8
# FOR: test---TE1F7T3L
# hostname: echo-grpc-67458bf84f-rx28f
# hostname: echo-grpc-67458bf84f-rx28f
# hostname: echo-grpc-67458bf84f-rx28f
# hostname: echo-grpc-67458bf84f-rx28f
# hostname: echo-grpc-67458bf84f-rx28f

Remember to trash the cluster when done

Owner
Daniel William Clarke
Software Engineer trying to become the very best, like no one ever was!
Daniel William Clarke
Similar Resources

Go based grpc - grpc gateway micro service example

go-grpc-gateway-server This repository provides an example for go based microservice. Go micro services developed based on gRPC protobuf's and also us

Dec 8, 2021

Grpc-gateway-map-null - gRPC Gateway test using nullable values in map

Demonstrate gRPC gateway behavior with nullable values in maps Using grpc-gatewa

Jan 6, 2022

GRPC - A client-server mockup, using gRPC to expose functionality.

gRPC This is a mockup application that I built to help me visualise and understand the basic concepts of gRPC. In this exchange, the client can use a

Jan 4, 2022

Grpc-train - Train booking demo using gRPC

gRPC Demo: Train Booking Service Description Usage Contributing Development Tool

Feb 6, 2022

Go-grpc-tutorial - Simple gRPC server/client using go

Simple gRPC server/client using go Run server go run usermgmt_server/usermgmt_

Feb 14, 2022

In memory Key/Value store in go using gRPC.

In memory Key/Value store in go using gRPC.

In memory cache, using gRPC Contents About Running Server Local Docker Kubernetes Example Helm Terraform API Add Get GetByPrefix GetAllItems DeleteKey

Dec 26, 2022

Envoy filters in Go

Envoy filters in Go

EGo-Demo This is a demo of how to build a Golang filter for Envoy, based on the Envoy Filter Example project, by using Go's CGo feature. It is still a

Oct 7, 2022

Envoy introspection for golang

Envoy introspection Demo Build the extension (.wasm file): make wasm Start the upstream service: docker run -d -p 3030:80 kennethreitz/httpbin Run th

Nov 30, 2021

gRPC LRU-cache server and client with load test

gRPC k-v storage with LRU-cache server & client + load test. Specify LRU-cache capacity: server/cmd/app.go - StorageCapacity go build ./server/cmd/*

Dec 26, 2021
Related tags
Envoy-eds-server - Envoy EDS server is a working Envoy Discovery Service implementation

envoy-eds-server Intro Envoy EDS server is a working Envoy Discovery Service imp

Apr 2, 2022
Simple grpc web and grpc transcoding with Envoy
Simple grpc web and grpc transcoding with Envoy

gRPC Web and gRPC Transcoding with Envoy This is a simple stand-alone set of con

Dec 25, 2021
Golang Super Simple Load Balance

SSLB (Super Simple Load Balancer) ver 0.1.0 It's a Super Simple Load Balancer, just a little project to achieve some kind of performance. Features Hig

Dec 18, 2022
client on K8s with Istio Load balance

Demo gRPC server/client on K8s with Istio Load balance Prerequisites Acces to k8s cluster Istio installed Deploy make compile make build_client make b

Dec 3, 2021
Json to rpc example with envoy, go, grpc, nats

grpc-nats-envoy json to rpc example with envoy, go, grpc, redis This repo is a mirror of https://github.com/charlesonunze/grpc-redis-envoy-example It

Dec 7, 2021
Let's implement some basic ZeroMQ publisher and subscriber in Golang. Utilize Envoy as a proxy.
Let's implement some basic ZeroMQ publisher and subscriber in Golang. Utilize Envoy as a proxy.

Envy proxy with ZeroMQ Solution tested on DigitalOcean Droplet. In case of re-creation VM follow this article. Introduction Let's implement some basic

Jan 25, 2022
Header Block is a middleware plugin for Traefik to block request and response headers which regex matched by their name and/or value

Header Block is a middleware plugin for Traefik to block request and response headers which regex matched by their name and/or value Conf

May 24, 2022
protoc-gen-grpc-gateway-ts is a Typescript client generator for the grpc-gateway project. It generates idiomatic Typescript clients that connect the web frontend and golang backend fronted by grpc-gateway.

protoc-gen-grpc-gateway-ts protoc-gen-grpc-gateway-ts is a Typescript client generator for the grpc-gateway project. It generates idiomatic Typescript

Dec 19, 2022
Log4Shell is a middleware plugin for Traefik which blocks JNDI attacks based on HTTP header values.

Log4Shell Mitigation Log4Shell is a middleware plugin for Traefik which blocks JNDI attacks based on HTTP header values. Related to the Log4J CVE: htt

Dec 20, 2022
Sagadt - gRPC based micro services POC

Sagadt gRPC based micro service POC development Tools gRPC gRPC gateway buf Prot

Jan 4, 2022