Automatically create global & local Rate Limit in Istio, support EnvoyFilter versioning!

istio-ratelimit-operator

Istio ratelimit operator provide an easy way to configure Global or Local Ratelimit in Istio mesh. Istio ratelimit operator also support EnvoyFilter versioning!

Version: 1.0.0 Type: application AppVersion: 1.0.0 made with Go Github master branch build GitHub issues GitHub pull requests

Installing

To install the chart with the release name my-release:

helm repo add zufardhiyaulhaq https://charts.zufardhiyaulhaq.com/
helm install my-release zufardhiyaulhaq/istio-ratelimit-operator --values values.yaml

Usage

  1. Apply Global ratelimit example
kubectl apply -f examples/global/
  1. Check Object
kubectl get GlobalRateLimitConfig
kubectl get GlobalRateLimit
  1. Check EnvoyFilter
kubectl get envoyfilter
NAME                                            AGE
helloworld-zufardhiyaulhaq-dev-1.8              9m58s
helloworld-zufardhiyaulhaq-dev-1.9              9m54s
public-gateway-1.8                              14m
public-gateway-1.9                              14m

Values

Key Type Default Description
operator.image string "zufardhiyaulhaq/istio-ratelimit-operator"
operator.replica int 1
operator.tag string "v1.0.0"
resources.limits.cpu string "200m"
resources.limits.memory string "100Mi"
resources.requests.cpu string "100m"
resources.requests.memory string "20Mi"
Owner
Zufar Dhiyaulhaq
Gojek Engineering, want to chat? https://calendly.com/zufardhiyaulhaq
Zufar Dhiyaulhaq
Comments
  • (wip) shadow mode feature

    (wip) shadow mode feature

    Summary

    This PR implements the usage of shadow_mode of envoyproxy ratelimit service inside this operator, this is implemented in two ways, the first, enabling the global shadow_mode inside the RateLimitService manifest, so the deployment will include the SHADOW_MODE environment variable for this service as described in the official documentation and the second one enabling the shadow_mode per rule.

    Type of Change

    This PR fixes/implements the following bugs/features:

    • shadow_mode usage inside istio-ratelimit-operator

    How has this been tested?

    Proof: For the global shadow_mode: RateLimitService manifest

    ---
    apiVersion: ratelimit.zufardhiyaulhaq.com/v1alpha1
    kind: RateLimitService
    metadata:
      name: istio-test-ratelimit-service
      namespace: istio-system
    spec:
      kubernetes:
        shadow_mode: true
        replica_count: 2
        auto_scaling:
          max_replicas: 3
          min_replicas: 2
        resources:
          limits:
            cpu: "256m"
            memory: "256Mi"
          requests:
            cpu: "128m"
            memory: "128Mi"
      backend:
        redis:
          type: "single"
          url: "redis.istio-system.svc.cluster.local:6379"
    

    After the deployment:

    Checklist:

    • [ ] My code follows the code style of this project.
    • [ ] My change requires a change to the documentation.
      • [ ] I have updated the documentation accordingly.
    • [ ] I have written new tests for my changes.
      • [ ] My changes successfully ran and pass tests locally.

    Closes #18

  • Configurable RateLimitService image and tag

    Configurable RateLimitService image and tag

    Summary

    This PR includes two new properties for RateLimitService Spec inside Kubernetes object Image and ImageTag, this is necessary if we need to adjust the image of the RateLimitService that will be deployed by the RateLimitController

    Type of Change

    This PR fixes/implements the following bugs/features:

    • Configurable Image and ImageTag for RateLimitService

    Test

    ratelimitservice manifest file:

    ---
    apiVersion: ratelimit.zufardhiyaulhaq.com/v1alpha1
    kind: RateLimitService
    metadata:
      name: istio-test-ratelimit-service
      namespace: istio-system
    spec:
      kubernetes:
        image: envoyproxy/ratelimit
        imageTag: 7b60ac79
        replica_count: 2
        auto_scaling:
          max_replicas: 3
          min_replicas: 2
        resources:
          limits:
            cpu: "256m"
            memory: "256Mi"
          requests:
            cpu: "128m"
            memory: "128Mi"
      backend:
        redis:
          type: "single"
          url: "redis.istio-system.svc.cluster.local:6379"
    

    After applying the RatelimitService/GlobalRatelimitConfig/GlobalRatelimit the deployment of the service: Screen Shot 2022-09-06 at 11 52 40 AM

    Closes #15

  • Adding support for descriptor_value for request_headers method

    Adding support for descriptor_value for request_headers method

    Issue : https://github.com/zufardhiyaulhaq/istio-ratelimit-operator/issues/16

    Summary

    We need to add suppport for descriptor_value for request_headers method

    Type of Change

    Adding funcunallity

    How has this been tested?

    Deployed rate limit to my cluster with the following configuation. image

    image

    Checklist:

    • [x] My code follows the code style of this project.
    • [ ] My change requires a change to the documentation.
    • [ ] I have updated the documentation accordingly.
    • [x] I have written new tests for my changes.
    • [x ] My changes successfully ran and pass tests locally.
  • [FEATURE] Configure rate limit service image location

    [FEATURE] Configure rate limit service image location

    I was looking at the RateLimitService CRD and it looks like you've hardcoded the rate limit service image to be pulled from zufardhiyaulhaq/ratelimit:v1.0.0 I did try specifying the container just to test out my theory

         "container" = {
              image             = "${MY_AWS_ACCOUNT_ID}.dkr.ecr.us-east-1.amazonaws.com/envoyproxy-ratelimit:8d6488ea"
              image_pull_policy = "Always"
              command           = ["/bin/ratelimit"]
            }
    
    

    Can you verify that my assessment is correct or I've missed something. Thanks

  • added support for redis auth

    added support for redis auth

    IMPORTANT: Please do not create a Pull Request without creating an issue first!

    (Any change needs to be discussed before proceeding. Failure to do so may result in the rejection of the pull request).

    Summary

    Added support for REDIS_AUTH

    Type of Change

    • Added support for REDIS_AUTH

    How has this been tested?

    Proof:

    Checklist:

    • [x] My code follows the code style of this project.
    • [x] My change requires a change to the documentation.
      • [x] I have updated the documentation accordingly.
    • [ ] I have written new tests for my changes.
      • [x] My changes successfully ran and pass tests locally.
  • add custom environment in ratelimitservice object

    add custom environment in ratelimitservice object

    Signed-off-by: zufardhiyaulhaq [email protected]

    Summary

    Ability to add a custom environment variable to the RateLimitService object.

    Checklist:

    • [x] My code follows the code style of this project.
    • [x] My change requires a change to the documentation.
      • [x] I have updated the documentation accordingly.
    • [x] I have written new tests for my changes.
      • [x] My changes successfully ran and pass tests locally.
  • [FEATURE] shadow_mode support

    [FEATURE] shadow_mode support

    Any ideas or possibilities in the roadmap to support the latest release of ratelimit by the original repository of envoy proxy? Nowadays the HEAD git repository has the support of shadow_mode, this could be handy to include the ratelimit rules but without enforcing, only checking if the ratelimit will be aggressive, generate some kind of impact in the production environment If there is some kind of possible test that I could do, I would be glad to help

  • [FEATURE] Support for rate limit for specific path

    [FEATURE] Support for rate limit for specific path

    Is your feature request related to a problem? Please describe. We need to provide ratelimit by path or method (put for example) on the application and not for the entire application.

    Describe the solution you'd like my config should look like the following example:

    config.yaml: |
        domain: http-bin-server-ratelimit-config
        descriptors:
        - key: path
          value: /ip
          rate_limit:
            unit: hour
            requests_per_unit: 1
    

    Thank you

  • Extend global rate limit for sidecar

    Extend global rate limit for sidecar

    Signed-off-by: zufardhiyaulhaq [email protected]

    Summary

    Extend global rate limit for sidecar

    Type of Change

    • fixing a bug on builder & factory
    • extend global rate limit for sidecar

    How has this been tested?

    ratelimit $ k get globalratelimitconfig,globalratelimit,ratelimitservice
    NAME                                                             AGE
    globalratelimitconfig.ratelimit.zufardhiyaulhaq.com/echo-redis   100m
    
    NAME                                                                 AGE
    globalratelimit.ratelimit.zufardhiyaulhaq.com/echo-redis-http-8080   100m
    globalratelimit.ratelimit.zufardhiyaulhaq.com/echo-redis-http-8081   100m
    
    NAME                                                                          AGE
    ratelimitservice.ratelimit.zufardhiyaulhaq.com/echo-redis-ratelimit-service   100m
    
    root@examples-helloworld-v1-655db4f97b-jhcjv:/opt/microservices# curl echo-redis:8080/redis/hi -v
    * Hostname was NOT found in DNS cache
    *   Trying 10.32.185.200...
    * Connected to echo-redis (10.32.185.200) port 8080 (#0)
    > GET /redis/hi HTTP/1.1
    > User-Agent: curl/7.38.0
    > Host: echo-redis:8080
    > Accept: */*
    > 
    < HTTP/1.1 429 Too Many Requests
    < x-envoy-ratelimited: true
    < date: Sun, 31 Oct 2021 08:03:15 GMT
    * Server envoy is not blacklisted
    < server: envoy
    < content-length: 0
    < x-envoy-upstream-service-time: 4
    < 
    * Connection #0 to host echo-redis left intact
    

    Checklist:

    • [x] My code follows the code style of this project.
    • [x] My change requires a change to the documentation.
      • [x] I have updated the documentation accordingly.
    • [x] I have written new tests for my changes.
      • [x] My changes successfully ran and pass tests locally.
  • support Istio 1.16

    support Istio 1.16

    Signed-off-by: zufardhiyaulhaq [email protected]

    Summary

    support Istio 1.16

    Checklist:

    • [x] My code follows the code style of this project.
    • [x] My change requires a change to the documentation.
      • [x] I have updated the documentation accordingly.
    • [ ] I have written new tests for my changes.
      • [ ] My changes successfully ran and pass tests locally.
  • Generate statsd-exporter config as configmap

    Generate statsd-exporter config as configmap

    Signed-off-by: zufardhiyaulhaq [email protected]

    Summary

    Currently, all metrics from envoy/ratelimit are based on statsd. It's pretty hard to manage. Luckily, the community has written statsd-exporter to change statsd metrics to Prometheus metrics to comply with industry standards. This MR aim to generate statsd-exporter configuration as configmap.

    Type of Change

    This PR fixes/implements the following bugs/features:

    • generate statsd-exporter configmap

    https://github.com/zufardhiyaulhaq/istio-ratelimit-operator/issues/26

    How has this been tested?

    Proof:

    image

    Checklist:

    • [x] My code follows the code style of this project.
    • [x] My change requires a change to the documentation.
      • [x] I have updated the documentation accordingly.
    • [ ] I have written new tests for my changes.
      • [x] My changes successfully ran and pass tests locally.
  • [BUG] RateLimitService environment not sync

    [BUG] RateLimitService environment not sync

    Describe the bug env not updated automatically

    To Reproduce Create RateLimitService with statsd disable, and then update the object to enable the statsd

    Expected behavior statsd should enable automatically with updated environment variable

Topology-tester - Application to easily test microservice topologies and distributed tracing including K8s and Istio

Topology Tester The Topology Tester app allows you to quickly build a dynamic mi

Jan 14, 2022
LazyXds enables Istio only push needed xDS to sidecars to reduce resource consumption and speed up xDS configuration propagation.
LazyXds enables Istio only push needed xDS to sidecars to reduce resource consumption and speed up xDS configuration propagation.

LazyXds LazyXds enables Istio only push needed xDS to sidecars to reduce resource consumption and speed up xDS configuration propagation. Problems to

Dec 28, 2022
Watchtower for Git: automatically keep local Git repositories up to date with their remotes

CrowsNest Watchtower for Git: automatically keep local Git repositories up to date with their remotes. Configuration Flags --run-once or -r: Normally

Oct 30, 2022
Shared counter (with max limit) for k6 load testing tool

xk6-increment This is a k6 extension using the xk6 system. ❗ This is a proof of concept, isn't supported by the k6 team, and may break in the future.

Nov 30, 2021
kubernetes Display Resource (CPU/Memory/Gpu/PodCount) Usage and Request and Limit.
kubernetes Display Resource (CPU/Memory/Gpu/PodCount) Usage and Request and Limit.

kubectl resource-view A plugin to access Kubernetes resource requests, limits, and usage. Display Resource (CPU/Memory/Gpu/PodCount) Usage and Request

Apr 22, 2022
Local Storage is one of HwameiStor components. It will provision the local LVM volume.
Local Storage is one of HwameiStor components. It will provision the local LVM volume.

Local Storage Module English | Simplified_Chinese Introduction Local Storage is one of modules of HwameiStor which is a cloud native local storage sys

Aug 6, 2022
Create AWS Auto Scaling groups from running instances or AMIs automatically.

AWS Auto Scaling Groups Builder AWS Auto Scaling group is a great way of managing Amazon EC2 instances. AWS Auto Scaling group watches the correspondi

Oct 7, 2022
Write personal metadata to a global location

makemine Write user information into a global location for desktop linux computers. Desktop owner information is often baked into parts of the desktop

Dec 4, 2021
API for global CO2 measurements, powered by the Ribbit Network

Ribbit Network API (WIP) This repository contains the backend services supportin

Dec 22, 2022
GitHub Rate Limits Prometheus exporter. Works with both App and PAT credentials
GitHub Rate Limits Prometheus exporter. Works with both App and PAT credentials

Github Rate Limit Prometheus Exporter A prometheus exporter which scrapes GitHub API for the rate limits used by PAT/GitHub App. Helm Chart with value

Sep 19, 2022
Monitoring changes in the source file and automatically compile and run (restart).
Monitoring changes in the source file and automatically compile and run (restart).

dogo Monitoring changes in the source file and automatically compile and run (restart). 中文 Install go get github.com/liudng/dogo Create config Here's

Dec 28, 2022
k8s-image-swapper Mirror images into your own registry and swap image references automatically.
k8s-image-swapper Mirror images into your own registry and swap image references automatically.

k8s-image-swapper Mirror images into your own registry and swap image references automatically. k8s-image-swapper is a mutating webhook for Kubernetes

Dec 27, 2022
Automatically deploy from GitHub to Replit, lightning fast ⚡️

repl.deploy Automatically deploy from GitHub to Replit, lightning fast ⚡️ repl.deploy is split into A GitHub app, which listens for code changes and s

Dec 22, 2022
Annotated and kubez-autoscaler-controller will maintain the HPA automatically for kubernetes resources.

Kubez-autoscaler Overview kubez-autoscaler 通过为 deployment / statefulset 添加 annotations 的方式,自动维护对应 HorizontalPodAutoscaler 的生命周期. Prerequisites 在 kuber

Jan 2, 2023
A Kubernetes Mutating Webhook to automatically re-point pod images to mirrors

kubernetes-mimic Kubernetes Mimic is a Mutating Webhook that will watch for pod creation and update events in a Kubernetes cluster and automatically a

Nov 22, 2022
Cheiron is a Kubernetes Operator made with OperatorSDK for reconciling service account and attaching imagePullSecrets to service accounts automatically

anny-co/cheiron NOTE: Cheiron is currently in very early stages of development and and far from anything usable. Feel free to contribute if you want t

Sep 13, 2021
A Kubernetes CSI plugin to automatically mount SPIFFE certificates to Pods using ephemeral volumes
A Kubernetes CSI plugin to automatically mount SPIFFE certificates to Pods using ephemeral volumes

csi-driver-spiffe csi-driver-spiffe is a Container Storage Interface (CSI) driver plugin for Kubernetes to work along cert-manager. This CSI driver tr

Dec 1, 2022
Automatically capture all potentially useful information about each executed command (as well as its output) and get powerful querying mechanism
Automatically capture all potentially useful information about each executed command (as well as its output) and get powerful querying mechanism

nhi is a revolutionary tool which automatically captures all potentially useful information about each executed command and everything around, and delivers powerful querying mechanism.

Nov 29, 2022
Automatically stop GCE instances
Automatically stop GCE instances

auto-stop-gce-instances Architecture How to deploy 1. Deploy Pub/Sub $ gcloud pubsub topics create $PUBSUB_NAME 2. Deploy Cloud Functions $ gcloud fu

Nov 28, 2021