Blog microservices deployed in an Istio-enabled kubernetes cluster

blog-microservices

test

blog microservices deployed in an Istio-enabled kubernetes cluster.

架构

architecture

目录结构

主要遵循 Standard Go Project Layout 推荐的目录分层。

使用的依赖:

Makefile 简介

命令 说明
make init 安装各类 protoc-gen-* 、 wire 以及 migrate
make protoc 基于 *.proto 文件,生成各类 *_pb.go
make wire 基于 wire.go 文件,生成 wire_gen.go
make test 测试
make migrate-up 迁移数据库
make migrate-down 回滚数据库
make blog-server 启动 blog 服务(本地开发环境)
make user-server 启动 user 服务(本地开发环境)
make post-server 启动 post 服务(本地开发环境)
make comment-server 启动 comment 服务(本地开发环境)
make auth-server 启动 auth 服务(本地开发环境)
make dtm-server DTM 为外部依赖,启动本地服务,请浏览 官方文档
make docker-build 构建 Docker 镜像
make kube-deploy 在集群中部署 blog、user、post、comment、auth 以及 dtm 服务
make kube-delete 在集群中删除上述服务
make kube-redeploy 在集群中重新部署服务(数据库服务不删除)

本地环境搭建

  • docker-desktop >= 4.3.2
  • kubernetes >= 1.22.4
  • go >= 1.17
  • istioctl >= 1.12.1
  • protobuf >= 3.19.1

下载安装 Docker Desktop ,并启动内置的 Kubernetes 集群。

# 安装 Go
brew install go
# 安装 Protobuf
brew install protobuf
# 安装 Istio
brew install istioctl
kubectl config use-context docker-desktop
istioctl install -y
kubectl label namespace default istio-injection=enabled

访问服务

推荐使用 BloomRPC 或者 Insomnia

Owner
jxlwqq
Cloud Native and DevOps enthusiast Collaborator of laravel-admin
jxlwqq
Comments
  • gRPC 拦截器顺序的设置

    gRPC 拦截器顺序的设置

    https://github.com/jxlwqq/blog-microservices/commit/b796c75dc634e68cc80766c6d6ff2ea179c94c1c

    各个拦截器的放置顺序就显得非常重要,比如为啥 panic 拦截器要放在第一个位置?如果放在后面的位置,那么假设前面拦截器运行过程中发生了 panic,就无法捕获到异常了。同理,需要对错误进行处理的拦截器,如果放置的位置不合适,获取不到相关的错误,那么该拦截器就无意义了(比如对于客户端的 breaker 熔断 / 超时重试 / 参数检查拦截器的位置,一般而言,是按照先检查输入参数 -> 熔断拦截器 -> 超时重试的顺序,因为熔断需要检查超时错误,但是对于参数校验错误就不关心)

    https://pandaychen.github.io/2019/11/20/GRPC-INTERCEPTOR-APPLY/

  • 关于发送请求,返回 UNAVAILABLE: Trying to connect an http1.x server

    关于发送请求,返回 UNAVAILABLE: Trying to connect an http1.x server

    想问下大家有没有碰到这种情况,所有 pods 和 services 都正常的情况下,使用 bloomrpc 访问服务返回下面这样的错误: { "error": "14 UNAVAILABLE: Trying to connect an http1.x server" } 尝试搜索一些解决方法,但是没能找到

    附上环境: Docker Desktop for Mac Context docker-desktop Kubernetes version v1.22.5 BloomRPC 1.5.3

  • Istio 虚拟服务和 CORS

    Istio 虚拟服务和 CORS

    根据 GitHub 项目中反馈的问题,gRPC网关不直接支持跨源资源共享(Cross-Origin Resource Sharing, CORS)策略。因此,我使用虚拟服务资源的 CorsPolicy 配置将CORS的职责转移给了Istio。这使得CORS比硬编码到服务代码中 (如使用 https://github.com/rs/cors) 更容易管理。

  • linux (debian10) 下部署踩坑 (学习)

    linux (debian10) 下部署踩坑 (学习)

    安装 k8s

    我本来的系统是 archlinux ,但是 手动装 k8s 装不上…… 后来看到 一个脚本

    https://github.com/lework/kainstall 这个 支持 centos ubuntu 和 debian 安装 ,我就 在 本机装了 个 virtualbox ,下载了 debian 的 vdi 虚拟机 文件 装好了 debian 的虚拟机

    然后按照 文档说的 下载 好 离线包 和 sh 开始安装

    $ bash kainstall-debian.sh init \
                               --master 10.168.1.250  \  # 这个master 就 是 k8s 的大脑 (也是我的这台虚拟机)
                               --worker 10.168.1.251,10.168.1.253 \ # 这行可以 去掉 ,因为我电脑内存跑这三个虚拟机 后就没有多少了 ……
                               --user root \ # 操作系统的 用户名(我虚拟机都设置了 root)
                               --password root \ # 操作系统的 密码 (我虚拟机都设置了 root)
                               --port 22 \
                               --offline-file 1.23.1_debian10.tgz  # 这个是 k8s 离线包 因为k8s 下载的话 不翻墙 下不动 ,人家贴心的准备了离线包 
    

    装完后 没有 error 都是info 就是 k8s 安装成功了 看下 node

    $ kubectl get nodes
    NAME               STATUS   ROLES                  AGE   VERSION
    k8s-master-node1   Ready    control-plane,master   8h    v1.23.1
    
    

    为了操作方便 shell 使用 了 zsh , 可以 搭配 k8s 官方的 zsh 框架 进行 补全 https://kubernetes.io/zh/docs/tasks/tools/included/optional-kubectl-configs-zsh/

    (zsh 配置 可以 百度,有很多文章教怎么配)

    配置效果如下 image

    安装 docker buildx

    作者 的 dockerfile 里面 带有 $TARGETPLATFORM TARGETARCH TARGETOS 等参数 ,在debian 里面 跑 make docker-build 会报错 image

    询问作者得知要安装 docker buildx https://docs.docker.com/buildx/working-with-buildx/#manual-download 需要在 https://github.com/docker/buildx/releases/tag/v0.7.1 这里面 下载 对应版本 的 可执行文件 然后 把 他 移动到 $HOME/.docker/cli-plugins 这个目录里面 记得改名叫 docker-buildx 并给 可执行权限 然后 运行 docker buildx install 安装 docker-buildx

    $ docker build --help
    
    Usage:  docker buildx build [OPTIONS] PATH | URL | -
    
    Start a build
    
    可以看到 docker buildx build 就是成功 了 
    
    

    另外 make docker-build 的时候 下载 go 依赖包很慢 需要 给go 加上 国内 源 , image

    当运行 go download 的时候 我这里也会报错 image 可以 把 go download 这一行整行删除 ,因为go build 的时候 也会 下载依赖包的 我这里把 go download 这一行删除 能跑过,不删除 就 报错 ……

    安装 istio

    先在istio 下载界面 下载 压缩包 https://github.com/istio/istio/releases/tag/1.13.0 解压 后 cd 进去

    
    $ cd istio-1.13.0/bin  找到bin 目录 
    
    
    $ ll
    .rwxr-xr-x 95M root 14 Feb 16:28 istioctl
    
    $ cp istioctl /usr/bin/istioctl  把可执行文件 复制到 /usr/bin/istioctl 就可以 全局 使用 了
    
    $ whereis istioctl
    istioctl: /usr/bin/istioctl
    
    

    然后执行 istioctl install --set profile=default -y 安装

    创建持久化存储的一些文件

    作者 的 mysql.yaml 我这里 没有办法 跑 ,user-db post-db 等 pod 起不来 kubectl describe pod user-db-6b8765cdd5-qzzqh 查看 event

     Warning  FailedScheduling  110s                default-scheduler  0/1 nodes are available: 1 persistentvolumeclaim "user-db" not found.
      Warning  FailedScheduling  30s (x3 over 109s)  default-scheduler  0/1 nodes are available: 1 pod has unbound immediate PersistentVolumeClaims.
    

    说却 pvc ,问了作者才知道 他 苹果 的 pv sc 都是配好的…… linux 下 没有默认的 要自己配置 才行

    先创建 sc sc.yaml

    # sc  抽象层的磁盘 类型
    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: local-storage # sc 名字
    provisioner: kubernetes.io/no-provisioner
    volumeBindingMode: WaitForFirstConsumer
    
    

    因为项目有 4 个 mysql.yaml 所以创建 4 个 对应的pv

    以 user-pv.yaml 为例 ,别的都是 metadata.name 换个对应的 名字就行了

    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: user-pv # 这里 对应 comment pvc 的话就 叫 comment-pv 
      labels:
        type: local
    spec:
      storageClassName: local-storage # sc 名称
      capacity:
        storage: 10Gi
      accessModes: # 卷可以被一个节点以读写方式挂载
        - ReadWriteOnce
      local: # 节点上面  挂在 pv 的 路径
        path: "/mnt/data"
    
      nodeAffinity: # 指定 哪一个节点来 放pv 挂载的 路径
        required:
          nodeSelectorTerms:
          - matchExpressions:
            - key: kubernetes.io/hostname
              operator: In
              values:
              - k8s-master-node1 # 这个node 可以根据 kubectl get node 获取 节点名字
    

    接着修改 作者 的 mysql.yaml 加上 两行 分别绑定 sc 和对应 的 pv

    image

    然后 在 执行 make kube-deploy 命令 部署 执行 k get pods -o wide 查看容器 可以 看到 至少有一 个 db 是 跑起来 了…… 别的全失败 了,去看看原因 …… image

    边学习边写 , 没有写完 ,后面 在写 ……

  • feat: dockerfile set go china source

    feat: dockerfile set go china source

    我这边 dockerfile 不给国内 源 下载 很慢 ,甚至会失败 …… 另外问下 这些个变量 是 干什么的? image 我这里 不删除的话 make docker-build 会报 错

    $ make docker-build
    docker build -t blog/blog-server:latest -f ./build/docker/blog/Dockerfile .
    Sending build context to Docker daemon  19.03MB
    Step 1/21 : FROM --platform=$TARGETPLATFORM golang:1.17-alpine as builder
    failed to parse platform : "" is an invalid component of "": platform specifier component must match "^[A-Za-z0-9_-]+$": invalid argument
    make: *** [Makefile:85: docker-build] Error 1
    
    

    第一步都过不去…… 我的环境是 debian 10.0 搭了 k8s 和istio ,另外我在 archlinux 下也是这个表现

TinyHat.Me: Microservices deployed with Kubernetes that enable users to propose hat pictures and try on hats from a user-curated database.
TinyHat.Me: Microservices deployed with Kubernetes that enable users to propose hat pictures and try on hats from a user-curated database.

Click here to see the "buggy" version ?? The Scenario TinyHat.Me is an up and coming startup that provides an API to allow users to try on tiny hats v

Jun 17, 2022
Example golang microservices deployed on kubernetes.
Example golang microservices deployed on kubernetes.

Tech Stack Golang RabbitMQ Docker K8S MongoDB Services There are two services which communicate via http(sync) and rabbitmq(async). Services opened to

Sep 6, 2022
Istio - An open platform to connect, manage, and secure microservices

Istio An open platform to connect, manage, and secure microservices. For in-dept

Jan 5, 2022
Create production ready microservices mono repo pattern wired with Neo4j. Microservices for other languages and front end repos to be added as well in future.
Create production ready microservices mono repo pattern wired with Neo4j. Microservices for other languages and front end repos to be added as well in future.

Create Microservices MonoRepo in GO/Python Create a new production-ready project with backend (Golang), (Python) by running one CLI command. Focus on

Oct 26, 2022
Go-kit-microservices - Example microservices implemented with Go Kit

Go Kit Microservices Example microservices implemented with go kit, a programmin

Jan 18, 2022
A code generator that turns plain old Go services into RPC-enabled (micro)services with robust HTTP APIs.

Frodo is a code generator and runtime library that helps you write RPC-enabled (micro) services and APIs.

Dec 16, 2022
Automatic Service Mesh and RPC generation for Go micro services, it's a humble alternative to gRPC with Istio.
Automatic Service Mesh and RPC generation for Go micro services, it's a humble alternative to gRPC with Istio.

Mesh RPC MeshRPC provides automatic Service Mesh and RPC generation for Go micro services, it's a humble alternative to gRPC with Istio. In a nutshell

Aug 22, 2022
Dubbo2istio watches Dubbo ZooKeeper registry and synchronize all the dubbo services to Istio.
Dubbo2istio watches Dubbo ZooKeeper registry and synchronize all the dubbo services to Istio.

Dubbo2Istio Dubbo2istio 将 Dubbo ZooKeeper 服务注册表中的 Dubbo 服务自动同步到 Istio 服务网格中。 Aeraki 根据 Dubbo 服务信息和用户设置的路由规则生成数据面相关的配置,通过 Istio 下发给数据面 Envoy 中的 Dubbo p

Dec 1, 2022
Todolist microservice with Istio

Todo 服务 环境依赖 docker-desktop >= 4.2.0 kubernetes >= 1.21.5 go >= 1.17 istioctl >= 1.11.4 protobuf >= 3.17.3 grpcurl >= 1.8.5 下载安装 Docker Desktop ,并启动内置

Feb 20, 2022
A db proxy for distributed transaction, read write splitting and sharding! Support any language! It can be deployed as a sidecar in a pod.
A db proxy for distributed transaction, read write splitting and sharding! Support any language! It can be deployed as a sidecar in a pod.

DBPack DBPack means a database cluster tool pack. It can be deployed as a sidecar in a pod, it shields complex basic logic, so that business developme

Dec 29, 2022
🔥 Kubernetes multi-cluster deployment automation service.
🔥  Kubernetes multi-cluster deployment automation service.

Beetle Kubernetes multi-cluster deployment automation service ?? Check out the demo! Application deployment and management should be automated, audita

Dec 11, 2022
A standard library for microservices.

Go kit Go kit is a programming toolkit for building microservices (or elegant monoliths) in Go. We solve common problems in distributed systems and ap

Jan 1, 2023
Zeebe.io - Workflow Engine for Microservices Orchestration

Distributed Workflow Engine for Microservices Orchestration

Jan 2, 2023
Design-based APIs and microservices in Go
Design-based APIs and microservices in Go

Goa is a framework for building micro-services and APIs in Go using a unique design-first approach. Overview Goa takes a different approach to buildin

Jan 5, 2023
goTempM is a full stack Golang microservices sample application built on top of the Micro platform.
goTempM is a full stack Golang microservices sample application built on top of the Micro platform.

goTempM is a full stack Golang microservices sample application built on top of the Micro platform.

Sep 24, 2022
Microservices using Go, RabbitMQ, Docker, WebSocket, PostgreSQL, React
Microservices using Go, RabbitMQ, Docker, WebSocket, PostgreSQL, React

Microservices A basic example of microservice architecture which demonstrates communication between a few loosely coupled services. Written in Go Uses

Jan 1, 2023
Go microservices with REST, and gRPC using BFF pattern.
Go microservices with REST, and gRPC using BFF pattern.

Go microservices with REST, and gRPC using BFF pattern. This repository contains backend services. Everything is dockerized and ready to

Jan 4, 2023
This is an example to demonstrate implementation golang microservices using domain driven design principles and sugestions from go-kit

go-kit DDD Domain Driven Design is prevelent and rising standard for organizing your microservice code. This design architecture emphasis on Code orga

Feb 9, 2022
Box is an incrementally adoptable tool for building scalable, cloud native, microservices.

Box is a tool for building scalable microservices from predefined templates. Box is currently in Beta so if you find any issues or have some ideas

Feb 3, 2022