MOSN is a cloud native proxy for edge or service mesh. https://mosn.io

为什么热重启机制需要新老进程 两个 unix domain socket server 通信交互?

目前分为 旧进程 reconfigerServer 新进程 transferServer,

1. reconfigerServer 只是新进程主动判断 isReconfigure() 就结束了连接
2. 迁移 listener fd 以及 connection fd 让旧进程作为 client 角色

如果仅使用 reconfigerServer 作为交互通信会存在哪些问题.

Environment

• SOFAMosn 0.7.0

… context

https://github.com/mosn/mosn/issues/1809

Your PR should present related issues you want to solve.

Solutions

通过路由发生协议切换，需要更新 sub 协议的值，否则无法创建对应链接池的信息

Describe the bug

2个并发，500+请求，rpc请求，用的xprotocol

必现有一个或者多个请求超时，上游服务端已经正常返回消息，但是mosn并没有处理请求。

367e9b96-7751-408a-af4e-f1dd8f74ec61为客户端发送的streamid debug日志发现 [367e9b96-7751-408a-af4e-f1dd8f74ec61] Xprotocol get serverStreamConnectionEventListener 其他正常的请求 [b1a19589-fb67-41e9-a48c-130f632b12a5] Xprotocol get serverStreamConnectionEventListener &{0xc4205b9f90 0xc4204527e0 0xc4200274d0 0xc4200c7920 0xc4204f23c0 0xc420474d80 0xc4203b08c0 0xc420027440 {{0 0} 0 0 0 0} 0xc4203ae780 0xc4203ae640 [0xc420048a00]}

有一个客户端请求conn.serverStreamConnectionEventListener为空导致的问题，mosn认为请求是由upstream响应的请求，只转发了请求，但是并未处理回来的请求。

Concurrency Level: 2 Time taken for tests: 32.080 seconds Complete requests: 500 Failed requests: 1 (Connect: 0, Receive: 0, Length: 1, Exceptions: 0) Write errors: 0 Non-2xx responses: 1 Total transferred: 99769 bytes HTML transferred: 16254 bytes Requests per second: 15.59 [#/sec] (mean) Time per request: 128.322 [ms] (mean) Time per request: 64.161 [ms] (mean, across all concurrent requests) Transfer rate: 3.04 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 1 1.8 0 13 Processing: 6 69 1343.9 9 30059 Waiting: 6 69 1343.9 8 30058 Total: 7 71 1343.9 10 30062 Percentage of the requests served within a certain time (ms) 50% 10 66% 11 75% 12 80% 13 90% 15 95% 17 98% 19 99% 21 100% 30062 (longest request)

Environment

0.5

Your question

我的mosn前面有个负载均衡器，当mosn的端口开始监听的时候，负载均衡器就会认为业务正常，但是实际情况，mosn监听端口不一定代表业务正常。

所以咨询一下mosn是否支持动态设置BindPort，比如我的listenerConfig默认设置bindToPort=false，等业务逻辑ready了（如：健康检查通过了），再主动set bindToPort = true，开始监听端口。

Environment

• MOSN Version

Logs

• Paste the logs you see.

Describe the bug

本地启动了两个端口，分别监听http2和http1端口，配置cluster的两个实例分别对应到这两个端口。mosn转发的端口配置HTTP2协议，开启http2_use_stream配置，期望请求在转发到http1实例失败后成功重试并得到http2端口的响应。

但实际测试时，请求hang住了。我加上日志打点后，发现第一次http2的请求到http1端口后，upstream关闭了连接，然后触发了异步协程写data时的报错，然后 HandleError pkg/module/http2/mhttp2.go L1296 这个函数会关闭downstream的SendData，再次重试时，SendData在 pkg/module/http2/mhttp2.go L1140 读取时失败，goroutine退出，因为第一次的http2 server没有关闭connection，导致waitNotify的请求协程一直等待，直到超时退出。

总结，一共两个问题：

1. 请求失败后直接关闭SendData是否应该？
2. 异步协程退出后是否需要有相应机制保证请求的协程能收到事件？

Issues associated with this PR

Your PR should present related issues you want to solve.

Solutions

You should show your solutions about the issues in your PR, including the overall solutions, details and the changes. At this time, Chinese is allowed to describe these.

UT result

Unit Test is needed if the code is changed, your unit test should cover boundary cases, corner cases, and some exceptional cases. And you need to show the UT result.

Benchmark

If your code involves the processing of every request, you should give the Benchmark Result.

Code Style

• Make sure Goimports has run
• Show Golint result

The New Feature

Describe the new feature you want to support clearly.

Your scenes

Describe your use scenes (why need this feature).

Your advice

Describe the advice or solution of this new feature.

Environment

• MOSN Version

背景

MOSN 基于go grpc server框架提供一个GRPC Server的能力，相比于原生的go grpc server框架，可以获得如下能力：

• 完全复用MOSN Sidecar的部署、升级、运维能力
• 可复用MOSN中通用的基础能力：热升级、Listener Filter、部分Network Filter 等

设计思路

• MOSN的gRPC 能力主要还是基于官方的gRPC 库进行核心能力的实现，并且尽量减少gRPC Server开发者感受到的差异
• 基于NetworkFilter机制进行实现

详细设计

• 首先梳理一下NetworkFilter机制与处理流程
• 在配置解析时，完成gRPC Server的启动，随着MOSN的Listener监听开始提供服务
• 一个连接对应一个NetworkFilter对象
  • InitializeReadFilterCallbacks 和 OnNewConnection 也是在连接创建时调用的接口，负责进行连接初始化的工作
  • OnData是在收到数据以后调用的接口，负责数据的传递
• go gRPC Server库，从“监听”的Listener中Accept一个连接，然后进行读写，而在MOSN框架中，Listener的监听和连接数据的读写都处理过了，这里需要进行一层处理
  • Listener和Conn 都是interface，可以在MOSN的Filter中进行处理以后，再把数据传给gRPC Server，做到gRPC Server无感知
  • 在配置解析时，实现Listener的封装
  • 在InitializeReadFilterCallbacks中实现Conn的封装
  • 在OnNewConnection中将封装的Conn传递给封装的Listener，触发Listener.Accept
  • 在OnData中将读取到的数据传递给封装的Conn，触发Conn.Read
• gRPC Server的实现
  • 在使用官方gRPC 框架实现gRPC Server的时候，开发者需要基于proto文件生成一个.pb.go文件，同时需要实现一组接口满足proto中定义的接口实现，将其注册（Register）到gRPC Server框架中
  • MOSN的gRPC NetworkFilter也需要提供类似的注册能力，让开发者只关注对应gRPC Server实现逻辑，然后注册到MOSN框架中即可
• MOSN GRPC框架要求开发者实现一个函数，该函数返回一个未调用Serve方法的grpc server。框架会使用自定义的Listener去调用Serve方法实现对数据的拦截处理

func init() {
    mgrpc.RegisterServerHandler("mygrpc", MyFunc)
}
func MyFunc(_ json.RawMessage) *grpc.Server {
    s := grpc.NewServer()
    // pb 是.pb.go所在的package路径
    // server 是开发者实现的api接口
    pb.RegisterGreeterServer(s, &server{})
    return s
}

• 预期使用的配置示例

{
        "servers":[
                {
                        "default_log_path":"stdout",
                        "default_log_level":"DEBUG",
                        "listeners":[
                                {
                                        "address":"127.0.0.1:2045",
                                        "bind_port": true,
                                        "filter_chains": [{
                                                "filters": [
                                                        {
                                                                "type":"grpc",
                                                                "config": {
                                                                        "server_name":"mygrpc",
                                                                        "address":"127.0.0.1:2045"
                                                                }
                                                        }
                                                ]
                                        }]
                                }
                        ]
                }
        ]
}

Issues associated with this PR

#1311

Solutions

Update xDS v3 API with Istio 1.7.7

UT result

Unit Test is needed if the code is changed, your unit test should cover boundary cases, corner cases, and some exceptional cases. And you need to show the UT result.

Benchmark

If your code involves the processing of every request, you should give the Benchmark Result.

Code Style

• Make sure Goimports has run
• Show Golint result

add IPAllowlist filter

Issues associated with this PR

Your PR should present related issues you want to solve.

Solutions

You should show your solutions about the issues in your PR, including the overall solutions, details and the changes. At this time, Chinese is allowed to describe these.

UT result

Unit Test is needed if the code is changed, your unit test should cover boundary cases, corner cases, and some exceptional cases. And you need to show the UT result.

Benchmark

If your code involves the processing of every request, you should give the Benchmark Result.

Code Style

• Make sure Goimports has run
• Show Golint result

The New Feature

Does the MOSN have a plugin or filter mechanism to add a tracing/APM agent? SkyWalking has its LUA agent for Nginx, Kong and OpenResty, https://github.com/apache/skywalking-nginx-lua As it has the go agent https://github.com/SkyAPM/go2sky, so if you have the mechanism, we could try to invite our committer, @arugal to work with your team.

WDYT?

Issues associated with this PR

Your PR should present related issues you want to solve.

Solutions

You should show your solutions about the issues in your PR, including the overall solutions, details and the changes. At this time, Chinese is allowed to describe these.

UT result

Unit Test is needed if the code is changed, your unit test should cover boundary cases, corner cases, and some exceptional cases. And you need to show the UT result.

Benchmark

If your code involves the processing of every request, you should give the Benchmark Result.

Code Style

• Make sure Goimports has run
• Show Golint result

Issues associated with this PR

Your PR should present related issues you want to solve.

Solutions

You should show your solutions about the issues in your PR, including the overall solutions, details and the changes. At this time, Chinese is allowed to describe these.

UT result

Unit Test is needed if the code is changed, your unit test should cover boundary cases, corner cases, and some exceptional cases. And you need to show the UT result.

Benchmark

If your code involves the processing of every request, you should give the Benchmark Result.

Code Style

• Make sure Goimports has run
• Show Golint result

Issues associated with this PR

参考 apisix 的 key-auth 功能，https://apisix.apache.org/zh/docs/apisix/plugins/key-auth/

Solutions

match 部分的结构和代码主要参考 jwtauthn，对 envoy/jwt_authn/v3 进行了一定的删减

UT result

Unit Test is needed if the code is changed, your unit test should cover boundary cases, corner cases, and some exceptional cases. And you need to show the UT result.

Benchmark

If your code involves the processing of every request, you should give the Benchmark Result.

Code Style

• Make sure Goimports has run
• Show Golint result

Your question

Debug server init here,

func DefaultInitStage(c *v2.MOSNConfig) {
	InitDefaultPath(c)
	InitDebugServe(c)
	InitializePidFile(c)
	InitializeTracing(c)
	InitializePlugin(c)
	InitializeWasm(c)
	InitializeThirdPartCodec(c)
}

And started here:

func (m *Mosn) inheritConfig(c *v2.MOSNConfig) (err error) {
	m.Config = c
	server.EnableInheritOldMosnconfig(c.InheritOldMosnconfig)

	// default is graceful mode, turn graceful off by set it to false
	if !c.DisableUpgrade && server.IsReconfigure() {
		m.isFromUpgrade = true
		if err = m.inheritHandler(); err != nil {
			return
		}
	}
	log.StartLogger.Infof("[mosn] [NewMosn] new mosn created")
	// start init services
	if err = store.StartService(nil); err != nil {
		log.StartLogger.Errorf("[mosn] [NewMosn] start service failed: %v, exit", err)
	}
	return
}

StartService(nil) means do not inherit old listeners. So if "pprof" enabled, hot upgrade will fail.

Issues associated with this PR

Your PR should present related issues you want to solve.

Solutions

You should show your solutions about the issues in your PR, including the overall solutions, details and the changes. At this time, Chinese is allowed to describe these.

UT result

Unit Test is needed if the code is changed, your unit test should cover boundary cases, corner cases, and some exceptional cases. And you need to show the UT result.

Benchmark

If your code involves the processing of every request, you should give the Benchmark Result.

Code Style

• Make sure Goimports has run
• Show Golint result