A cross-language distributed transaction manager

license Build Status codecov Go Report Card Go Reference Mentioned in Awesome Go

English Docs

跨语言分布式事务管理器

DTM是一款golang开发的分布式事务管理器,解决了跨数据库、跨服务、跨语言栈更新数据的一致性问题。

他优雅的解决了幂等、空补偿、悬挂等分布式事务难题,提供了简单易用、高性能、易水平扩展的解决方案。

作者受邀参加中国数据库大会分享多语言环境下分布式事务实践

谁在使用dtm

Tencent 腾讯

Ivydad 常青藤爸爸

Eglass 视咖镜小二

极欧科技

金数智联

亮点

  • 极易接入
    • 支持HTTP,提供非常简单的接口,极大降低上手分布式事务的难度,新手也能快速接入
  • 使用简单
    • 开发者不再担心悬挂、空补偿、幂等各类问题,框架层代为处理
  • 跨语言
    • 可适合多语言栈的公司使用。方便go、python、php、nodejs、ruby、c# 各类语言使用。
  • 易部署、易扩展
    • 仅依赖mysql,部署简单,易集群化,易水平扩展
  • 多种分布式事务协议支持
    • TCC、SAGA、XA、事务消息

与其他框架对比

目前开源的分布式事务框架,Java的框架较多,有大厂开源的SEATA、ServiceComb-Pack,shardingsphere,以及个人开源的himly,tcc-transaction,ByteTCC等等,其中以Seata的应用最为广泛。

非Java语言类的,暂未看到除dtm之外的成熟框架,因此这里仅将DTM和Java中最成熟的Seata对比:

特性 DTM SEATA 备注
支持语言 Go、Java、python、php、c#... Java dtm可轻松接入一门新语言
异常处理 子事务屏障自动处理 手动处理 dtm解决了幂等、悬挂、空补偿
TCC事务
XA事务
AT事务 建议使用XA AT与XA类似,性能更好,但有脏回滚
SAGA事务 支持并发 状态机模式
二阶段消息 dtm提供类似rocketmq的事务消息
单服务多数据源
通信协议 HTTP、gRPC、go-zero dubbo等协议 dtm对云原生更加友好
star数量 github stars github stars dtm从20210604发布0.1,发展快

从上面对比的特性来看,如果您的语言栈包含了Java之外的语言,那么dtm是您的首选。如果您的语言栈是Java,您也可以选择接入dtm,使用子事务屏障技术,简化您的业务编写。

详细的对比可以点击特性中的链接,跳到相关文档

性能测试报告

教程与文档

各语言客户端及示例

微服务框架支持

  • go-zero:一开源就非常火爆的微服务框架,首家接入dtm的微服务框架。感谢go-zero作者kevwan的大力支持
  • polaris: 腾讯开源的注册发现组件,以及在其上构建的微服务框架。感谢腾讯同学ychensha的PR
  • 其他:看用户需求量,择机接入

具体微服务接入使用,参见微服务支持

快速开始

获取代码

git clone https://github.com/dtm-labs/dtm && cd dtm

dtm依赖于mysql

安装docker 20.04+之后

docker-compose -f helper/compose.mysql.yml up

您也可以配置使用现有的mysql,需要高级权限,允许dtm创建数据库

cp conf.sample.yml conf.yml # 修改conf.yml

启动并运行saga示例

go run app/main.go qs

开始使用

使用

  // 具体业务微服务地址
  const qsBusi = "http://localhost:8081/api/busi_saga"
  req := &gin.H{"amount": 30} // 微服务的载荷
  // DtmServer为DTM服务的地址,是一个url
  DtmServer := "http://localhost:36789/api/dtmsvr"
  saga := dtmcli.NewSaga(DtmServer, dtmcli.MustGenGid(DtmServer)).
    // 添加一个TransOut的子事务,正向操作为url: qsBusi+"/TransOut", 补偿操作为url: qsBusi+"/TransOutCompensate"
    Add(qsBusi+"/TransOut", qsBusi+"/TransOutCompensate", req).
    // 添加一个TransIn的子事务,正向操作为url: qsBusi+"/TransIn", 补偿操作为url: qsBusi+"/TransInCompensate"
    Add(qsBusi+"/TransIn", qsBusi+"/TransInCompensate", req)
  // 提交saga事务,dtm会完成所有的子事务/回滚所有的子事务
  err := saga.Submit()

成功运行后,可以看到TransOut、TransIn依次被调用,完成了整个分布式事务

时序图

上述saga分布式事务的时序图如下:

完整示例

参考examples/quick_start.go

公众号

您可以关注公众号:分布式事务,及时跟踪dtm的最新内容

交流群

请加 yedf2008 好友或者扫码加好友,验证回复 dtm 按照指引进群

yedf2008

欢迎使用dtm,或者通过dtm学习实践分布式事务相关知识,欢迎star支持我们

谁在使用

腾讯 常青藤爸爸 镜小二 极欧科技 金数智联
Owner
DTM Development and Communities
Make distributed transaction easy!
DTM Development and Communities
Comments
  • Clarification on workflow.Interceptor

    Clarification on workflow.Interceptor

    Is it safe to use a grpc client for which I have installed a workflow.Interceptor also outside a workflow definition? Or will all calls to the client be intercepted and cached by the interceptor?

  • saga事务进入无限补偿重试,1个补偿大致每秒重试80次,请问是什么原因?

    saga事务进入无限补偿重试,1个补偿大致每秒重试80次,请问是什么原因?

    问题:saga事务进入无限补偿重试,1个补偿大致每秒重试80次,请问是什么原因? 事务:

    • /goods.service.v1.Goods/StockDeduct商品库存扣减,/goods.service.v1.Goods/StockDeductRevert商品库存扣减补偿
    • /order.service.v1.Order/Create订单创建,/order.service.v1.Order/CreateRevert订单创建补偿

    在任务的server端添加了auth jwt 验证,dtm 未添加对应的 auth header token,在这种情况下,商品库存扣减商品库存扣减补偿都无法成功,dtm进入无限重试商品库存扣减补偿状态。

    文档注明会进入 指数退避算法重试 状态,但实际没有,请问是什么原因呢?文档参考

  • Seeking advice for usage of dtm

    Seeking advice for usage of dtm

    I build a service which connected to multiple MySQLs,and has a logic that will start an XA transaction on these dbs. On the consideration of distributed transaction, should I set up the grpc server for the logic and use DTM to control the XA transaction, or i just need to let my service to be the TM and control the XA transaction?

    Looking forward to suggestions!

  • Customize ErrFailure message

    Customize ErrFailure message

    Currently, if our application needs to stop the ongoing workflow, it must return either a http/grpc failure error or an ErrFailure. However, if we are not calling a remote endpoint, the only available alternative is the ErrFailure error. Is there a way to create custom errors (wich a custom message also) which are treated by dtm as ErrFailure?

  • windows版  http://localhost:36789 无法显示

    windows版 http://localhost:36789 无法显示

    image 配置了redis 和端口 也还是原来默认的 image

    #####################################################################

    dtm can be run without any config.

    all config in this file is optional. the default value is as specified in each line

    all configs can be specified from env. for example:

    MicroService.EndPoint => MICRO_SERVICE_END_POINT

    #####################################################################

    Store: # specify which engine to store trans status

    Driver: 'mysql'

    Host: 'localhost'

    User: 'root'

    Password: ''

    Port: 3306

    Db: 'dtm'

    Driver: 'boltdb' # default store engine

    Driver: 'redis' Host: 'localhost' User: '' Password: '' Port: 6379

    Driver: 'postgres'

    Host: 'localhost'

    User: 'postgres'

    Password: 'mysecretpassword'

    Port: '5432'

    following config is for only Driver postgres/mysql

    MaxOpenConns: 500

    MaxIdleConns: 500

    ConnMaxLifeTime: 5 # default value is 5 (minutes)

    flollowing config is only for some Driver

    DataExpire: 604800 # Trans data will expire in 7 days. only for redis/boltdb.

    FinishedDataExpire: 86400 # finished Trans data will expire in 1 days. only for redis.

    RedisPrefix: '{a}' # default value is '{a}'. Redis storage prefix. store data to only one slot in cluster

    MicroService: # grpc based microservice config

    Driver: 'dtm-driver-gozero' # name of the driver to handle register/discover

    Target: 'etcd://localhost:2379/dtmservice' # register dtm server to this url

    EndPoint: 'localhost:36790'

    HttpMicroService: # http based microservice config

    Driver: 'dtm-driver-http' # name of the driver to handle register/discover

    RegistryType: 'nacos'

    RegistryAddress: '127.0.0.1:8848,127.0.0.1:8848'

    RegistryOptions: '{"UserName":"nacos","Password":"nacos","NotLoadCacheAtStart":true}'

    Target: '{"ServiceName":"dtmService","Enable":true,"Healthy":true,"Weight":10}' # target and options

    EndPoint: '127.0.0.1:36789'

    the unit of following configurations is second

    TransCronInterval: 3 # the interval to poll unfinished global transaction for every dtm process

    TimeoutToFail: 35 # timeout for XA, TCC to fail. saga's timeout default to infinite, which can be overwritten in saga options

    RetryInterval: 10 # the subtrans branch will be retried after this interval

    RequestTimeout: 3 # the timeout of HTTP/gRPC request in dtm

    LogLevel: 'info' # default: info. can be debug|info|warn|error

    Log:

    Outputs: 'stderr' # default: stderr, split by ",", you can append files to Outputs if need. example:'stderr,/tmp/test.log'

    RotationEnable: 0 # default: 0

    RotationConfigJSON: '{}' # example: '{"maxsize": 100, "maxage": 0, "maxbackups": 0, "localtime": false, "compress": false}'

    HttpPort: 36798 GrpcPort: 36799

    JsonRpcPort: 36791

    advanced options

    UpdateBranchAsyncGoroutineNum: 1 # num of async goroutine to update branch status

  • postman grpc  调用 dtm server 导致dtm server 停止

    postman grpc 调用 dtm server 导致dtm server 停止

    dtm version:1.14.4 docker 部署 dtm env:

    STORE_DRIVER: mysql
    STORE_HOST: xxxxx.com
    STORE_USER: root
    STORE_PASSWORD: 'pwd'
    STORE_PORT: 3306
    
    MICRO_SERVICE_DRIVER: dtm-driver-gozero
    MICRO_SERVICE_TARGET: "consul://host.docker.internal:8500/dtm-server"
    MICRO_SERVICE_END_POINT: "host.docker.internal:36790"
    

    postman version:v9.24.2 grpc Request:
    method: Prepare message: postman 自动生成的

        "BinPayloads": [
            "u1NVblZ8IRIrJN==",
            "jaUYjyap9Mhf2h1A3Zbs1WRv",
            "iENGtAdeW4VHxlMcrjqIFHiDaUR"
        ],
        "CustomedData": "consectetur non esse sit",
        "Gid": "incididunt",
        "QueryPrepared": "aliquip eu",
        "ReqExtra": {},
        "RollbackReason": "est esse ut",
        "Steps": "amet proident nisi",
        "TransOptions": {
            "PassthroughHeaders": [
                "magna proident fugiat exercitation",
                "adipisicing voluptate ullamco",
                "amet ut dolor culpa qui"
            ],
            "RequestTimeout": "18686",
            "RetryInterval": "1313",
            "TimeoutToFail": "862415057",
            "WaitResult": false
        },
        "TransType": "in adipisicing ex fugiat aliquip"
    }
    

    dtm 报错信息

    panic: invalid character 'a' looking for beginning of value
    
    goroutine 67 [running]:
    github.com/dtm-labs/dtm/dtmcli/dtmimp.E2P(...)
            /app/dtm/dtmcli/dtmimp/utils.go:62
    github.com/dtm-labs/dtm/dtmcli/dtmimp.MustUnmarshal(...)
            /app/dtm/dtmcli/dtmimp/utils.go:122
    github.com/dtm-labs/dtm/dtmcli/dtmimp.MustUnmarshalString(...)
            /app/dtm/dtmcli/dtmimp/utils.go:127
    github.com/dtm-labs/dtm/dtmsvr.TransFromDtmRequest(0x2781578, 0xc00017bf80, 0xc0003f3d60, 0x20bc3c0)
            /app/dtm/dtmsvr/trans_class.go:113 +0x469
    github.com/dtm-labs/dtm/dtmsvr.(*dtmServer).Prepare(0x369ad80, 0x2781578, 0xc00017bf80, 0xc0003f3d60, 0x369ad80, 0x0, 0x0)
            /app/dtm/dtmsvr/api_grpc.go:33 +0x45
    github.com/dtm-labs/dtm/dtmgrpc/dtmgpb._Dtm_Prepare_Handler.func1(0x2781578, 0xc00017bf80, 0x218b660, 0xc0003f3d60, 0x2, 0x2, 0x1f1fa00, 0xc0009e7988)
            /app/dtm/dtmgrpc/dtmgpb/dtmgimp_grpc.pb.go:179 +0x89
    github.com/dtm-labs/dtm/dtmgrpc/dtmgimp.GrpcServerLog(0x2781578, 0xc00017bf80, 0x218b660, 0xc0003f3d60, 0xc00041d600, 0xc000a7eb28, 0x49ba06, 0x62cf8585, 0x1c06170d, 0x1c9266e5f0a)
            /app/dtm/dtmgrpc/dtmgimp/types.go:27 +0x1d6
    google.golang.org/grpc.chainUnaryInterceptors.func1.1(0x2781578, 0xc00017bf80, 0x218b660, 0xc0003f3d60, 0x203000, 0x0, 0x0, 0x7f2be3976698)
            /go/pkg/mod/google.golang.org/[email protected]/server.go:1117 +0x8c
    github.com/dtm-labs/dtm/dtmsvr.grpcMetrics(0x2781578, 0xc00017bf80, 0x218b660, 0xc0003f3d60, 0xc00041d600, 0xc00053b300, 0x0, 0x0, 0x0, 0x0)
            /app/dtm/dtmsvr/metrics.go:86 +0x182
    google.golang.org/grpc.chainUnaryInterceptors.func1.1(0x2781578, 0xc00017bf80, 0x218b660, 0xc0003f3d60, 0xc0009e7b48, 0x40e398, 0x18, 0x2054a20)
            /go/pkg/mod/google.golang.org/[email protected]/server.go:1120 +0x11c
    google.golang.org/grpc.chainUnaryInterceptors.func1(0x2781578, 0xc00017bf80, 0x218b660, 0xc0003f3d60, 0xc00041d600, 0xc000a7eb28, 0xc000a6abb8, 0x51d766, 0x214e1a0, 0xc00017bf80)
            /go/pkg/mod/google.golang.org/[email protected]/server.go:1122 +0xeb
    github.com/dtm-labs/dtm/dtmgrpc/dtmgpb._Dtm_Prepare_Handler(0x20bc3c0, 0x369ad80, 0x2781578, 0xc00017bf80, 0xc00012d080, 0xc000468400, 0x2781578, 0xc00017bf80, 0xc0001fe5a0, 0x118)
            /app/dtm/dtmgrpc/dtmgpb/dtmgimp_grpc.pb.go:181 +0x150
    google.golang.org/grpc.(*Server).processUnaryRPC(0xc0009cc000, 0x279b5b8, 0xc000601520, 0xc000160480, 0xc000766a20, 0x364bf30, 0x0, 0x0, 0x0)
            /go/pkg/mod/google.golang.org/[email protected]/server.go:1283 +0x544
    google.golang.org/grpc.(*Server).handleStream(0xc0009cc000, 0x279b5b8, 0xc000601520, 0xc000160480, 0x0)
            /go/pkg/mod/google.golang.org/[email protected]/server.go:1620 +0xd0c
    google.golang.org/grpc.(*Server).serveStreams.func1.2(0xc000334f40, 0xc0009cc000, 0x279b5b8, 0xc000601520, 0xc000160480)
            /go/pkg/mod/google.golang.org/[email protected]/server.go:922 +0xab
    created by google.golang.org/grpc.(*Server).serveStreams.func1
            /go/pkg/mod/google.golang.org/[email protected]/server.go:920 +0x1fd
    
  • 可否支持多通信协议混用?

    可否支持多通信协议混用?

    粗略的看了一些示例,配置一组事务时只需指定一个 DTM Server,通过其 URL 区分子事务的通信协议是 HTTP、gRPC 或 JSON-RPC。(补充,用到的 cli 也是不同的) 而在实际使用上,我其实并不太关心与 DTM Server 的通信方式,而是关心每个子事务的通信协议。 我的理想使用方式是配置事务时可以给每个子事务分别指定其通信协议,从而支持多协议混用。

  • Problem when setting config.Store values inside helm chart

    Problem when setting config.Store values inside helm chart

    I was trying to deploy dtm in my kubernetes cluster using the helm chart provided in the repo. However, when I set the following configuration key for pointing to a postgres instance:

    # Default values for dtm.
    # This is a YAML-formatted file.
    # Declare variables to be passed into your templates.
    
    # DTM configuration. Specify content for config.yaml
    # ref: https://github.com/dtm-labs/dtm/blob/main/conf.sample.yml
    configuration: |-
     Store: # specify which engine to store trans status
      Driver: 'postgres' # default store engine
      Host: 'someurl'
      User: 'someuser'
      Password: 'somepassword'
      Port: 5432
      Db: 'dtm'
      Schema: 'public'
    

    I get the following error:

    2022/11/28 15:43:14 fatal error: yaml: unmarshal errors:
    line 7: field Db not found in type config.Store
    line 8: field Schema not found in type config.Store
    

    Values seem exactly the ones provided in the sample config.yml file. Any idea on how to fix this?

  • Query saga state through dtmcli

    Query saga state through dtmcli

    Hi, everyone. As the title suggests, my question is: is there a way to query a saga state through the dtm client? The documentation section for WaitResult says: "There are many possible cases, and it is better for the client to query the status of the global transaction through the query interface of dtm." Can you provide a link to the documentation for the mentioned query API?

  • Saga state

    Saga state

    Hi, all! First of all thank you for the excellent library! I wondered if this framework support persisting the saga state on database to recover from failures.

    Thank you

  • consul服务疑问

    consul服务疑问

    docker是虚拟机,内部搭建consul/dtm服务 虚拟机ip 192.168.38.128 两个服务都是开启并可用的 grpc服务器在本地 192.168.0.112:8001 这个服务是一定可用的

    配置中心 配置 MicroService: Driver: 'dtm-driver-gozero' # Target: 'consul://192.168.38.128:2379/dtmservice' # EndPoint: '192.168.38.128:36790'

    具体代码

    `var dtmServer = "consul://192.168.38.128:2379/dtmservice"

    orderRpcBusiServer, err := l.svcCtx.Config.UserRpcConf.BuildTarget()
    if err != nil {
    	//这个地方没报错
    	return nil, fmt.Errorf("下单异常超时")
    }
    //	orderRpcBusiServer = consul://192.168.38.128:8500/user.rpc 服务绝对可用
    fmt.Println(orderRpcBusiServer)
    
    gid := dtmgrpc.MustGenGid(dtmServer)
    saga := dtmgrpc.NewSagaGrpc(dtmServer, gid).
    	//测试过
    	//consul://192.168.38.128:8500/user.rpc
    	//192.168.38.128:8500
    	Add(orderRpcBusiServer+"/usercenter/QueryUser", orderRpcBusiServer"+"/usercenter/QueryUser", &grpc_.UniversalUserRequest{
    		Pattern: 1,
    		Data:    nil,
    	})
    err = saga.Submit()
    dtmimp.FatalIfError(err)`
    

    最终报错

    {"@timestamp":"2022-06-18T22:00:36.758+08:00","caller":"handler/loghandler.go:174","content":"[HTTP] 503 - POST - /order/quickCreate 127.0.0.1:2510 - PostmanRuntime/7.26.2 - slowcall(slowcall(3000.7ms))","duration":"3000.7ms","level":"slow","span":"8c7601ab80ff29ff","trace":"1de87c93118a14f1ec8cc806c7506795"} {"@timestamp":"2022-06-18T22:00:36.758+08:00","caller":"handler/loghandler.go:199","content":"[HTTP] 503 - POST /order/quickCreate - 127.0.0.1:2510 - PostmanRuntime/7.26.2\nPOST /order/quickCreate HTTP/1.1\r\nHost: 127.0.0.1:8889\r\nAccept: /\r\nAccept-Encoding: gzip, deflate, br\r\nConnection: keep-alive\r\nContent-Length: 55\r\nContent-Type: application/json\r\nPostman-Token: 79c6e169-cc47-4ccf-b244-e6f6b041d165\r\nUser-Agent: PostmanRuntime/7.26.2\r\n\r\n{\r\n "userId": 1,\r\n "goodsId": 1,\r\n "num": 1\r\n}","duration":"3000.7ms","level":"error","span":"8c7601ab80ff29ff","trace":"1de87c93118a14f1ec8cc806c7506795"} {"@timestamp":"2022-06-18T22:00:54.764+08:00","caller":"[email protected]/resolver.go:68","content":"[Consul resolver] Couldn't fetch endpoints. target={service='dtmservice' healthy='false' tag=''}; error={Get "http://192.168.38.128:2379/v1/health/service/dtmservice?near=_agent": dial tcp 192.168.38.128:2379: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.}","level":"error"}

    //这个时候我以为是端口错误, consul://192.168.38.128:8500/dtmservice 更换为这个端口,都更换了 报错 {"level":"error","ts":"2022-06-18T22:03:23.036+0800","caller":"dtmgimp/types.go:46","msg":"grpc client called: consul://192.168.38.128:8500/dtmservice/dtmgimp.Dtm/NewGid {} result: {} err: rpc error: code = Unavailable desc = name resolver error: produced zero addresses","stacktrace":"github.com/dtm-labs/dtmgrpc/dtmgimp.GrpcClientLog\n\tH:/z包/Golang/go1.18.1/pkg/mod/github.com/dtm-labs/[email protected]/dtmgimp/types.go:46\ngoogle.golang.org/grpc.(*ClientConn).Invoke\n\tH:/z包/Golang/go1.18.1/pkg/mod/google.golang.org/[email protected]/call.go:35\ngithub.com/dtm-labs/dtmgrpc/dtmgpb.(*dtmClient).NewGid\n\tH:/z包/Golang/go1.18.1/pkg/mod/github.com/dtm-labs/[email protected]/dtmgpb/dtmgimp_grpc.pb.go:43\ngithub.com/dtm-labs/dtmgrpc.MustGenGid\n\tH:/z包/Golang/go1.18.1/pkg/mod/github.com/dtm-labs/[email protected]/type.go:51\nawesomeProject9/internal/logic.(*CreateLogic).Create\n\tC:/Users/z/go/src/awesomeProject9/internal/logic/createLogic.go:44\nawesomeProject9/internal/handler.createHandler.func1\n\tC:/Users/z/go/src/awesomeProject9/internal/handler/createHandler.go:22\nnet/http.HandlerFunc.ServeHTTP\n\tH:/z包/Golang/go1.18.1/go/src/net/http/server.go:2084\ngithub.com/zeromicro/go-zero/rest/handler.GunzipHandler.func1\n\tH:/z包/Golang/go1.18.1/pkg/mod/github.com/zeromicro/[email protected]/rest/handler/gunziphandler.go:26\nnet/http.HandlerFunc.ServeHTTP\n\tH:/z包/Golang/go1.18.1/go/src/net/http/server.go:2084\ngithub.com/zeromicro/go-zero/rest/handler.MaxBytesHandler.func2.1\n\tH:/z包/Golang/go1.18.1/pkg/mod/github.com/zeromicro/[email protected]/rest/handler/maxbyteshandler.go:24\nnet/http.HandlerFunc.ServeHTTP\n\tH:/z包/Golang/go1.18.1/go/src/net/http/server.go:2084\ngithub.com/zeromicro/go-zero/rest/handler.MetricHandler.func1.1\n\tH:/z包/Golang/go1.18.1/pkg/mod/github.com/zeromicro/[email protected]/rest/handler/metrichandler.go:21\nnet/http.HandlerFunc.ServeHTTP\n\tH:/z包/Golang/go1.18.1/go/src/net/http/server.go:2084\ngithub.com/zeromicro/go-zero/rest/handler.RecoverHandler.func1\n\tH:/z包/Golang/go1.18.1/pkg/mod/github.com/zeromicro/[email protected]/rest/handler/recoverhandler.go:21\nnet/http.HandlerFunc.ServeHTTP\n\tH:/z包/Golang/go1.18.1/go/src/net/http/server.go:2084\ngithub.com/zeromicro/go-zero/rest/handler.(*timeoutHandler).ServeHTTP.func1\n\tH:/z包/Golang/go1.18.1/pkg/mod/github.com/zeromicro/[email protected]/rest/handler/timeouthandler.go:79"}

    dtm服务报错

    {"level":"info","ts":"2022-06-18T14:04:25.458Z","caller":"dtmsvr/trans_status.go:27","msg":"TouchCronTime for: {"ID":0,"create_time":"2022-06-18T08:18:28.28577398Z","update_time":"2022-06-18T14:04:25.457628306Z","gid":"gJTkRAxRbomhKBccRqUcxM","trans_type":"tcc","status":"submitted","protocol":"grpc","next_cron_interval":20,"next_cron_time":"2022-06-18T14:04:45.457628Z","wait_result":true}"} {"level":"error","ts":"2022-06-18T14:04:25.458Z","caller":"dtmsvr/trans_process.go:52","msg":"processInner got error: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup host.docker.internal on 192.168.38.2:53: no such host"","stacktrace":"github.com/dtm-labs/dtm/dtmsvr.(*TransGlobal).processInner.func1\n\t/app/dtm/dtmsvr/trans_process.go:52\ngithub.com/dtm-labs/dtm/dtmsvr.(*TransGlobal).processInner\n\t/app/dtm/dtmsvr/trans_process.go:63\ngithub.com/dtm-labs/dtm/dtmsvr.(*TransGlobal).process\n\t/app/dtm/dtmsvr/trans_process.go:38\ngithub.com/dtm-labs/dtm/dtmsvr.(*TransGlobal).Process\n\t/app/dtm/dtmsvr/trans_process.go:20\ngithub.com/dtm-labs/dtm/dtmsvr.CronTransOnce\n\t/app/dtm/dtmsvr/cron.go:35\ngithub.com/dtm-labs/dtm/dtmsvr.CronExpiredTrans\n\t/app/dtm/dtmsvr/cron.go:42"} {"level":"info","ts":"2022-06-18T14:04:25.459Z","caller":"dtmsvr/cron.go:54","msg":"cron job return a trans: {"ID":0,"create_time":"2022-06-18T10:57:47.975316619Z","update_time":"2022-06-18T14:04:13.858789044Z","gid":"tCPN73GUWBSohXjEuQmpzW","trans_type":"msg","steps":[{"action":"127.0.0.1:8001/usercenter/QueryUser"}],"status":"submitted","protocol":"grpc","next_cron_interval":10,"next_cron_time":"2022-06-18T14:04:35.458458466Z","wait_result":true}"} {"level":"error","ts":"2022-06-18T14:04:25.459Z","caller":"dtmgimp/types.go:46","msg":"grpc client called: 127.0.0.1:8001/usercenter/QueryUser "IAEqensicGFzc1dvcmQiOiIxMjM0NTYiLCJtb2JpbGUiOjEzNTY4NjU0NDIsIndlQ2hhdFRva2VuIjoiIiwibW9kZWwiOiJwNTAiLCJicmFuZCI6IuWNjuS4uiIsImlkZW50aWZpY2F0aW9uQ29kZSI6Ijk4NzY1NDMyMSJ9" result: "" err: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:8001: connect: connection refused"","stacktrace":"github.com/dtm-labs/dtm/dtmgrpc/dtmgimp.GrpcClientLog\n\t/app/dtm/dtmgrpc/dtmgimp/types.go:46\ngithub.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryClient.func1.1.1\n\t/go/pkg/mod/github.com/grpc-ecosystem/[email protected]/chain.go:72\ngithub.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryClient.func1\n\t/go/pkg/mod/github.com/grpc-ecosystem/[email protected]/chain.go:81\ngoogle.golang.org/grpc.(*ClientConn).Invoke\n\t/go/pkg/mod/google.golang.org/[email protected]/call.go:35\ngithub.com/dtm-labs/dtm/dtmsvr.(*TransGlobal).getURLResult\n\t/app/dtm/dtmsvr/trans_status.go:90\ngithub.com/dtm-labs/dtm/dtmsvr.(*TransGlobal).getBranchResult\n\t/app/dtm/dtmsvr/trans_status.go:123\ngithub.com/dtm-labs/dtm/dtmsvr.(*TransGlobal).execBranch\n\t/app/dtm/dtmsvr/trans_status.go:138\ngithub.com/dtm-labs/dtm/dtmsvr.(*transMsgProcessor).ProcessOnce\n\t/app/dtm/dtmsvr/trans_type_msg.go:69\ngithub.com/dtm-labs/dtm/dtmsvr.(*TransGlobal).processInner\n\t/app/dtm/dtmsvr/trans_process.go:62\ngithub.com/dtm-labs/dtm/dtmsvr.(*TransGlobal).process\n\t/app/dtm/dtmsvr/trans_process.go:38\ngithub.com/dtm-labs/dtm/dtmsvr.(*TransGlobal).Process\n\t/app/dtm/dtmsvr/trans_process.go:20\ngithub.com/dtm-labs/dtm/dtmsvr.CronTransOnce\n\t/app/dtm/dtmsvr/cron.go:35\ngithub.com/dtm-labs/dtm/dtmsvr.CronExpiredTrans\n\t/app/dtm/dtmsvr/cron.go:42"} {"level":"info","ts":"2022-06-18T14:04:25.459Z","caller":"dtmsvr/trans_status.go:27","msg":"TouchCronTime for: {"ID":0,"create_time":"2022-06-18T10:57:47.975316619Z","update_time":"2022-06-18T14:04:25.459222335Z","gid":"tCPN73GUWBSohXjEuQmpzW","trans_type":"msg","steps":[{"action":"127.0.0.1:8001/usercenter/QueryUser"}],"status":"submitted","protocol":"grpc","next_cron_interval":20,"next_cron_time":"2022-06-18T14:04:45.459222025Z","wait_result":true}"} {"level":"error","ts":"2022-06-18T14:04:25.459Z","caller":"dtmsvr/trans_process.go:52","msg":"processInner got error: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:8001: connect: connection refused"","stacktrace":"github.com/dtm-labs/dtm/dtmsvr.(*TransGlobal).processInner.func1\n\t/app/dtm/dtmsvr/trans_process.go:52\ngithub.com/dtm-labs/dtm/dtmsvr.(*TransGlobal).processInner\n\t/app/dtm/dtmsvr/trans_process.go:63\ngithub.com/dtm-labs/dtm/dtmsvr.(*TransGlobal).process\n\t/app/dtm/dtmsvr/trans_process.go:38\ngithub.com/dtm-labs/dtm/dtmsvr.(*TransGlobal).Process\n\t/app/dtm/dtmsvr/trans_process.go:20\ngithub.com/dtm-labs/dtm/dtmsvr.CronTransOnce\n\t/app/dtm/dtmsvr/cron.go:35\ngithub.com/dtm-labs/dtm/dtmsvr.CronExpiredTrans\n\t/app/dtm/dtmsvr/cron.go:42"} 这里有个很奇怪的事情 127.0.0.1:8001,但没有地方上传 grpc 服务对应的地址是192.168.0.112:8001

  • Initialization of dtm logger

    Initialization of dtm logger

    Hi, I'm experimenting an issue with dtm logging initialization. Basically, whenever dtm is imported, the logger is initialized as follows:

    func init() {
         logger.Init(os.GetEnv("LOG_LEVEL")
    }
    

    Now, the LOG_LEVEL env var is already being used by other packages in my case (logrus) to initialize other logging middlewares. Since the value I'm using is TRACE, my code panic inside dtm, because it says that TRACE is not a valid value for dtm logger. Can we fix this piece of code to not panic in the case the LOG_LEVEL var contains something not valid? For example, something like:

    func init() {
         level := os.GetEnv("LOG_LEVEL")
         if !isValidDtmLogLevel(level) {
             level = aDefaultLevelValue
         }
         logger.Init(level)
    }
    

    Or we could also handle this inside the logger.Init() method. What do you think?

  • Feature Request:以更直观的 Concurrent 来表达并行

    Feature Request:以更直观的 Concurrent 来表达并行

    现在的并发是如下写法,其中用数字0、1、2……来表达顺序依赖:

    	saga := dtmcli.NewSaga(DtmServer, shortuuid.New()).
    		Add(Busi+"/CanRollback1", Busi+"/CanRollback1Revert", req).
    		Add(Busi+"/CanRollback2", Busi+"/CanRollback2Revert", req).
    		Add(Busi+"/UnRollback1", "", req).
    		Add(Busi+"/UnRollback2", "", req).
    		EnableConcurrent().
    		AddBranchOrder(2, []int{0, 1}). // 指定step 2,需要在0,1完成后执行
    		AddBranchOrder(3, []int{0, 1}) // 指定step 3,需要在0,1完成后执行
    

    期望支持如下表达式,以更直观的 Concurrent 来表达并行:

    	saga := dtmcli.NewSaga(DtmServer, shortuuid.New()).
    		Concurrent(
    			Add(Busi+"/CanRollback1", Busi+"/CanRollback1Revert", req),
    			Add(Busi+"/CanRollback2", Busi+"/CanRollback2Revert", req)
    		).
    		Add(Busi+"/UnRollback1", "", req).
    		Add(Busi+"/UnRollback2", "", req)
    
An open-source, distributed, cloud-native CD (Continuous Delivery) product designed for developersAn open-source, distributed, cloud-native CD (Continuous Delivery) product designed for developers
An open-source, distributed, cloud-native CD (Continuous Delivery) product designed for developersAn open-source, distributed, cloud-native CD (Continuous Delivery) product designed for developers

Developer-oriented Continuous Delivery Product ⁣ English | 简体中文 Table of Contents Zadig Table of Contents What is Zadig Quick start How to use? How to

Oct 19, 2021
Becca - A simple dynamic language for exploring language design

Becca A simple dynamic language for exploring language design What is Becca Becc

Aug 15, 2022
Cross-platform beanstalkd queue server admin console.
Cross-platform beanstalkd queue server admin console.

Overview aurora is a web-based Beanstalkd queue server console written in Go and works on macOS, Linux, and Windows machines. The main idea behind usi

Dec 30, 2022
Fast cross-platform HTTP benchmarking tool written in Go

bombardier bombardier is a HTTP(S) benchmarking tool. It is written in Go programming language and uses excellent fasthttp instead of Go's default htt

Jan 2, 2023
:rocket: Modern cross-platform HTTP load-testing tool written in Go
:rocket: Modern cross-platform HTTP load-testing tool written in Go

English | 中文 Cassowary is a modern HTTP/S, intuitive & cross-platform load testing tool built in Go for developers, testers and sysadmins. Cassowary d

Dec 29, 2022
Build Go Toolchains /w native libs for cross-compilation

gonative Cross compiled Go binaries are not suitable for production applications because code in the standard library relies on Cgo for DNS resolution

Dec 20, 2022
A dead simple, no frills Go cross compile tool

Gox - Simple Go Cross Compilation Gox is a simple, no-frills tool for Go cross compilation that behaves a lot like standard go build. Gox will paralle

Jan 1, 2023
a build tool for Go, with a focus on cross-compiling, packaging and deployment

goxc NOTE: goxc has long been in maintenance mode. Ever since Go1.5 supported simple cross-compilation, this tool lost much of its value. There are st

Dec 9, 2022
A cross platform CLI for Flyte. Written in Golang. Offers an intuitive interface to Flyte https://flytectl.readthedocs.io/en/latest/
A cross platform CLI for Flyte. Written in Golang. Offers an intuitive interface to Flyte https://flytectl.readthedocs.io/en/latest/

FlyteCTL Flyte's official command-line interface Documentation · Contribution Guide FlyteCTL was designed as a portable and lightweight command-line i

Nov 7, 2022
Quickly cross-compile your C code
Quickly cross-compile your C code

WORK IN PROGRESS: Viceroy is a very early work in progress and is subject to breaking changes. It's also subject to not really working all that well y

Aug 19, 2022
Based on the electron Cross-platform Mini browser

Based on the electron Cross-platform Mini browser

May 1, 2022
Defalyzer: A cross-platform software that focuses on defacement analyzing and mirror tracking
Defalyzer: A cross-platform software that focuses on defacement analyzing and mirror tracking

Project Defalyzer is a cross-platform software that focuses on defacement analyz

Jul 26, 2022
Go Version Manager

gvm By Josh Bussdieker (jbuss, jaja, jbussdieker) while working at Moovweb Currently lovingly maintained by Benjamin Knigge Pull requests and other an

Jan 2, 2023
A simple and powerful SSH keys manager
A simple and powerful SSH keys manager

SKM is a simple and powerful SSH Keys Manager. It helps you to manage your multiple SSH keys easily! Features Create, List, Delete your SSH key(s) Man

Dec 17, 2022
Go version manager. Super simple tool to install and manage Go versions. Install go without root. Gobrew doesn't require shell rehash.

gobrew Go version manager Install or update With curl $ curl -sLk https://git.io/gobrew | sh - or with go $ go get -u github.com/kevincobain2000/gobre

Jan 5, 2023
Kubernetes Lazy User Manager

klum - Kubernetes Lazy User Manager klum does the following basic tasks: Create/Delete/Modify users Easily manage roles associated with users Issues k

Dec 6, 2022
network-node-manager is a kubernetes controller that controls the network configuration of a node to resolve network issues of kubernetes.
network-node-manager is a kubernetes controller that controls the network configuration of a node to resolve network issues of kubernetes.

Network Node Manager network-node-manager is a kubernetes controller that controls the network configuration of a node to resolve network issues of ku

Dec 18, 2022
The smart virtual machines manager. A modern CLI for Vagrant Boxes.
The smart virtual machines manager.  A modern CLI for Vagrant Boxes.

The smart virtual machines manager Table of Contents: What is Vermin Install Vermin Usage Contributors TODO What is Vermin Vermin is a smart, simple a

Dec 22, 2022
Grafana Dashboard Manager

Grafana dash-n-grab Grafana Dash-n-Grab (GDG) -- Dashboard/DataSource Manager. The purpose of this project is to provide an easy to use CLI to interac

Dec 31, 2022