Go Micro is a framework for distributed systems development

Go Micro License Go.Dev reference Travis CI Go Report Card

Go Micro is a framework for distributed systems development.

Overview

Go Micro provides the core requirements for distributed systems development including RPC and Event driven communication. The Micro philosophy is sane defaults with a pluggable architecture. We provide defaults to get you started quickly but everything can be easily swapped out.

Features

Go Micro abstracts away the details of distributed systems. Here are the main features.

  • Authentication - Auth is built in as a first class citizen. Authentication and authorization enable secure zero trust networking by providing every service an identity and certificates. This additionally includes rule based access control.

  • Dynamic Config - Load and hot reload dynamic config from anywhere. The config interface provides a way to load application level config from any source such as env vars, file, etcd. You can merge the sources and even define fallbacks.

  • Data Storage - A simple data store interface to read, write and delete records. It includes support for memory, file and CockroachDB by default. State and persistence becomes a core requirement beyond prototyping and Micro looks to build that into the framework.

  • Service Discovery - Automatic service registration and name resolution. Service discovery is at the core of micro service development. When service A needs to speak to service B it needs the location of that service. The default discovery mechanism is multicast DNS (mdns), a zeroconf system.

  • Load Balancing - Client side load balancing built on service discovery. Once we have the addresses of any number of instances of a service we now need a way to decide which node to route to. We use random hashed load balancing to provide even distribution across the services and retry a different node if there's a problem.

  • Message Encoding - Dynamic message encoding based on content-type. The client and server will use codecs along with content-type to seamlessly encode and decode Go types for you. Any variety of messages could be encoded and sent from different clients. The client and server handle this by default. This includes protobuf and json by default.

  • RPC Client/Server - RPC based request/response with support for bidirectional streaming. We provide an abstraction for synchronous communication. A request made to a service will be automatically resolved, load balanced, dialled and streamed.

  • Async Messaging - PubSub is built in as a first class citizen for asynchronous communication and event driven architectures. Event notifications are a core pattern in micro service development. The default messaging system is a HTTP event message broker.

  • Synchronization - Distributed systems are often built in an eventually consistent manner. Support for distributed locking and leadership are built in as a Sync interface. When using an eventually consistent database or scheduling use the Sync interface.

  • Pluggable Interfaces - Go Micro makes use of Go interfaces for each distributed system abstraction. Because of this these interfaces are pluggable and allows Go Micro to be runtime agnostic. You can plugin any underlying technology.

Getting Started

To make use of Go Micro

import "github.com/asim/go-micro/v3"

// create a new service
service := micro.NewService(
    micro.Name("helloworld"),
)

// initialise flags
service.Init()

// start the service
service.Run()

See the docs for detailed information on the architecture, installation and use of go-micro.

Code Generation

See cmd/protoc-gen-micro for protobuf code generation.

Example Usage

See examples directory for usage examples.

Plugins

See plugins directory for all the plugins.

License

Go Micro is Apache 2.0 licensed.

Upgrade

Micro v3 consolidates Go Micro into it as a single unified platform. Please see the upgrade guide.

Owner
Asim Aslam
Working on Micro
Asim Aslam
https://github.com/micro/go-micro https://go-micro.dev
Comments
  • example of compose two or more services with single binary

    example of compose two or more services with single binary

    I like go-micro design, but if i want to provide single binary with all my services together. For example that implements some api with 10 different services. How can i do that with go-micro? All this services need to communicate via message bus (nats or etcd).

  • RPC stream client/server mutex fix

    RPC stream client/server mutex fix

    As described in #878 (and #532) it is not possible to send messages while waiting to receive them as well (or vice-versa), due to RPC stream client and server being locked by both Send() and Recv() functions.

    In order to solve this issue, I unlock the Mutex before calling the codec to receive data, and locking it again after codec returned. This way we still have the Mutex protection of the client/server structs, but we can send/receive at same time.

    :warning: Note that this assumes that codec does not need protection while being active.

    I did not update Send() because I assumed that it won't block. By only fixing Recv() the Send() worked as welll. Maybe depending on transport/codec it can block as well?

    I only considered these 2 components/files, as we are (currently) not having issues in other parts of the go-micro framework regarding to this. If needed I can update other parts as well to stay in sync with this change.

    I have tested this in our micro-service environment and it all seems to work fine. I did not trigger race conditions or other panics. But I did not yet perform (high) load or long running tests, which would make race conditions more likely to pop up (if there are any).

  • build command-line-arguments: cannot load crypto/ed25519: cannot find module providing package crypto/ed2551

    build command-line-arguments: cannot load crypto/ed25519: cannot find module providing package crypto/ed2551

    When I run the go-micro,it tips build command-line-arguments: cannot load crypto/ed25519: cannot find module providing package crypto/ed25519,but i find the module is exists in gopath.

  • consul registry broken: mismatch in type due to import being expected from

    consul registry broken: mismatch in type due to import being expected from "github.com/hashicorp/consul/vendor"

    My Go version is 1.12.5.

    I'm building my package binary using the golang1.12.5-alpine3.9 Docker image as a package builder and I'm running the following commands in my Dockerfile to install all the necessary dependencies -

    # install source dependencies
RUN go get -d -v ./...
RUN go install -v ./...

    the go get command works as expected but the go install command crashes with an exit code: 2 and the following message -

    # github.com/micro/go-micro/registry/consul /go/src/github.com/micro/go-micro/registry/consul/watcher.go:48:33: cannot use cr.Client (type *"github.com/hashicorp/consul/api".Client) as type *"github.com/hashicorp/consul/vendor/github.com/hashicorp/consul/api".Client in argument to wp.RunWithClientAndLogger /go/src/github.com/micro/go-micro/registry/consul/watcher.go:213:37: cannot use cw.r.Client (type *"github.com/hashicorp/consul/api".Client) as type *"github.com/hashicorp/consul/vendor/github.com/hashicorp/consul/api".Client in argument to wp.RunWithClientAndLogger

    Just FYI, I was able to generate the Docker based package builder successfully up till 3 days ago.

  • gossip registry

    gossip registry

    I'm experiment with gossip registry and may be found some specific issue. When i'm start first service1 with specify gossip.Address("172.16.1.254:4223") and on other server service2 with registry.Addrs("172.16.1.254:4223") and gossip.Address("172.16.1.1:0")

    i see that members equal 2 on both sides, but the second service does not registered in registry. But if i stop service1 and start it with registry.Addrs("172.16.1.254:4223") and gossip.Address("172.16.1.1:xxx") where xxx is port provided by service2 all works fine. So issue appears only on first service1 when it start first without other members.

  • [BUG] Micro API fails on streaming endpoint unexpected nil error

    [BUG] Micro API fails on streaming endpoint unexpected nil error

    Describe the bug

    1. What are you trying to do? Run micro api gateway and call a streaming endpoint
    2. What did you expect to happen? Success, actual response
    3. What happens instead?
    {
    "code": 500,
    "detail": "grpc: client streaming protocol violation: get <nil>, want <EOF>",
    "id": "go.micro.client",
    "status": "Internal Server Error"
}

    How to reproduce the bug: micro version 2.0.0 micro api --address :8085 --namespace some.ns --handler rpc http localhost:8085/my/streaming/endpoint

    If possible, please include a minimal code snippet here.

           ctx, cancel := context.WithCancel(ctx)
	ch, errCh := data.GetSomeData(ctx, cancel, s.db, s.service)

	for {
		select {
		case <-ctx.Done():
			stream.Close()
			return nil
		case err := <-errCh:
			return err
		case msg := <-ch:
			if msg == nil {
				return nil
			}
			err := stream.Send(msg)
			if err != nil {
				cancel()
				for len(ch) > 0 {
					<-ch
				}
				return errors.InternalServerError(s.service.Name(), err.Error())
			}
		}
	}

    Environment: Go Version: please paste go version output here

    go version go1.13.1 darwin/amd64
  • Disable timeout on RPC stream connection

    Disable timeout on RPC stream connection

    For a stream RPC connection I'd like to get rid of timeout, I see/found two solutions:

    1. Set timeout to some huge value and hope the client never stays connected long enough.
    2. Set timeout to -1 which 'somehow' seems to work. (Actually strconv.ParseUint() chokes on it and context timeout is not set (and no error is issued...))

    Both options feel hacky to me... Is there a better/recommended solution instead?

  • go get -u fails, apparently due to coreos/etcd move

    go get -u fails, apparently due to coreos/etcd move

    Go version: go1.13 Problem: when I run go get -u github.com/micro/go-micro I get the following error:

    # github.com/coreos/etcd/clientv3
src/github.com/coreos/etcd/clientv3/auth.go:121:72: cannot use auth.callOpts (type []"github.com/coreos/etcd/vendor/google.golang.org/grpc".CallOption) as type []"go.etcd.io/etcd/vendor/google.golang.org/grpc".CallOption in argument to auth.remote.AuthEnable
src/github.com/coreos/etcd/clientv3/auth.go:126:74: cannot use auth.callOpts (type []"github.com/coreos/etcd/vendor/google.golang.org/grpc".CallOption) as type []"go.etcd.io/etcd/vendor/google.golang.org/grpc".CallOption in argument to auth.remote.AuthDisable
src/github.com/coreos/etcd/clientv3/auth.go:131:152: cannot use auth.callOpts (type []"github.com/coreos/etcd/vendor/google.golang.org/grpc".CallOption) as type []"go.etcd.io/etcd/vendor/google.golang.org/grpc".CallOption in argument to auth.remote.UserAdd
src/github.com/coreos/etcd/clientv3/auth.go:136:144: cannot use auth.callOpts (type []"github.com/coreos/etcd/vendor/google.golang.org/grpc".CallOption) as type []"go.etcd.io/etcd/vendor/google.golang.org/grpc".CallOption in argument to auth.remote.UserAdd
src/github.com/coreos/etcd/clientv3/auth.go:141:86: cannot use auth.callOpts (type []"github.com/coreos/etcd/vendor/google.golang.org/grpc".CallOption) as type []"go.etcd.io/etcd/vendor/google.golang.org/grpc".CallOption in argument to auth.remote.UserDelete
src/github.com/coreos/etcd/clientv3/auth.go:146:122: cannot use auth.callOpts (type []"github.com/coreos/etcd/vendor/google.golang.org/grpc".CallOption) as type []"go.etcd.io/etcd/vendor/google.golang.org/grpc".CallOption in argument to auth.remote.UserChangePassword
src/github.com/coreos/etcd/clientv3/auth.go:151:104: cannot use auth.callOpts (type []"github.com/coreos/etcd/vendor/google.golang.org/grpc".CallOption) as type []"go.etcd.io/etcd/vendor/google.golang.org/grpc".CallOption in argument to auth.remote.UserGrantRole
src/github.com/coreos/etcd/clientv3/auth.go:156:80: cannot use auth.callOpts (type []"github.com/coreos/etcd/vendor/google.golang.org/grpc".CallOption) as type []"go.etcd.io/etcd/vendor/google.golang.org/grpc".CallOption in argument to auth.remote.UserGet
src/github.com/coreos/etcd/clientv3/auth.go:161:72: cannot use auth.callOpts (type []"github.com/coreos/etcd/vendor/google.golang.org/grpc".CallOption) as type []"go.etcd.io/etcd/vendor/google.golang.org/grpc".CallOption in argument to auth.remote.UserList
src/github.com/coreos/etcd/clientv3/auth.go:166:106: cannot use auth.callOpts (type []"github.com/coreos/etcd/vendor/google.golang.org/grpc".CallOption) as type []"go.etcd.io/etcd/vendor/google.golang.org/grpc".CallOption in argument to auth.remote.UserRevokeRole
src/github.com/coreos/etcd/clientv3/auth.go:166:106: too many errors

    We first saw this error when our CI build began to fail, and at first couldn't reproduce it locally, but then were able to by adding the -u to the command. Alternately, here's a simple Dockerfile that demonstrates the issue when building it.

    This seems to be caused by the moving of github.com/coreos/etcd to github.com/etcd-io/etcd, as described here: https://github.com/etcd-io/etcd/pull/10044#issuecomment-417125341

    That change and the issue are from last year, so I'm not sure why this is just showing up now, but my colleagues and I (and our CI) can all reproduce the problem now.

    I will look into creating a PR that changes the imports from coreos/etcd to etcd-io/etcd.

  • Adding TLS to RPC

    Adding TLS to RPC

    I'm feeling a little bit dumb, I don't suppose you have an example of adding TLS to a RPC service?

    	security.Setup(log, security.Vault())
	tlsConfig := &tls.Config{
		GetCertificate: security.Vault().CertificateGetter(),
	}

	// initialise service
	service.Init(
		micro.Action(func(c *cli.Context) {
			if c.Bool("debug") {
				log.Level = logrus.DebugLevel
				log.Debug("Debug log level enabled")
			}
		}),
// transport.TLSConfig(tlsConfig),
	)

	countryHandler, err := handler.NewCountryHandler()
	if err != nil {
		log.Fatal(err)
	}

	country.RegisterCountriesHandler(service.Server(), countryHandler)

	if err := service.Run(); err != nil {
		log.Fatal(err)
	}

    I tried adding transport options to the service.Init but it's clearly not that simple?

  • How to add value to metadata?

    How to add value to metadata?

    Hello,

    I want to add some value to metadata from grpc method, then process in wrapper, however I found since grpc 1.40, we can't modify the context, the proposed solution is to use metadata.NewIncomingContext.

    https://github.com/grpc/grpc-go/issues/4363

    I don't know how to do it in micro?

    func (s *ServerImpl) Location(ctx context.Context, req *pb.LocationRequest, rsp *pb.City) error {
// add value to context
	md, _ := metadata.FromContext(ctx)
	if md != nil {
		md.Set("tag", "value")
	}
}

func LogWrapper(fn server.HandlerFunc) server.HandlerFunc {
	return func(ctx context.Context, req server.Request, rsp interface{}) error {
 		err := fn(ctx, req, rsp)
 		value, _ := md.Get("tag")  // value is empty
}
}
  • Documentation unavailable : not displayed due to license restrictions

    Documentation unavailable : not displayed due to license restrictions

    Hello,

    This is not a #1956 or #1953 duplicate. :)

    I tried to access : https://pkg.go.dev/github.com/micro/go-micro/v3?tab=overview But the documentation is unavailable due to license restrictions.

    Thanks,

  • [FEATURE] Adding Sentry reporter

    [FEATURE] Adding Sentry reporter

    Is your feature request related to a problem? Please describe. Hi Guys, i'm really need adding sentry for error reporter but i cannot get recover error data because recover already called when program had error on this file https://github.com/go-micro/plugins/blob/0682c1c82ec59cf5c67c3f859e2e25e011ee0c7a/v4/server/grpc/grpc.go#L401

    Describe the solution you'd like if posible can we add 2 lines code of sentry like this image below image

    Additional context

  • timestamppb.timestamp cause

    timestamppb.timestamp cause "json: cannot unmarshal object into Go value of type string" after upgrade v4.6

    Describe the bug

    1. I use golang and upgrade go micro from v4.5 to v4.6
    2. I expected that all of my integration test should pass
    3. What happens instead? My logger print errors below: {"id":"go.micro.client.codec","code":500,"detail":"json: cannot unmarshal object into Go value of type string","status":"Internal Server Error"} In general it is said go.micro.client.codec has error json: cannot unmarshal object into Go value of type string.

    How to reproduce the bug

    My message structure in proto file

    message Meeting {
    // @inject_tag: json:"start" validate:"required"
    google.protobuf.Timestamp start = 1;
}

    The golang structure created by protoc

    type Meeting struct {
	state         protoimpl.MessageState
	sizeCache     protoimpl.SizeCache
	unknownFields protoimpl.UnknownFields

	// @inject_tag: json:"start" validate:"required"
	Start *timestamppb.Timestamp `protobuf:"bytes,1,opt,name=start,proto3" json:"start" validate:"required"`
}

    When I assign value to meeting.Start, the code like:

    	meeting.Start = timestamp.New(model.Start) // timestamp imported from "google.golang.org/protobuf/types/known/timestamppb"

    There is no such value if I continue using go micro v4.5. I am using libprotoc 3.21.6 and protoc-gen-go v1.27.1

    Go Version output:

    go version go1.17.13 darwin/amd64

    The go env output:

    GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/zhihong/Library/Caches/go-build"
GOENV="/Users/zhihong/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/zhihong/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/zhihong/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.17.13"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/r7/f018pp653m3_fw0xv9jx_tgh0000gn/T/go-build2635807831=/tmp/go-build -gno-record-gcc-switches -fno-common"

    Any suggestions welcomed, thanks,

    James

  • [BUG] go-micro error reported during press test

    [BUG] go-micro error reported during press test

    go-micro

    Concurrency: 5000 Connection pooling: 10

    2022/09/01 17:17:03 client.go:47: INFO : concurrency: 5000
requests per client: 200
 
2022/09/01 17:17:03 client.go:49: INFO : Servers: 10.130.12.14:8900
 
2022/09/01 17:17:03 client.go:57: INFO : message size: 581 bytes
 
2022/09/01 17:17:07 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:07 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:07 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:08 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:09 client.go:125: ERROR: 失败请求, 错误信息: {"id":"go.micro.client","code":503,"detail":"connection closed before server preface received","status":"Service Unavailable"}, 返回值: <nil>
2022/09/01 17:17:35 stats.go:16: INFO : 花了 31302 ms 用于 1000000 请求
2022/09/01 17:17:35 stats.go:38: INFO : 发送请求    : 1000000
2022/09/01 17:17:35 stats.go:39: INFO : 收到的请求    : 1000000
2022/09/01 17:17:35 stats.go:40: INFO : 收到的成功请求 : 999963
2022/09/01 17:17:35 stats.go:48: INFO : 吞吐量  (TPS)    : 31946
 
2022/09/01 17:17:35 stats.go:60: INFO : 平均值: 137153203.650 ns, 中位数: 4337898.500 ns, 最大值: 4303066047.000 ns, 最小值: 243587.000 ns, p99.9: 1817774720.500 ns
2022/09/01 17:17:35 stats.go:61: INFO : 平均值: 137.153 ms, 中位数: 4.338 ms, 最大值: 4303.066 ms, 最小值: 0.244 ms, p99.9: 1817.775 ms
2022/09/01 17:17:35 stats.go:62: INFO : [go-micro]: TPS: 31946.84, TP99: 887.68ms, TP999: 1817.77ms (b=581 Byte, c=5000, n=1000000)
Info: [go-micro]: TPS: 31946.84, TP99: 887.68ms, TP999: 1817.77ms (b=581 Byte, c=5000, n=1000000)

    grpc

    Concurrency: 5000 Connection pooling: 10

    2022/09/01 17:18:27 grpc_client.go:43: INFO : concurrency: 5000
requests per client: 200

2022/09/01 17:18:27 grpc_client.go:46: INFO : Servers: 10.130.12.14:8900

2022/09/01 17:18:27 grpc_client.go:52: INFO : message size: 581 bytes


2022/09/01 17:18:33 stats.go:16: INFO : 花了 5953 ms 用于 1000000 请求
2022/09/01 17:18:34 stats.go:38: INFO : 发送请求    : 1000000
2022/09/01 17:18:34 stats.go:39: INFO : 收到的请求    : 1000000
2022/09/01 17:18:34 stats.go:40: INFO : 收到的成功请求 : 1000000
2022/09/01 17:18:34 stats.go:48: INFO : 吞吐量  (TPS)    : 167982

2022/09/01 17:18:34 stats.go:60: INFO : 平均值: 28944916.637 ns, 中位数: 25278866.000 ns, 最大值: 255435674.000 ns, 最小值: 207720.000 ns, p99.9: 137116793.500 ns
2022/09/01 17:18:34 stats.go:61: INFO : 平均值: 28.945 ms, 中位数: 25.279 ms, 最大值: 255.436 ms, 最小值: 0.208 ms, p99.9: 137.117 ms
2022/09/01 17:18:34 stats.go:62: INFO : [grpc]: TPS: 167982.53, TP99: 80.10ms, TP999: 137.12ms (b=581 Byte, c=5000, n=1000000)
Info: [grpc]: TPS: 167982.53, TP99: 80.10ms, TP999: 137.12ms (b=581 Byte, c=5000, n=1000000)

    A grpc press test on a similarly configured machine shows no errors

    @asim

  • [BUG] go-micro registry node nil pointer

    [BUG] go-micro registry node nil pointer

    Describe the bug

    1. What are you trying to do?
    • Build a Rpc Client to request a Rpc Server
    1. What did you expect to happen?
    • when rpc client try to get rpc server addr and request, "nil pointer exception" should not happen
    1. What happens instead?
    • a panic occur which have already get a Rlock wihout unlock
    /usr/local/go17/src/runtime/panic.go:221 (0xc76946)
/usr/local/go17/src/runtime/signal_unix.go:735 (0xc76916)
mydir/pkg/mod/go-micro.dev/[email protected]/util/registry/util.go:65 (0x1226a8c)
mydir/pkg/mod/go-micro.dev/[email protected]/util/registry/util.go:85 (0x1226cfc)
mydir/pkg/mod/go-micro.dev/[email protected]/registry/cache/cache.go:126 (0x1228a78)
mydir/pkg/mod/go-micro.dev/[email protected]/registry/cache/cache.go:436 (0x122a95d)
mydir/pkg/mod/go-micro.dev/[email protected]/selector/default.go:52 (0x127feeb)
mydir/pkg/mod/github.com/asim/go-micro/plugins/client/grpc/[email protected]/grpc.go:90 (0x1de1242)
mydir/pkg/mod/github.com/asim/go-micro/plugins/client/grpc/[email protected]/grpc.go:403 (0x1de4df6)
[email protected]/gomicro/wrap.go:595 (0x1e0c50c)
[email protected]/gomicro/wrap.go:575 (0x1e0b99e)
[email protected]/gomicro/wrap.go:312 (0x1e0936f)
myproject/cmu.pb.micro.go:238 (0x1b71d4e)
myproject/upcw_service.go:53 (0x1c0f57a)
myproject/upcw_common.go:33 (0x1e52904)
mydir/pkg/mod/github.com/gin-gonic/[email protected]/context.go:147 (0x1abd0ea)
[email protected]/middleware/http/recovery.go:39 (0x1abd0d4)
mydir/pkg/mod/github.com/gin-gonic/[email protected]/context.go:147 (0x1af3dca)
myproject/comm/middleware/session/session.go:82 (0x1af3dad)
mydir/pkg/mod/github.com/gin-gonic/[email protected]/context.go:147 (0x1af714d)
myproject/comm/metadata/metadata.go:113 (0x1af7134)
mydir/pkg/mod/github.com/gin-gonic/[email protected]/context.go:147 (0x1ab8ef3)
[email protected]/middleware/http/gin.go:293 (0x1ab8d73)
mydir/pkg/mod/github.com/gin-gonic/[email protected]/context.go:147 (0x116c369)
mydir/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:403 (0x116bfb9)
mydir/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:364 (0x116bb33)
mydir/pkg/mod/github.com/gorilla/[email protected]/mux.go:210 (0x1cb0b6e)
/usr/local/go17/src/net/http/server.go:2879 (0xf0ee7a)
/usr/local/go17/src/net/http/server.go:1930 (0xf0a467)
/usr/local/go17/src/runtime/asm_amd64.s:1581 (0xc945a0)

    • a goroutine panic with lock and other goroutine can't get the lock: pprof image

    • cause other goroutine can not get lock resource until I restart it.

    How to reproduce the bug: I do not know how to reproduce and why happen. I found it in product environment sometime and I have to restart the program to fix it or the program can not make rpc request.

    Environment: Go Version: please paste go version output here

    
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/myname/.cache/go-build"
GOENV="/home/myname/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/myname/goproject17/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/myname/goproject17"
GOPRIVATE=""
GOPROXY=""
GOROOT="/usr/local/go17"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go17/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.17.6"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build4281984978=/tmp/go-build -gno-record-gcc-switches"
  • Remove cmd package

    Remove cmd package

    Looking at removing the cmd package. I think flag and env var parsing was initially quite useful when the surface area of go-micro was small. Over time that's gotten broader and more complex to manage. Ultimately if go-micro is pluggable, then either the cmd package needs to be totally rewritten or removed entirely.

Related tags
Microservices go golang distributed-systems microservices rpc micro
Sample full stack micro services application built using the go-Micro framework.
 Sample full stack micro services application built using the go-Micro framework.

goTemp goTemp is a full stack Golang microservices sample application built using go-micro. The application is built as a series of services that prov

Dec 26, 2022
a microservice framework for rapid development of micro services in Go with rich eco-system
 a microservice framework for rapid development of micro services in Go with rich eco-system

中文版README Go-Chassis is a microservice framework for rapid development of microservices in Go. it focus on helping developer to deliver cloud native a

Dec 27, 2022
NewSQL distributed storage database based on micro service framework
 NewSQL distributed storage database based on micro service framework

QLite 是基于微服务的 NewSQL 型数据库系统，与传统的一体化数据库不同，该系统将本该内置的多种数据结构（STL）拆分成多个服务模块，每个模块都是独立的一个节点，每个节点都与其主网关进行连接，从而形成分布式存储结构。

Jun 19, 2022
Micro is a platform for cloud native development
 Micro is a platform for cloud native development

Micro Overview Micro addresses the key requirements for building services in the cloud. It leverages the microservices architecture pattern and provid

Dec 29, 2022
GoLang utility packages to assist with the development of web micro-services.

GoTil Golang utility packages to assist with the development of web micro-services. Installation As a library. go get github.com/ccthomas/gotil Usage

Nov 26, 2021
Labs for MIT 6.824 Distributed Systems (Spring 2020)

6.824-2020-labs This repo contains labs for MIT 6.824 Distributed Systems (Spring 2020) The master branch contains the source code with no implementat

Jan 18, 2022
Micro-service framework in Go
 Micro-service framework in Go

Kite Micro-Service Framework Kite is a framework for developing micro-services in Go. Kite is both the name of the framework and the micro-service tha

Jan 9, 2023
🦄🌈 YoyoGo is a simple, light and fast , dependency injection based micro-service framework written in Go.
🦄🌈 YoyoGo is a simple, light and fast , dependency injection based micro-service framework written in Go.

???? YoyoGo is a simple, light and fast , dependency injection based micro-service framework written in Go. Support Nacos ,Consoul ,Etcd ,Eureka ,kubernetes.

Jan 4, 2023
Fastglue is an opinionated, bare bones wrapper that glues together fasthttp and fasthttprouter to act as a micro HTTP framework.

fastglue Overview fastglue is an opinionated, bare bones wrapper that glues together fasthttp and fasthttprouter to act as a micro HTTP framework. It

Jun 14, 2022
Another excellent micro service framework

xservice [WIP] Another excellent micro service framework Features RESTful API (base on echo/v4) gRPC & gRPC gateway service & Swagger document generat

May 28, 2022
😈 Simple micro-front-end framework.
 😈 Simple micro-front-end framework.

Development and Maintenance Status RancherOS 1.x is no longer being actively maintained. There are two significant reasons behind this product decisio

Jan 5, 2022
A code generator that turns plain old Go services into RPC-enabled (micro)services with robust HTTP APIs.

Frodo is a code generator and runtime library that helps you write RPC-enabled (micro) services and APIs.

Dec 16, 2022
goTempM is a full stack Golang microservices sample application built on top of the Micro platform.
goTempM is a full stack Golang microservices sample application built on top of the Micro platform.

goTempM is a full stack Golang microservices sample application built on top of the Micro platform.

Sep 24, 2022
Automatic Service Mesh and RPC generation for Go micro services, it's a humble alternative to gRPC with Istio.
 Automatic Service Mesh and RPC generation for Go micro services, it's a humble alternative to gRPC with Istio.

Mesh RPC MeshRPC provides automatic Service Mesh and RPC generation for Go micro services, it's a humble alternative to gRPC with Istio. In a nutshell

Aug 22, 2022
微服务架构-micro-基于go-zero zrpc etcd 单独集成orm-gorm 日志-uber/zap

micro目录结构 pkg ├── api 业务接口逻辑层 所有的业务逻辑存放目录。 │ ├── model 数据模型 数据管理层，仅用于操作管理数据，如数据库操作 ├── assets

Jul 12, 2022
A Micro-UTP, plug-able sanity checker for any on-prem JFrog platform instance

hello-frog About this plugin This plugin is a template and a functioning example for a basic JFrog CLI plugin. This README shows the expected structur

Dec 7, 2021
Go micro frame cli tool

go-micro-frame cli tool install git clone [email protected]:jettjia/go-micro-frame-cli.git cd go-micro-frame-cli go build -ldflags "-w -s" -o go-micro-f

Sep 19, 2022
An open network for Micro services.
 An open network for Micro services.

Micro Network The micro network is an open network for Micro services. Note: The network is still in early development. This document serves as a star

Nov 1, 2022
micro-draft-manager is a microservice that helps you to manage unstructured data in your application with sorting and full-text search

micro-draft-manager is a microservice that helps you to manage unstructured data in your application with sorting and full-text search. For example, y

Nov 24, 2021