Sentinel Go version (Reliability & Resilience)

Last update: Dec 27, 2022

Comments: 17

Sentinel: The Sentinel of Your Microservices

Introduction

As distributed systems become increasingly popular, the reliability between services is becoming more important than ever before. Sentinel takes "flow" as breakthrough point, and works on multiple fields including flow control, circuit breaking and system adaptive protection, to guarantee reliability and resiliency of microservices.

Sentinel provides the following features:

Rich applicable scenarios: Sentinel has been wildly used in Alibaba, and has covered almost all the core-scenarios in Double-11 (11.11) Shopping Festivals in the past 10 years, such as “Second Kill” which needs to limit burst flow traffic to meet the system capacity, throttling, circuit breaking for unreliable downstream services, distributed rate limiting, etc.
Real-time monitoring: Sentinel also provides real-time monitoring ability. You can see the runtime information of a single machine in real-time, and pump the metrics to outside metric components like Prometheus.
Cloud-native ecosystem: Sentinel Go provides out-of-box integrations with cloud-native components.

Documentation

See the 中文文档 for document in Chinese.

See the Wiki for full documentation, examples, blog posts, and other information.

If you are using Sentinel, please leave a comment here to tell us your scenario to make Sentinel better. It's also encouraged to add the link of your blog post, tutorial, demo or customized components to Awesome Sentinel.

Sub-projects

All integration modules and sub-projects are located in sentinel-group.

Bugs and Feedback

For bug report, questions and discussions please submit GitHub Issues.

Contributing

Contributions are always welcomed! Please see CONTRIBUTING for detailed guidelines.

You can start with the issues labeled with good first issue.

Communication

DingTalk Group (钉钉群): 23339422
Gitter

Owner

Alibaba

Alibaba Open Source

https://github.com/sentinel-group/sentinel-golang

Comments

Basic abstraction for data-source extension

Issue Description

Type: feature request

Describe what feature you want

Define datasource extension framework implement FileRefreshableDataSource

Additional context

Add any other context or screenshots about the feature request here.
Sentinel Golang Version V1 Discussions
Describe what feature you want

Implement Golang version of Sentinel, including basic statistic and flow functions. Some expected features:

Basic statistic structure (sliding window that can achieve high concurrency and accuracy)

Rate limiting (based on token bucket algorithm, maybe we can leverage golang.org/x/time/rate.Limiter)

Logging

Metrics interface that can pump real-time metrics to outside consumer

Embedded API server

Integrate with Sentinel dashboard (machine discovery)

Discussions are welcomed!
Fix integer overflow on 32-bit platform

Signed-off-by: Long Dai [email protected]

Describe what this PR does / why we need it

Fix https://github.com/dapr/components-contrib/issues/1033

Does this pull request fix one issue?

Describe how you did it

Describe how to verify it

Special notes for reviews
Add etcd data-source extenstion

Issue Description

Type: feature request Adapt etcd as dynamic datasource for Sentinel

Describe what feature you want

Need detail design. PR

Additional context

Add any other context or screenshots about the feature request here.
feat(adater): Implement go-micro adapter

Describe what this PR does / why we need it

This PR integrate the Adapter of go-micro

Does this pull request fix one issue?

This PR add integration for go-micro, Fixes #150

Describe how you did it

Add this feature via go-micro server wrapper and intercept client call

Describe how to verify it

Run tests inside the adapter/go_micro package

Special notes for reviews

[BUG] Data-Race for metricBucket.addRt

Issue Description

func (mb *metricBucket) addRt(rt int64) {
	mb.add(base.MetricEventRt, rt)
	if rt < atomic.LoadInt64(&mb.minRt) {
		atomic.StoreInt64(&mb.minRt, rt)
	}
}

here, not concurrent safe for mb.minRt. It's better to use Mutex

another missing:

func (n *BaseStatNode) AddRtAndCompleteRequest(rt, count uint64) {
	n.rollingCounter.AddCount(base.MetricEventComplete, int64(count))
	n.rollingCounter.AddCount(base.MetricEventRt, int64(count)) // should be `rt`
}

Type: bug report

Describe what happened

Describe what you expected to happen

How to reproduce it (as minimally and precisely as possible)

Tell us your environment

Anything else we need to know?

Thanks for your work.

fix bug in circuit_breaker.go about SlowRequestRatio

Describe what this PR does / why we need it

when slowRequestRatio was set to 1.0d, requests will never be blocked. Because the currentRatio will never be bigger than 100%

Does this pull request fix one issue?

Describe how you did it

Special treatment for case when slowRequestRatio was set to 1.0d

Describe how to verify it

Special notes for reviews
datasource: support apollo

Describe what this PR does / why we need it

Datasource support apollo

Does this pull request fix one issue?

Describe how you did it

Describe how to verify it

Special notes for reviews
Implement consul datasource

Describe what this PR does / why we need it

implement consul datasource

Does this pull request fix one issue?

Describe how you did it

adapter consul client api; read the configuration immediately after connecting to consul server, then start polling.

Describe how to verify it

Special notes for reviews
Add etcd data-source extenstion

Describe what this PR does / why we need it #51 Does this pull request fix one issue? Describe how you did it Describe how to verify it Special notes for reviews
Rule Manager: Support update rule hook fix #275

Describe what this PR does / why we need it

support update rule hook

Does this pull request fix one issue?

Fixes #275

Describe how you did it

Describe how to verify it

Special notes for reviews
feat: return entry when request is blocked.

Describe what this PR does / why we need it

Does this pull request fix one issue?

Fixed #493

Describe how you did it

Describe how to verify it

Special notes for reviews
feat: returns entry info even when request is blocked.

Issue Description

Type: feature request

Describe what feature you want

This returns entry info when request Entry is blocked, it's very useful when we need additional information to collect metrics or log stuff.

Additional context

Add any other context or screenshots about the feature request here.
[Feature] Carry params by key-value in sentinel.Entry | 热点参数埋点支持以 key-value 的方式传入参数
Issue Description

Type: feature request

Describe what feature you want

Support carrying params by key-value in sentinel.Entry(options...). An example:

sentinel.Entry(WithArg("key1", "value1"), WithArg("key2", "value2"));

热点参数埋点支持以 key-value 的方式传入参数，这样开发者可以无需像之前传入 args 一样关心参数顺序，从而更方便的配置热点规则（按照 key 来匹配即可，不需按参数匹配）。

热点规则的设计，以及 Entry API 的设计需要做相应的调整。

Additional context

Add any other context or screenshots about the feature request here.
[feature] integrate datasource of OpenSergo by OpenSergo Go SDK
Does this pull request fix one issue?

fixed #483

Special notes for reviews

~~### Prepare Environment for Test Version~~

~~- Prepare the OpenSergo GO SDK. Because of the OpenSergo GO SDK has no published version, so should download the sourcecode of OpenSergo GO SDKjnan806/opensergo-go-sdk/tree/initial-version, and move it into you GOPATH.~~ ~~- Rename the right version in directory name of OpenSergo GO SDK sourcecode. Make sure the version in sourcecode directory name is the same with go.mod.
eg. $GOPATH/pkg/mod/github.com/opensergo/[email protected]~~

The jnan806/opensergo-go-sdk/tree/initial-version has been merged into opensergo/opensergo-go-sdk. So this step of Prepare theOpenSergo GO SDK` is unnecessary. It's only need to add use sentinel-go with the opensergo-datasource.

Samples

datasource_opensergo_example.go

[BUG] Race condition in metric.DefaultMetricSearcher

Issue Description

Type: bug report

Describe what happened

Race condition was detected in metric.DefaultMetricSearcher:

WARNING: DATA RACE
Read at 0x00c000510f00 by goroutine 59:
  github.com/alibaba/sentinel-golang/core/log/metric.(*DefaultMetricSearcher).getOffsetStartAndFileIdx()
      /Users/sczyh30/go/pkg/mod/github.com/alibaba/[email protected]/core/log/metric/searcher.go:101 +0x106
  github.com/alibaba/sentinel-golang/core/log/metric.(*DefaultMetricSearcher).searchOffsetAndRead()
      /Users/sczyh30/go/pkg/mod/github.com/alibaba/[email protected]/core/log/metric/searcher.go:71 +0x12a
  github.com/alibaba/sentinel-golang/core/log/metric.(*DefaultMetricSearcher).FindByTimeAndResource()
      /Users/sczyh30/go/pkg/mod/github.com/alibaba/[email protected]/core/log/metric/searcher.go:53 +0xc7
   ...

Previous write at 0x00c000510f00 by goroutine 31:
  github.com/alibaba/sentinel-golang/core/log/metric.(*DefaultMetricSearcher).findOffsetToStart()
      /Users/sczyh30/go/pkg/mod/github.com/alibaba/[email protected]/core/log/metric/searcher.go:165 +0x5ca
  github.com/alibaba/sentinel-golang/core/log/metric.(*DefaultMetricSearcher).searchOffsetAndRead()
      /Users/sczyh30/go/pkg/mod/github.com/alibaba/[email protected]/core/log/metric/searcher.go:80 +0x1fd
  github.com/alibaba/sentinel-golang/core/log/metric.(*DefaultMetricSearcher).FindByTimeAndResource()
      /Users/sczyh30/go/pkg/mod/github.com/alibaba/[email protected]/core/log/metric/searcher.go:53 +0xc7
  ...

Describe what you expected to happen

No race condition.

How to reproduce it (as minimally and precisely as possible)

Run a demo and invoke the search function of the MetricSearcher concurrently.

Tell us your environment

Sentinel Go v1.0.3

Anything else we need to know?

NONE

Bump github.com/labstack/echo/v4 from 4.1.17 to 4.9.0 in /pkg/adapters/echo
Bumps github.com/labstack/echo/v4 from 4.1.17 to 4.9.0.

Release notes

Sourced from github.com/labstack/echo/v4's releases.

v4.9.0

Security

Fix open redirect vulnerability in handlers serving static directories (e.Static, e.StaticFs, echo.StaticDirectoryHandler) #2260

Enhancements

Allow configuring ErrorHandler in CSRF middleware #2257

Replace HTTP method constants in tests with stdlib constants #2247

v4.8.0

Most notable things

You can now add any arbitrary HTTP method type as a route #2237

e.Add("COPY", "/*", func(c echo.Context) error return c.String(http.StatusOK, "OK COPY") })

You can add custom 404 handler for specific paths #2217

e.RouteNotFound("/*", func(c echo.Context) error { return c.NoContent(http.StatusNotFound) })
g := e.Group("/images") g.RouteNotFound("/*", func(c echo.Context) error { return c.NoContent(http.StatusNotFound) })

Enhancements

Add new value binding methods (UnixTimeMilli,TextUnmarshaler,JSONUnmarshaler) to Valuebinder #2127

Refactor: body_limit middleware unit test #2145

Refactor: Timeout mw: rework how test waits for timeout. #2187

BasicAuth middleware returns 500 InternalServerError on invalid base64 strings but should return 400 #2191

Refactor: duplicated findStaticChild process at findChildWithLabel #2176

Allow different param names in different methods with same path scheme #2209

Add support for registering handlers for different 404 routes #2217

Middlewares should use errors.As() instead of type assertion on HTTPError #2227

Allow arbitrary HTTP method types to be added as routes #2237

v4.7.2

Fixes

Fix nil pointer exception when calling Start again after address binding error #2131

Fix CSRF middleware not being able to extract token from multipart/form-data form #2136

Fix Timeout middleware write race #2126

Enhancements

... (truncated)

Changelog

Sourced from github.com/labstack/echo/v4's changelog.

v4.9.0 - 2022-09-04

Security

Fix open redirect vulnerability in handlers serving static directories (e.Static, e.StaticFs, echo.StaticDirectoryHandler) #2260

Enhancements

Allow configuring ErrorHandler in CSRF middleware #2257

Replace HTTP method constants in tests with stdlib constants #2247

v4.8.0 - 2022-08-10

Most notable things

You can now add any arbitrary HTTP method type as a route #2237

e.Add("COPY", "/*", func(c echo.Context) error return c.String(http.StatusOK, "OK COPY") })

You can add custom 404 handler for specific paths #2217

e.RouteNotFound("/*", func(c echo.Context) error { return c.NoContent(http.StatusNotFound) })
g := e.Group("/images") g.RouteNotFound("/*", func(c echo.Context) error { return c.NoContent(http.StatusNotFound) })

Enhancements

Add new value binding methods (UnixTimeMilli,TextUnmarshaler,JSONUnmarshaler) to Valuebinder #2127

Refactor: body_limit middleware unit test #2145

Refactor: Timeout mw: rework how test waits for timeout. #2187

BasicAuth middleware returns 500 InternalServerError on invalid base64 strings but should return 400 #2191

Refactor: duplicated findStaticChild process at findChildWithLabel #2176

Allow different param names in different methods with same path scheme #2209

Add support for registering handlers for different 404 routes #2217

Middlewares should use errors.As() instead of type assertion on HTTPError #2227

Allow arbitrary HTTP method types to be added as routes #2237

v4.7.2 - 2022-03-16

Fixes

Fix nil pointer exception when calling Start again after address binding error #2131

Fix CSRF middleware not being able to extract token from multipart/form-data form #2136

Fix Timeout middleware write race #2126

... (truncated)

Commits

16d3b65 Changelog for 4.9.0

0ac4d74 Fix #2259 open redirect vulnerability in echo.StaticDirectoryHandler (used by...

d77e8c0 Added ErrorHandler and ErrorHandlerWithContext in CSRF middleware (#2257)

534bbb8 replace POST constance with stdlib constance

fb57d96 replace GET constance with stdlib constance

d48197d Changelog for 4.8.0

cba12a5 Allow arbitrary HTTP method types to be added as routes

a327884 add:README.md-Third-party middlewares-github.com/go-woo/protoc-gen-echo

61422dd Update CI-flow (Go 1.19 +deps)

a9879ff Middlewares should use errors.As() instead of type assertion on HTTPError

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

Bump-version - Bump a given semantic version, following a given version fragment

bump-version Bump a given semantic version, following a given version fragment.

Feb 7, 2022

Easy proxy for redis sentinel

sentinel-proxy Easy proxy for redis sentinel. Main purpose of the proxy is easy work with redis sentinel without changing the client code. Especially

Sep 6, 2022

Zero Trust Network Communication Sentinel provides peer-to-peer, multi-protocol, automatic networking, cross-CDN and other features for network communication.

Thank you for your interest in ZASentinel ZASentinel helps organizations improve information security by providing a better and simpler way to protect

Nov 1, 2022

go-fastdfs 是一个简单的分布式文件系统(私有云存储)，具有无中心、高性能，高可靠，免维护等优点，支持断点续传，分块上传，小文件合并，自动同步，自动修复。Go-fastdfs is a simple distributed file system (private cloud storage), with no center, high performance, high reliability, maintenance free and other advantages, support breakpoint continuation, block upload, small file merge, automatic synchronization, automatic repair.(similar fastdfs).

中文 English 愿景：为用户提供最简单、可靠、高效的分布式文件系统。 go-fastdfs是一个基于http协议的分布式文件系统，它基于大道至简的设计理念，一切从简设计，使得它的运维及扩展变得更加简单，它具有高性能、高可靠、无中心、免维护等优点。大家担心的是这么简单的文件系统，靠不靠谱，可不

Jan 8, 2023

A fast port scanner written in go with a focus on reliability and simplicity. Designed to be used in combination with other tools for attack surface discovery in bug bounties and pentests

Naabu is a port scanning tool written in Go that allows you to enumerate valid ports for hosts in a fast and reliable manner. It is a really simple to

Dec 31, 2022

httpx is a fast and multi-purpose HTTP toolkit allows to run multiple probers using retryablehttp library, it is designed to maintain the result reliability with increased threads.

Features • Installation • Usage • Running httpx • Notes • Join Discord httpx is a fast and multi-purpose HTTP toolkit allow to run multiple probers us

Jan 8, 2023

Elastic Key-Value Storage With Strong Consistency and Reliability

What is Elasticell? Elasticell is a distributed NoSQL database with strong consistency and reliability. Compatible with Redis protocol Use Elasticell

Dec 28, 2022

Continuous profiling for analysis of CPU, memory usage over time, and down to the line number. Saving infrastructure cost, improving performance, and increasing reliability.

Jan 2, 2023

Sentinel Go version (Reliability & Resilience)

Sentinel: The Sentinel of Your Microservices

Introduction

Documentation

Sub-projects

Bugs and Feedback

Contributing

Communication

Owner

Alibaba

Comments

Basic abstraction for data-source extension

Issue Description

Describe what feature you want

Additional context

Sentinel Golang Version V1 Discussions

Describe what feature you want

Fix integer overflow on 32-bit platform

Describe what this PR does / why we need it

Does this pull request fix one issue?

Describe how you did it

Describe how to verify it

Special notes for reviews

Add etcd data-source extenstion

Issue Description

Describe what feature you want

Additional context

feat(adater): Implement go-micro adapter

Describe what this PR does / why we need it

Does this pull request fix one issue?

Describe how you did it

Describe how to verify it

Special notes for reviews

[BUG] Data-Race for metricBucket.addRt

Issue Description

Describe what happened

Describe what you expected to happen

How to reproduce it (as minimally and precisely as possible)

Tell us your environment

Anything else we need to know?

fix bug in circuit_breaker.go about SlowRequestRatio

Describe what this PR does / why we need it

Does this pull request fix one issue?

Describe how you did it

Describe how to verify it

Special notes for reviews

datasource: support apollo

Describe what this PR does / why we need it

Does this pull request fix one issue?

Describe how you did it

Describe how to verify it

Special notes for reviews

Implement consul datasource

Describe what this PR does / why we need it

Does this pull request fix one issue?

Describe how you did it

Describe how to verify it

Special notes for reviews

Add etcd data-source extenstion

Rule Manager: Support update rule hook fix #275

Describe what this PR does / why we need it

Does this pull request fix one issue?

Describe how you did it

Describe how to verify it

Special notes for reviews

feat: return entry when request is blocked.

Describe what this PR does / why we need it

Does this pull request fix one issue?

Describe how you did it

Describe how to verify it

Special notes for reviews

feat: returns entry info even when request is blocked.

Issue Description

Describe what feature you want

Additional context

[Feature] Carry params by key-value in sentinel.Entry | 热点参数埋点支持以 key-value 的方式传入参数

Issue Description

Describe what feature you want

Additional context

[feature] integrate datasource of OpenSergo by OpenSergo Go SDK