go-fasttld is a high performance top level domains (TLD) extraction module.

go-fasttld

Go Reference Go Report Card Codecov Coverage Mentioned in Awesome Go

GitHub license

go-fasttld is a high performance top level domains (TLD) extraction module implemented with compressed tries.

This module is a port of the Python fasttld module, with additional modifications to support extraction of subcomponents from full URLs, IPv4 addresses, and IPv6 addresses.

Trie

Background

go-fasttld extracts subcomponents like top level domains (TLDs), subdomains and hostnames from URLs efficiently by using the regularly-updated Mozilla Public Suffix List and the compressed trie data structure.

For example, it extracts the com TLD, maps subdomain, and google domain from https://maps.google.com:8080/a/long/path/?query=42.

go-fasttld also supports extraction of private domains listed in the Mozilla Public Suffix List like 'blogspot.co.uk' and 'sinaapp.com', extraction of IPv4 addresses, and extraction of IPv6 addresses.

Why not split on "." and take the last element instead?

Splitting on "." and taking the last element only works for simple TLDs like .com, but not more complex ones like oseto.nagasaki.jp.

Compressed trie example

Valid TLDs from the Mozilla Public Suffix List are appended to the compressed trie in reverse-order.

Given the following TLDs
au
nsw.edu.au
com.ac
edu.ac
gov.ac

and the example URL host `example.nsw.edu.au`

The compressed trie will be structured as follows:

START
 ╠═ au 🚩 ✅
 ║  ╚═ edu ✅
 ║     ╚═ nsw 🚩 ✅
 ╚═ ac
    ╠═ com 🚩
    ╠═ edu 🚩
    ╚═ gov 🚩

=== Symbol meanings ===
🚩 : path to this node is a valid TLD
✅ : path to this node found in example URL host `example.nsw.edu.au`

The URL host subcomponents are parsed from right-to-left until no more matching nodes can be found. In this example, the path of matching nodes are au -> edu -> nsw. Reversing the nodes gives the extracted TLD nsw.edu.au.

Installation

go get github.com/elliotwutingfeng/go-fasttld

Quick Start

Full demo available in the examples folder

Domain

// Initialise fasttld extractor
extractor, _ := fasttld.New(fasttld.SuffixListParams{})

//Extract URL subcomponents
url := "https://[email protected]:5000/a/b/c/d/e/f/g/h/i?id=42"
res := extractor.Extract(fasttld.URLParams{URL: url})

// Display results
fmt.Println(res.Scheme)           // https://
fmt.Println(res.UserInfo)         // some-user
fmt.Println(res.SubDomain)        // a.long.subdomain
fmt.Println(res.Domain)           // ox
fmt.Println(res.Suffix)           // ac.uk
fmt.Println(res.RegisteredDomain) // ox.ac.uk
fmt.Println(res.Port) // 5000
fmt.Println(res.Path) // a/b/c/d/e/f/g/h/i?id=42

IPv4 Address

extractor, _ := fasttld.New(fasttld.SuffixListParams{})

url = "https://127.0.0.1:5000"
res = extractor.Extract(fasttld.URLParams{URL: url})

// res.Scheme = https://
// res.UserInfo = <no output>
// res.SubDomain = <no output>
// res.Domain = 127.0.0.1
// res.Suffix = <no output>
// res.RegisteredDomain = 127.0.0.1
// res.Port = 5000
// res.Path = <no output>

IPv6 Address

extractor, _ := fasttld.New(fasttld.SuffixListParams{})

url = "https://[aBcD:ef01:2345:6789:aBcD:ef01:2345:6789]:5000"
res = extractor.Extract(fasttld.URLParams{URL: url})

// res.Scheme = https://
// res.UserInfo = <no output>
// res.SubDomain = <no output>
// res.Domain = aBcD:ef01:2345:6789:aBcD:ef01:2345:6789
// res.Suffix = <no output>
// res.RegisteredDomain = aBcD:ef01:2345:6789:aBcD:ef01:2345:6789
// res.Port = 5000
// res.Path = <no output>

Internationalised label separators

go-fasttld supports the following internationalised label separators (IETF RFC 3490)

  • U+002E (full stop)
  • U+3002 (ideographic full stop)
  • U+FF0E (fullwidth full stop)
  • U+FF61 (halfwidth ideographic full stop)
extractor, _ := fasttld.New(fasttld.SuffixListParams{})

url = "https://brb\u002ei\u3002am\uff0egoing\uff61to\uff0ebe\u3002a\uff61fk"
res = extractor.Extract(fasttld.URLParams{URL: url})

// res.Scheme = https://
// res.UserInfo = <no output>
// res.SubDomain = brb\u002ei\u3002am\uff0egoing\uff61to
// res.Domain = be
// res.Suffix = a\uff61fk
// res.RegisteredDomain = be\u3002a\uff61fk
// res.Port = <no output>
// res.Path = <no output>

Public Suffix List options

Specify custom public suffix list file

You can use a custom public suffix list file by setting CacheFilePath in fasttld.SuffixListParams{} to its absolute path.

cacheFilePath := "/absolute/path/to/file.dat"
extractor, _ := fasttld.New(fasttld.SuffixListParams{CacheFilePath: cacheFilePath})

Updating the default Public Suffix List cache

Whenever fasttld.New is called without specifying CacheFilePath in fasttld.SuffixListParams{}, the local cache of the default Public Suffix List is updated automatically if it is more than 3 days old. You can also manually update the cache by using Update().

// Automatic update performed if `CacheFilePath` is not specified
// and local cache is more than 3 days old
extractor, _ := fasttld.New(fasttld.SuffixListParams{})

// Manually update local cache
if err := extractor.Update(); err != nil {
    log.Println(err)
}

Private domains

According to the Mozilla.org wiki, the Mozilla Public Suffix List contains private domains like blogspot.com and sinaapp.com.

By default, go-fasttld excludes these private domains (i.e. IncludePrivateSuffix = false)

extractor, _ := fasttld.New(fasttld.SuffixListParams{})

url := "https://google.blogspot.com"
res := extractor.Extract(fasttld.URLParams{URL: url})

// res.Scheme = https://
// res.UserInfo = <no output>
// res.SubDomain = google
// res.Domain = blogspot
// res.Suffix = com
// res.RegisteredDomain = blogspot.com
// res.Port = <no output>
// res.Path = <no output>

You can include private domains by setting IncludePrivateSuffix = true

extractor, _ := fasttld.New(fasttld.SuffixListParams{IncludePrivateSuffix: true})

url := "https://google.blogspot.com"
res := extractor.Extract(fasttld.URLParams{URL: url})

// res.Scheme = https://
// res.UserInfo = <no output>
// res.SubDomain = <no output>
// res.Domain = google
// res.Suffix = blogspot.com
// res.RegisteredDomain = google.blogspot.com
// res.Port = <no output>
// res.Path = <no output>

Extraction options

Ignore Subdomains

You can ignore subdomains by setting IgnoreSubDomains = true. By default, subdomains are extracted.

extractor, _ := fasttld.New(fasttld.SuffixListParams{})

url := "https://maps.google.com"
res := extractor.Extract(fasttld.URLParams{URL: url, IgnoreSubDomains: true})

// res.Scheme = https://
// res.UserInfo = <no output>
// res.SubDomain = <no output>
// res.Domain = google
// res.Suffix = com
// res.RegisteredDomain = google.com
// res.Port = <no output>
// res.Path = <no output>

Punycode

Convert internationalised URLs to punycode before extraction by setting ConvertURLToPunyCode = true. By default, URLs are not converted to punycode.

extractor, _ := fasttld.New(fasttld.SuffixListParams{})

url := "https://hello.世界.com"
res := extractor.Extract(fasttld.URLParams{URL: url, ConvertURLToPunyCode: true})

// res.Scheme = https://
// res.UserInfo = <no output>
// res.SubDomain = hello
// res.Domain = xn--rhqv96g
// res.Suffix = com
// res.RegisteredDomain = xn--rhqv96g.com
// res.Port = <no output>
// res.Path = <no output>

res = extractor.Extract(fasttld.URLParams{URL: url, ConvertURLToPunyCode: false})

// res.Scheme = https://
// res.UserInfo = <no output>
// res.SubDomain = hello
// res.Domain = 世界
// res.Suffix = com
// res.RegisteredDomain = 世界.com
// res.Port = <no output>
// res.Path = <no output>

Testing

go test -v -coverprofile=test_coverage.out && go tool cover -html=test_coverage.out -o test_coverage.html

Benchmarks

go test -bench=. -benchmem -cpu 1

Modules used

Benchmark Name Source
GoFastTld go-fasttld (this module)
JPilloraGoTld github.com/jpillora/go-tld
JoeGuoTldExtract github.com/joeguo/tldextract
Mjd2021USATldExtract github.com/mjd2021usa/tldextract
M507Tlde github.com/M507/tlde

Results

Benchmarks performed on AMD Ryzen 7 5800X, Manjaro Linux.

go-fasttld performs especially well on longer URLs.


#1

https://news.google.com

Benchmark Name Iterations ns/op B/op allocs/op Fastest
GoFastTld 2389614 496.8 ns/op 176 B/op 4 allocs/op ✔️
JPilloraGoTld 2300103 521.2 ns/op 224 B/op 2 allocs/op
JoeGuoTldExtract 1480351 822.2 ns/op 208 B/op 7 allocs/op
Mjd2021USATldExtract 1336317 876.7 ns/op 208 B/op 7 allocs/op
M507Tlde 2276070 513.1 ns/op 160 B/op 5 allocs/op

#2

https://iupac.org/iupac-announces-the-2021-top-ten-emerging-technologies-in-chemistry/

Benchmark Name Iterations ns/op B/op allocs/op Fastest
GoFastTld 2254648 537.6 ns/op 304 B/op 4 allocs/op ✔️
JPilloraGoTld 1633924 737.0 ns/op 224 B/op 2 allocs/op
JoeGuoTldExtract 1532829 781.0 ns/op 288 B/op 6 allocs/op
Mjd2021USATldExtract 1444665 832.5 ns/op 288 B/op 6 allocs/op
M507Tlde 2032639 584.8 ns/op 272 B/op 5 allocs/op

#3

https://www.google.com/maps/dir/Parliament+Place,+Parliament+House+Of+Singapore,+Singapore/Parliament+St,+London,+UK/@25.2440033,33.6721455,4z/data=!3m1!4b1!4m14!4m13!1m5!1m1!1s0x31da19a0abd4d71d:0xeda26636dc4ea1dc!2m2!1d103.8504863!2d1.2891543!1m5!1m1!1s0x487604c5aaa7da5b:0xf13a2197d7e7dd26!2m2!1d-0.1260826!2d51.5017061!3e4

Benchmark Name Iterations ns/op B/op allocs/op Fastest
GoFastTld 1519119 785.9 ns/op 784 B/op 4 allocs/op ✔️
JPilloraGoTld 399526 2848 ns/op 928 B/op 4 allocs/op
JoeGuoTldExtract 778827 1420 ns/op 1120 B/op 6 allocs/op
Mjd2021USATldExtract 755976 1523 ns/op 1120 B/op 6 allocs/op
M507Tlde 806964 1584 ns/op 1120 B/op 6 allocs/op

Acknowledgements

Comments
  • benchmar跑出来的结果跟你对不上

    benchmar跑出来的结果跟你对不上

    我benchmark,joeguo/tldextract更快,耗内存更小 BenchmarkJoeguoTld-12 1348 4245811 ns/op 2226227 B/op 30087 allocs/op BenchmarkFastTld-12 962 6263159 ns/op 3492143 B/op 40125 allocs/op

  • Update module go to 1.19

    Update module go to 1.19

    Mend Renovate

    This PR contains the following updates:

    | Package | Type | Update | Change | |---|---|---|---| | go (source) | golang | minor | 1.18 -> 1.19 |


    Release Notes

    golang/go

    v1.19.0


    Configuration

    📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    🚦 Automerge: Enabled.

    Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    🔕 Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, click this checkbox.

    This PR has been generated by Mend Renovate. View repository job log here.

  • Update module golang.org/x/net to v0.4.0

    Update module golang.org/x/net to v0.4.0

    Mend Renovate

    This PR contains the following updates:

    | Package | Type | Update | Change | |---|---|---|---| | golang.org/x/net | require | minor | v0.3.0 -> v0.4.0 |


    Release Notes

    golang/net

    v0.4.0

    Compare Source


    Configuration

    📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    🚦 Automerge: Enabled.

    Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    🔕 Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, check this box

    This PR has been generated by Mend Renovate. View repository job log here.

  • Update module golang.org/x/net to v0.3.0

    Update module golang.org/x/net to v0.3.0

    Mend Renovate

    This PR contains the following updates:

    | Package | Type | Update | Change | |---|---|---|---| | golang.org/x/net | require | minor | v0.2.0 -> v0.3.0 |


    Release Notes

    golang/net

    v0.3.0

    Compare Source


    Configuration

    📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    🚦 Automerge: Enabled.

    Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    🔕 Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, check this box

    This PR has been generated by Mend Renovate. View repository job log here.

  • Update module github.com/spf13/cobra to v1.6.1

    Update module github.com/spf13/cobra to v1.6.1

    Mend Renovate

    This PR contains the following updates:

    | Package | Type | Update | Change | |---|---|---|---| | github.com/spf13/cobra | require | patch | v1.6.0 -> v1.6.1 |


    Release Notes

    spf13/cobra

    v1.6.1

    Compare Source

    Bug fixes 🐛
    • Fixes a panic when AddGroup isn't called before AddCommand(my-sub-command) is executed. This can happen within more complex cobra file structures that have many different inits to be executed. Now, the check for groups has been moved to ExecuteC and provides more flexibility when working with grouped commands - @​marckhouzam (and shout out to @​aawsome, @​andig and @​KINGSABRI for a deep investigation into this! 👏🏼)

    Configuration

    📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    🚦 Automerge: Enabled.

    Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    🔕 Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, click this checkbox.

    This PR has been generated by Mend Renovate. View repository job log here.

  • Update module golang.org/x/net to v0.1.0

    Update module golang.org/x/net to v0.1.0

    Mend Renovate

    This PR contains the following updates:

    | Package | Type | Update | Change | |---|---|---|---| | golang.org/x/net | require | minor | v0.0.0-20221019024206-cb67ada4b0ad -> v0.1.0 |


    Configuration

    📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    🚦 Automerge: Enabled.

    Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    🔕 Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, click this checkbox.

    This PR has been generated by Mend Renovate. View repository job log here.

  • Update golang.org/x/net digest to cb67ada

    Update golang.org/x/net digest to cb67ada

    Mend Renovate

    This PR contains the following updates:

    | Package | Type | Update | Change | |---|---|---|---| | golang.org/x/net | require | digest | f25eb7e -> cb67ada |


    Configuration

    📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    🚦 Automerge: Enabled.

    Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    🔕 Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, click this checkbox.

    This PR has been generated by Mend Renovate. View repository job log here.

  • Update golang.org/x/net digest to f15817d

    Update golang.org/x/net digest to f15817d

    Mend Renovate

    This PR contains the following updates:

    | Package | Type | Update | Change | |---|---|---|---| | golang.org/x/net | require | digest | 0b7e1fb -> f15817d |


    Configuration

    📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    🚦 Automerge: Enabled.

    Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    🔕 Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, click this checkbox.

    This PR has been generated by Mend Renovate. View repository job log here.

  • Update golang.org/x/net digest to 0b7e1fb

    Update golang.org/x/net digest to 0b7e1fb

    Mend Renovate

    This PR contains the following updates:

    | Package | Type | Update | Change | |---|---|---|---| | golang.org/x/net | require | digest | 8021a29 -> 0b7e1fb |


    Configuration

    📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    🚦 Automerge: Enabled.

    Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    🔕 Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, click this checkbox.

    This PR has been generated by Mend Renovate. View repository job log here.

  • Update module github.com/spf13/cobra to v1.6.0

    Update module github.com/spf13/cobra to v1.6.0

    Mend Renovate

    This PR contains the following updates:

    | Package | Type | Update | Change | |---|---|---|---| | github.com/spf13/cobra | require | minor | v1.5.0 -> v1.6.0 |


    Release Notes

    spf13/cobra

    v1.6.0

    Compare Source

    Summer 2022 Release

    Some exciting changes make their way to Cobra! Command completions continue to get better and better (including adding --help and --version automatic flags to the completions list). Grouping is now possible in your help output as well! And you can now use the OnFinalize method to cleanup things when all "work" is done. Checkout the full changelog below:


    Features 🌠
    Deprecation 👎🏼
    • ExactValidArgs is deprecated (but not being removed entirely). This is abit nuanced, so checkout #​1643 for further information and the updated user_guide.md on how this may affect you (and how you can take advantage of the correct behavior in the validators): @​umarcor #​1643
    Bug fixes 🐛
    Dependencies 🗳️
    Testing 🤔
    Docs ✏️
    Misc 💭

    Note: Per #​1804, we will be moving away from "seasonal" releases and doing more generic point release targets. Continue to track the milestones and issues in the spf13/cobra GitHub repository for more information!

    Great work everyone! Cobra would never be possible without your contributions! 🐍

    Full Changelog: https://github.com/spf13/cobra/compare/v1.5.0...v1.6.0


    Configuration

    📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    🚦 Automerge: Enabled.

    Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    🔕 Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, click this checkbox.

    This PR has been generated by Mend Renovate. View repository job log here.

  • Update golang.org/x/net digest to 8021a29

    Update golang.org/x/net digest to 8021a29

    Mend Renovate

    This PR contains the following updates:

    | Package | Type | Update | Change | |---|---|---|---| | golang.org/x/net | require | digest | bcab684 -> 8021a29 |


    Configuration

    📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

    🚦 Automerge: Enabled.

    Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    🔕 Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, click this checkbox.

    This PR has been generated by Mend Renovate. View repository job log here.

  • Dependency Dashboard

    Dependency Dashboard

    This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

    This repository currently has no open or pending branches.

    Detected dependencies

    github-actions
    .github/workflows/ci.yml
    • actions/checkout v3
    • actions/setup-go v3
    • codecov/codecov-action v3
    gomod
    go.mod
    • github.com/fatih/color v1.13.0
    • github.com/joeguo/tldextract v0.0.0-20220507100122-d83daa6adef8@d83daa6adef8
    • github.com/jpillora/go-tld v1.2.1
    • github.com/karlseguin/intset v1.0.3-0.20221130142345-37ee0d7df651@37ee0d7df651
    • github.com/mjd2021usa/tldextract v0.9.2
    • github.com/spf13/afero v1.9.3
    • github.com/spf13/cobra v1.6.1
    • github.com/tidwall/hashmap v1.8.0
    • golang.org/x/net v0.4.0

    • [ ] Check this box to trigger a request for Renovate to run again on this repository
High-performance minimalist queue implemented using a stripped-down lock-free ringbuffer, written in Go (golang.org)

This project is no longer maintained - feel free to fork the project! gringo A high-performance minimalist queue implemented using a stripped-down loc

Oct 24, 2022
skipmap is a high-performance concurrent sorted map based on skip list. Up to 3x ~ 10x faster than sync.Map in the typical pattern.
skipmap is a high-performance concurrent sorted map based on skip list. Up to 3x ~ 10x faster than sync.Map in the typical pattern.

Introduction skipmap is a high-performance concurrent map based on skip list. In typical pattern(one million operations, 90%LOAD 9%STORE 1%DELETE), th

Jan 8, 2023
A feature complete and high performance multi-group Raft library in Go.
A feature complete and high performance multi-group Raft library in Go.

Dragonboat - A Multi-Group Raft library in Go / 中文版 News 2021-01-20 Dragonboat v3.3 has been released, please check CHANGELOG for all changes. 2020-03

Jan 5, 2023
A Go queue manager on top of Redis

Queue A Go library for managing queues on top of Redis. It is based on a hiring exercise but later I found it useful for myself in a custom task proce

Aug 12, 2022
low level data type and utils in Golang.

low low level data type and utils in Golang. A stable low level function set is the basis of a robust architecture. It focuses on stability and requir

Dec 24, 2022
Package mafsa implements Minimal Acyclic Finite State Automata in Go, essentially a high-speed, memory-efficient, Unicode-friendly set of strings.

MA-FSA for Go Package mafsa implements Minimal Acyclic Finite State Automata (MA-FSA) with Minimal Perfect Hashing (MPH). Basically, it's a set of str

Oct 27, 2022
TLDs finder: check domain name availability across all valid top-level domains

TLD:er TLDs finder — check domain name availability across all valid top-level d

Oct 31, 2022
fast tool for separate existing domains from list of domains using DNS/HTTP.

NETGREP How To Install • How to use Description netgrep can send http/https request or resolve domain from dns (can customize dns server) to separate

Jan 27, 2022
Godaddy-domains-client-go - Godaddy domains api Client golang - Write automaticly from swagger codegen

Go API client for swagger Overview This API client was generated by the swagger-codegen project. By using the swagger-spec from a remote server, you c

Jan 9, 2022
🔑A high performance Key/Value store written in Go with a predictable read/write performance and high throughput. Uses a Bitcask on-disk layout (LSM+WAL) similar to Riak.

bitcask A high performance Key/Value store written in Go with a predictable read/write performance and high throughput. Uses a Bitcask on-disk layout

Sep 26, 2022
the pluto is a gateway new time, high performance, high stable, high availability, easy to use

pluto the pluto is a gateway new time, high performance, high stable, high availability, easy to use Acknowledgments thanks nbio for providing low lev

Sep 19, 2021
🚥 Yet another pinger: A high-performance ICMP ping implementation build on top of BPF technology.

yap Yet-Another-Pinger: A high-performance ICMP ping implementation build on top of BPF technology. yap uses the gopacket library to receive and handl

Nov 9, 2022
top in container - Running the original top command in a container
top in container - Running the original top command in a container

Running the original top command in a container will not get information of the container, many metrics like uptime, users, load average, tasks, cpu, memory, are about the host in fact. topic(top in container) will retrieve those metrics from container instead, and shows the status of the container, not the host.

Dec 2, 2022
Gue is Golang queue on top of PostgreSQL that uses transaction-level locks.

Gue is Golang queue on top of PostgreSQL that uses transaction-level locks.

Jan 4, 2023
Coordinates shutdown of processes with multiple top-level services.

package supervisor Package supervisor coordinates shutdown among multiple services and the OS. Example func main() { m := supervisor.New(context.Back

Dec 12, 2021
Dec 28, 2022
go-fastdfs 是一个简单的分布式文件系统(私有云存储),具有无中心、高性能,高可靠,免维护等优点,支持断点续传,分块上传,小文件合并,自动同步,自动修复。Go-fastdfs is a simple distributed file system (private cloud storage), with no center, high performance, high reliability, maintenance free and other advantages, support breakpoint continuation, block upload, small file merge, automatic synchronization, automatic repair.(similar fastdfs).
go-fastdfs 是一个简单的分布式文件系统(私有云存储),具有无中心、高性能,高可靠,免维护等优点,支持断点续传,分块上传,小文件合并,自动同步,自动修复。Go-fastdfs is a simple distributed file system (private cloud storage), with no center, high performance, high reliability, maintenance free and other advantages, support breakpoint continuation, block upload, small file merge, automatic synchronization, automatic repair.(similar fastdfs).

中文 English 愿景:为用户提供最简单、可靠、高效的分布式文件系统。 go-fastdfs是一个基于http协议的分布式文件系统,它基于大道至简的设计理念,一切从简设计,使得它的运维及扩展变得更加简单,它具有高性能、高可靠、无中心、免维护等优点。 大家担心的是这么简单的文件系统,靠不靠谱,可不

Jan 8, 2023
LinDB is an open-source Time Series Database which provides high performance, high availability and horizontal scalability.
LinDB is an open-source Time Series Database which provides high performance, high availability and horizontal scalability.

LinDB is an open-source Time Series Database which provides high performance, high availability and horizontal scalability. LinDB stores all monitoring data of ELEME Inc, there is 88TB incremental writes per day and 2.7PB total raw data.

Jan 1, 2023
A Golang library for text processing, including tokenization, part-of-speech tagging, and named-entity extraction.

prose is a natural language processing library (English only, at the moment) in pure Go. It supports tokenization, segmentation, part-of-speech tagging, and named-entity extraction.

Jan 4, 2023
A Go port of the Rapid Automatic Keyword Extraction algorithm (RAKE)

A Go implementation of the Rapid Automatic Keyword Extraction (RAKE) algorithm as described in: Rose, S., Engel, D., Cramer, N., & Cowley, W. (2010).

Nov 23, 2022