A corpus of popular Go modules

corpus

A corpus of popular Go modules. See top-100.tsv for the latest sample.

This corpus is to be used when analyzing or studying Go code. For example, when one wants to change the Go language and estimate how much existing code would need to be adapted.

For now, this repository simply contains a table with module information, including where to find the source code and what precise version was recorded. Downloading all the source code is an exercise left to the user, but will likely be provided as part of the program soon. Until then, try go get -d module-path@version in a loop.

Quickstart

Set up a github access token with the public_repo permission, and run:

export GITHUB_TOKEN=...
go run . >output.tsv

FAQ

Measuring popularity is a bit pointless.

Completely agreed. This is just an honest estimation for the purposes of research. It should not be used as a "top 100 best Go modules" leaderboard.

Can't the score be gamed?

In practice, not really. We stick to metrics which require manual work; for example, starring or forking a GitHub repository requires creating an account. You would need to fake that process tens of thousands of times, which likely goes against the site's terms of use.

This list is too GitHub-centric.

I'd love to extend it, for example with gitlab.com and any other popular code hosting sites which have useful statistics like stars/forks. If you know of any sites which qualify and are not yet in the issue tracker, please file an issue.

My project is popular yet it isn't listed.

Note that a Go project must be a Go module and mainly contain Go code in order to be matched by the code hosting site searches. The popularity score is also an estimation, not an objective metric.

If you still think there is a bug in the code, please file a bug.

Owner
Daniel Martí
I work on stuff in Go.
Daniel Martí
Comments
  • CI: github seems to have lowered the GraphQL API rate limits

    CI: github seems to have lowered the GraphQL API rate limits

    Our weekly sync job has been failing for some weeks. See https://github.com/mvdan/corpus/pull/7#issuecomment-842102840.

    Perhaps one fix is to add sleeps between each API request?

    The requests already use a personal access token on my account, so that should be as much API access as one can get without having an enterprise account/setup.

  • Sleep in between API calls to respect ratelimit

    Sleep in between API calls to respect ratelimit

    When calling GraphQL APIs, there's a chance of hitting rate limits 1 or in some cases "secondary rate limits" 2. The score of the query is ~101 and it seems sleep of 1 second in between will be enough to honor the rate limits.

    Fixes #9

  • deduplicate entries

    deduplicate entries

    k8s.io/kops	d8ce157d81e704dfdac0412fcb2b0de74553952d	https://github.com/kubernetes/kops	18757
    k8s.io/kops	d8ce157d81e704dfdac0412fcb2b0de74553952d	https://github.com/kubernetes/kops	18757
    

    On commit https://github.com/mvdan/corpus/commit/1ae32d9009c32dcf3f6759e4c0f8fdc370f4d518 there is a duplicate entry.

  • consider GitHub repositories where the Go module isn't at the root

    consider GitHub repositories where the Go module isn't at the root

    We use GitHub's GraphQL API, which is great for querying 100 repositories at a time while getting the contents of HEAD:go.mod at the same time. This is great because it lets us do the work we need in fewer API calls. GitHub limits API calls to 5k/hr, so one extra API call per repository would be very limiting.

    This brings us a problem, though - HEAD:go.mod simply looks for the go.mod file in the root directory of the default branch. If a repository has the go.mod file elsewhere, such as pkg/go.mod or projectname/go.mod, we won't find it and we'll discard the repository as not being a Go module.

    The solution might be to do an extra API call to do a file search for go.mod files in the repositories which do not contain a root go.mod file. If that results in exactly one go.mod file and it's not part of a vendored module, we use it. This might be okay, given that only a small portion of Go repositories lack a root go.mod file.

  • add a command to easily download the modules from a list

    add a command to easily download the modules from a list

    I think it should also have two modes:

    1. Only download the source for those modules, and nothing else. Useful for very simple analysis, like counting lines of Go code, applying code formatting changes, or grepping for simple patterns.

    2. Download the modules and their dependencies into a module cache, likely with go get -d.

    These modules could be called "shallow" and "module cache". Both should use the module proxy, for the sake of being consistent, fast, and reproducible.

Related tags
GoThanks automatically stars Go's official repository and your go.mod github dependencies, providing a simple way to say thanks to the maintainers of the modules you use and the contributors of Go itself.
GoThanks automatically stars Go's official repository and your go.mod github dependencies, providing a simple way  to say thanks to the maintainers of the modules you use and the contributors of Go itself.

Give thanks (in the form of a GitHub ★) to your fellow Go modules maintainers. About GoThanks performs the following operations Sends a star to Go's r

Dec 24, 2022
将原项目改造成go module项目。GoReporter Use Go Modules.

GoReporter This project is a clone of https://github.com/qax-os/goreporter, use go modules. Installing Requirements go 1.16+ Graphviz go get -u github

Apr 20, 2022
Golang-module-references - A reference for how to setup a Golang project with modules - Task Management + Math Examples

Golang Module Project The purpose of this project is to act as a reference for setting up future Golang projects using modules. This project has a mat

Jan 2, 2022
This example implements a basic example of how to create your own modules, and how to call them from other modules

This example implements a basic example of how to create your own modules, and how to call them from other modules. In addition, an example of how to do unit tests is developed.

Feb 1, 2022
Binaryscarf generates double-knitting patterns for some corpus of input text.

binaryscarf binaryscarf generates double-knit patterns for some corpus of input text. The layout follows the same style as described here. Output is s

Dec 31, 2022
Test your code without writing mocks with ephemeral Docker containers 📦 Setup popular services with just a couple lines of code ⏱️ No bash, no yaml, only code 💻

Gnomock – tests without mocks ??️ Spin up entire dependency stack ?? Setup initial dependency state – easily! ?? Test against actual, close to product

Dec 29, 2022
🐳 A most popular sql audit platform for mysql
🐳 A most popular sql audit platform for mysql

?? A most popular sql audit platform for mysql

Jan 6, 2023
An implementation of the popular game Codenames created with Go and React.

OpenCodenames A real-time implementation of Codenames created with React/TypeScript and Golang. You can play the game here! Installation Stack: React

Aug 8, 2021
A collection of 100+ popular LeetCode problems solved in Go.

go-leetcode A collection of 100+ popular LeetCode problems that I've solved in Go. Each directory includes a: Description with link to LeetCode proble

Dec 25, 2022
fim is a collection of some popular frequent itemset mining algorithms implemented in Go.

fim fim is a collection of some popular frequent itemset mining algorithms implemented in Go. fim contains the implementations of the following algori

Jul 14, 2022
Timeouts for popular Go packages

Go Timeouts An unresponsive service can be worse than a down one. It can tie up your entire system if not handled properly. All network requests shoul

Aug 23, 2022
A simple server which can be used as an RPC endpoint in popular Ethereum wallets.

RPC Endpoint This repository contains code for a simple server which can be used as an RPC endpoint in popular Ethereum wallets. The endpoint is https

Jan 2, 2023
Package implement reading and writing popular playlist formats: PLS, ASX, M3U, XSPF and others.

go-playlist ⚠️ WARNING The API is not stable yet and can change. Package playlist implement reading and writing popular playlist formats: PLS, ASX, M3

Oct 14, 2021
Just Dance Unlimited mock-up server written on Golang and uses a popular Gin framework for Go.

BDCS Just Dance Unlimited mock-up server written on Golang and uses a popular Gin framework for Go. Features Security Authorization works using UbiSer

Nov 10, 2021
Go serverless functions examples with most popular Cloud Providers

go-serverless Go serverless functions examples with most popular Cloud Providers Creating zip archive go mod download go build ./cmd/<aws|gcp> zip -

Nov 16, 2021
Vale-compatible implementations of many popular "readability" metrics.

Readability This repository contains a Vale-compatible implementation of many popular "readability" metrics. Getting Started ❗ Readability requires Va

Aug 31, 2022
Examples of using various popular database libraries and ORM in Go.

Introduction Examples of using various popular database libraries and ORM in Go. sqlx sqlc Gorm sqlboiler ent The aim is to demonstrate and compare us

Dec 12, 2021
Examples of using various popular database libraries and ORM in Go.

Introduction Examples of using various popular database libraries and ORM in Go. sqlx sqlc Gorm sqlboiler ent The aim is to demonstrate and compare us

Dec 28, 2022
Go-fastapi: a library to quickly build APIs. It is inspired by Python's popular FastAPI
Go-fastapi: a library to quickly build APIs. It is inspired by Python's popular FastAPI

go-fastapi go-fastapi is a library to quickly build APIs. It is inspired by Pyth

Jan 1, 2023
Implementation of a popular graphics benchmark written on Ebiten.
Implementation of a popular graphics benchmark written on Ebiten.

Ebiten Bunny Mark This is an implementation of the popular graphics benchmark written on Ebiten. The initial benchmark was created by Ian Lobb (code)

Dec 7, 2022