7 days golang programs from scratch (web framework Gee, distributed cache GeeCache, object relational mapping ORM framework GeeORM, rpc framework GeeRPC etc) 7天用Go动手写/从零实现系列

7 days golang programs from scratch

CodeSize LICENSE

README 中文版本

7天用Go从零实现系列

7天能写什么呢?类似 gin 的 web 框架?类似 groupcache 的分布式缓存?或者一个简单的 Python 解释器?希望这个仓库能给你答案。

推荐先阅读 Go 语言简明教程,一篇文章了解Go的基本语法、并发编程,依赖管理等内容。

推荐 Go 语言笔试面试题,加深对 Go 语言的理解。

推荐 Go 语言高性能编程(项目地址),写出高性能的 Go 代码。

期待关注我的「知乎专栏」和「微博」,查看最近的文章和动态。

7天用Go从零实现Web框架 - Gee

Gee 是一个模仿 gin 实现的 Web 框架,Go Gin简明教程可以快速入门。

7天用Go从零实现分布式缓存 GeeCache

GeeCache 是一个模仿 groupcache 实现的分布式缓存系统

7天用Go从零实现ORM框架 GeeORM

GeeORM 是一个模仿 gormxorm 的 ORM 框架

gorm 准备推出完全重写的 v2 版本(目前还在开发中),相对 gorm-v1 来说,xorm 的设计更容易理解,所以 geeorm 接口设计上主要参考了 xorm,一些细节实现上参考了 gorm。

7天用Go从零实现RPC框架 GeeRPC

GeeRPC 是一个基于 net/rpc 开发的 RPC 框架 GeeRPC 是基于 Go 语言标准库 net/rpc 实现的,添加了协议交换、服务注册与发现、负载均衡等功能,代码约 1k。

WebAssembly 使用示例

具体的实践过程记录在 Go WebAssembly 简明教程

  • 示例一:Hello World | Code
  • 示例二:注册函数 | Code
  • 示例三:操作 DOM | Code
  • 示例四:回调函数 | Code

What can be accomplished in 7 days? A gin-like web framework? A distributed cache like groupcache? Or a simple Python interpreter? Hope this repo can give you the answer.

Web Framework - Gee

Gee is a gin-like framework

  • Day 1 - http.Handler Interface Basic Code
  • Day 2 - Design a Flexiable Context Code
  • Day 3 - Router with Trie-Tree Algorithm Code
  • Day 4 - Group Control Code
  • Day 5 - Middleware Mechanism Code
  • Day 6 - Embeded Template Support Code
  • Day 7 - Panic Recover & Make it Robust Code

Distributed Cache - GeeCache

GeeCache is a groupcache-like distributed cache

  • Day 1 - LRU (Least Recently Used) Caching Strategy Code
  • Day 2 - Single Machine Concurrent Cache Code
  • Day 3 - Launch a HTTP Server Code
  • Day 4 - Consistent Hash Algorithm Code
  • Day 5 - Communication between Distributed Nodes Code
  • Day 6 - Cache Breakdown & Single Flight | Code
  • Day 7 - Use Protobuf as RPC Data Exchange Type | Code

Object Relational Mapping - GeeORM

GeeORM is a gorm-like and xorm-like object relational mapping library

Xorm's desgin is easier to understand than gorm-v1, so the main designs references xorm and some detailed implementions references gorm-v1.

  • Day 1 - database/sql Basic | Code
  • Day 2 - Object Schame Mapping | Code
  • Day 3 - Insert and Query | Code
  • Day 4 - Chain, Delete and Update | Code
  • Day 5 - Support Hooks | Code
  • Day 6 - Support Transaction | Code
  • Day 7 - Migrate Database | Code

RPC Framework - GeeRPC

GeeRPC is a net/rpc-like RPC framework

Based on golang standard library net/rpc, GeeRPC implements more features. eg, protocol exchange, service registration and discovery, load balance, etc.

  • Day 1 - Server Message Codec | Code
  • Day 2 - Concurrent Client | Code
  • Day 3 - Service Register | Code
  • Day 4 - Timeout Processing | Code
  • Day 5 - Support HTTP Protocol | Code
  • Day 6 - Load Balance | Code
  • Day 7 - Discovery and Registry | Code

Golang WebAssembly Demo

  • Demo 1 - Hello World Code
  • Demo 2 - Register Functions Code
  • Demo 3 - Manipulate DOM Code
  • Demo 4 - Callback Code
Comments
  • 类似var _ PeerPicker = (*HTTPPool)(nil)这种设计目的是什么

    类似var _ PeerPicker = (*HTTPPool)(nil)这种设计目的是什么

    首先感谢博主能写出这么好的文章,受益匪浅 然后就是在GeeCache中我看到有类似于 var _ PeerPicker = (*HTTPPool)(nil) 和 var _ PeerGetter = (*httpGetter)(nil) 请问这种设计的意义是什么,我把这一行代码去了也并没有看到报错,有点疑惑,向博主请教

  • 在模糊匹配下再次插入节点到同样树高,会被替换掉 handler

    在模糊匹配下再次插入节点到同样树高,会被替换掉 handler

    https://github.com/geektutu/7days-golang/blob/398679493db818265551707db70969a0e9e64529/gee-web/day3-router/gee/trie.go#L27

    maybe:

        if child == nil || child.isWild {
    
  • day4 启动报错 panic: runtime error: invalid memory address or nil pointer dereference

    day4 启动报错 panic: runtime error: invalid memory address or nil pointer dereference

    day4 启动报错:

    panic: runtime error: invalid memory address or nil pointer dereference
    [signal 0xc0000005 code=0x0 addr=0x8 pc=0x66dfda]
    
    goroutine 1 [running]:
    gee/gee.getNestPrefix(0x0, 0x0, 0x0, 0x8, 0x6a3440)
            E:/go_project/src/gee/gee/gee.go:88 +0x3a
    gee/gee.(*RouterGroup).addRoute(0xc0000562c0, 0x711e2d, 0x3, 0x7123d1, 0x6, 0x72
    ab98)
            E:/go_project/src/gee/gee/gee.go:63 +0x4b
    gee/gee.(*RouterGroup).GET(...)
            E:/go_project/src/gee/gee/gee.go:69
    main.main()
            E:/go_project/src/gee/main.go:45 +0x185
    

    在 gee.go 添加以下方法后就可以了:

    func (engine *Engine) addRoute(method string, pattern string, handler HandlerFunc) {
    	engine.router.addRoute(method, pattern, handler)
    }
    
    // GET defines the method to add GET request
    func (engine *Engine) GET(pattern string, handler HandlerFunc) {
    	engine.addRoute("GET", pattern, handler)
    }
    
    // POST defines the method to add POST request
    func (engine *Engine) POST(pattern string, handler HandlerFunc) {
    	engine.addRoute("POST", pattern, handler)
    }
    

    但是在 day5 没有这三个方法也正常运行,请问为什么呢? 还有非常喜欢大佬的教程,就是 day3 的 trie 树目前也不是特别懂

  • 为什么cache需要设计成自动回源的?

    为什么cache需要设计成自动回源的?

    你好,我最近在看geeCache的实现,有些不明白地方想请教一下

    // Get value for a key from cache
    func (g *Group) Get(key string) (ByteView, error) {
    	if key == "" {
    		return ByteView{}, fmt.Errorf("key is required")
    	}
    
    	if v, ok := g.mainCache.get(key); ok {
    		log.Println("[GeeCache] hit")
    		return v, nil
    	}
    
    	return g.load(key)
    }
    

    比如这段函数的末尾,在cache未命中的情况下,会自己fallback去load缓存

    我没有理解这里的逻辑,为什么缓存需要自己去fallbak?难道不应该直接返回未命中么? 如果涉及成这样,那缓存岂不是永远是命中的?

  • Go 小白问一个环境问题,google 没找到很好的解决方案

    Go 小白问一个环境问题,google 没找到很好的解决方案

    gee-web day1 的 base3 run main.go 遇到找不到 package gee 的问题:

    373356eec6e8f4fc4b39b2e05fb0905

    这是我的 go env:

    image

    google 了一下,好像要把当前项目的路径加到 gopath 去,但这样好像有点蠢(如果每个导入自己写的 package 都要这么做的话)? 请问有什么优雅的解决方案吗?

  • syscall/js报错

    syscall/js报错

    额,我用的goland在跑hello-world里的main.go,报了这个错,我搜了一下需要修改GOOS和GOARCH,但是没有搜到怎么修改这两个变量,求解决。 package command-line-arguments imports syscall/js: build constraints exclude all Go files in D:\Go\src\syscall\js

  • 中间件那部分的main.go中OnlyforV2方法

    中间件那部分的main.go中OnlyforV2方法

    func onlyForV2() gwebframe.HandlerFunc {
    	return func(c *gwebframe.Context) {
    		// Start timer
    		t := time.Now()
    		// if a server error occurred
    		//c.Fail(500, "Internal Server Error")
    		// Calculate resolution time
    		log.Printf("[%d] %s in %v for group v2", c.StatusCode, c.Req.RequestURI, time.Since(t))
    	}
    }
    

    加入了c.Fail会导致调用onlyForV2时必然报错

  • middlewares: remove the deformed anonymous function and the stupid in…

    middlewares: remove the deformed anonymous function and the stupid in…

    r.Use(gee.Logger())
    func Logger() HandlerFunc {
        return func(c *Context) {
            // ...        
        }
    }
    
    r.Use(gee.Logger)
    func Logger(c *Context) {
        // ...
    }
    

    我理解 go 的隐式接口实现在阅读上会有一定的障碍,显示返还接口的形式使得可读性在某些方面得到一定的缓解,但是无用的匿名函数和冗余的缩进严重的提高了程序复杂性。 您这边的教学对 golang 入门用户有深远的影响,但您知道一个中间件内,内嵌了一个三千行的匿名函数有多么的可怕么?

  • gee-web_Possible_problems

    gee-web_Possible_problems

    兔兔大佬好! 在第一次学习兔兔大佬的gee-web课程的时候有这些疑惑,在第二次学习并查阅一些资料后对我的疑惑以及评论的一些疑惑进行了解答,希望能够帮到大家!

    http.HandlerFunc和http.Handle的区别?

    为什么只要传入任何实现了 ServerHTTP接口的实例,所有的HTTP请求,就都会交给了该实例处理?

    go语言中原生的一个web服务器启动的流程?

    为什么我们自己定义的handler会比原始的快呢?原生的serverHttp是怎么实现的?

  • day1-rpc: 服务端是如何识别Option和后续内容的界限? 不同的Option编码结果的字节数并不相等

    day1-rpc: 服务端是如何识别Option和后续内容的界限? 不同的Option编码结果的字节数并不相等

    文中提到 动手写RPC框架 - GeeRPC第一天 服务端与消息编码

    一般来说,涉及协议协商的这部分信息,需要设计固定的字节来传输的。
    但是为了实现上更简单,GeeRPC 客户端固定采用 JSON 编码 Option,后续的 header 和 body 的编码方式由 Option 中的 CodeType 指定,服务端首先使用 JSON 解码 Option,然后通过 Option 得 CodeType 解码剩余的内容。
    

    什么叫固定字节来传输呢? 我理解下面的两个结构体,序列化之后的字节数目应该是不等的。

    {MagicNumber:3927900 CodecType:application/gob}
    {MagicNumber:3927900 CodecType:application/json}
    

    代码验证, 第一个结构体编码后54字节,第二个结构体编码后55字节。 所以我有个疑问哈, 服务端是如何识别Option和后续消息体的边界的呢?

    package main
    
    import (
    	"bytes"
    	"encoding/json"
    	"fmt"
    )
    
    type Type string
    
    const (
    	GobType  Type = "application/gob"
    	JsonType Type = "application/json"
    )
    
    const MagicNumber = 0x3bef5c
    
    type Option struct {
    	MagicNumber int
    	CodecType   Type
    }
    
    var DefaultOption = &Option{
    	MagicNumber: MagicNumber,
    	CodecType:   GobType,
    }
    
    func main() {
    	var buf bytes.Buffer
    	// send options
    	_ = json.NewEncoder(&buf).Encode(&Option{
    		MagicNumber: MagicNumber,
    		CodecType:   GobType,
    	})
    	fmt.Println(buf.Bytes())
    	fmt.Println(buf.Len())
    	fmt.Println(buf.String())
    
    	buf.Reset()
    	_ = json.NewEncoder(&buf).Encode(&Option{
    		MagicNumber: MagicNumber,
    		CodecType:   JsonType,
    	})
    
    	fmt.Println(buf.Bytes())
    	fmt.Println(buf.Len())
    	fmt.Println(buf.String())
    }
    
    // Output
    //[123 34 77 97 103 105 99 78 117 109 98 101 114 34 58 51 57 50 55 57 48 48 44 34 67 111 100 101 99 84 121 112 101 34 58 34 97 112 112 108 105 99 97 116 105 111 110 47 103 111 98 34 125 10]
    //54
    //{"MagicNumber":3927900,"CodecType":"application/gob"}
    //
    //[123 34 77 97 103 105 99 78 117 109 98 101 114 34 58 51 57 50 55 57 48 48 44 34 67 111 100 101 99 84 121 112 101 34 58 34 97 112 112 108 105 99 97 116 105 111 110 47 106 115 111 110 34 125 10]
    //55
    //{"MagicNumber":3927900,"CodecType":"application/json"}
    
  • [ImgBot] Optimize images

    [ImgBot] Optimize images

    Beep boop. Your images are optimized!

    Your image file size has been reduced by 13% 🎉

    Details

    | File | Before | After | Percent reduction | |:--|:--|:--|:--| | /gee-rpc/doc/geerpc-day5/geerpc_debug.png | 5.69kb | 3.86kb | 32.08% | | /gee-cache/doc/geecache-day4/hash.jpg | 21.63kb | 18.50kb | 14.46% | | /gee-cache/doc/geecache-day5/dist_nodes.jpg | 21.31kb | 18.29kb | 14.14% | | /gee-cache/doc/geecache-day7/protobuf.jpg | 20.87kb | 17.97kb | 13.89% | | /gee-cache/doc/geecache-day6/singleflight.jpg | 22.16kb | 19.16kb | 13.55% | | /gee-cache/doc/geecache-day6/singleflight_logo.jpg | 9.13kb | 8.17kb | 10.52% | | /gee-cache/doc/geecache-day4/hash_logo.jpg | 9.86kb | 8.87kb | 10.01% | | /gee-cache/doc/geecache-day7/protobuf_logo.jpg | 9.16kb | 8.25kb | 9.85% | | /gee-cache/doc/geecache-day5/dist_nodes_logo.jpg | 10.54kb | 9.52kb | 9.71% | | /gee-cache/doc/geecache-day1/lru_logo.jpg | 5.15kb | 5.10kb | 0.89% | | | | | | | Total : | 135.49kb | 117.71kb | 13.13% |


    Black Lives Matter | 💰 donate | 🎓 learn | ✍🏾 sign

    📝 docs | :octocat: repo | 🙋🏾 issues | 🏅 swag | 🏪 marketplace

  • gee-web项目中  路由注册顺序,导致的Bug

    gee-web项目中 路由注册顺序,导致的Bug

    func TestGetRoute(t *testing.T) {
    	r := newRouter()
    	r.addRoute("GET", "/", nil)
    	r.addRoute("GET", "/hello/:id", nil)
    	r.addRoute("GET", "/hello/b/c", nil)
    	n, ps := r.getRoute("GET", "/hello/21/c")
    	fmt.Printf("n: %v\n", n)  // n: node{pattern=/hello/b/c, part=c, isWild=false}
    	fmt.Printf("ps: %v\n", ps)//ps: map[]
    }
    

    我们期望得到 n == nil, ps == nil , 但实际上却匹配到了静态路由 pattern=/hello/b/c.

    由于注册顺序导致的 路由树可以简化为: [0] root: path: / [1] path: hello [2] path: :id pattern : "/hello/:id" [3] path: c pattern : "hello/b/c"

    主要原因为 先注册的 路由为 /hello/:id , 后注册的路由为 /hello/b/c , 在查找时,没有返回 nil, 源代码如下:

    func (n *node) matchChild(part string) *node {
    	for _, child := range n.children {
            if child.part == part || child.isWild {  // false,  true
                // 在查找 part为 :id 这一层时, 直接返回了child 
    			return child
    		}
    	}
    	return nil
    }
    func (n *node) insert(pattern string, parts []string, height int) {
    	if len(parts) == height {
    		n.pattern = pattern
    		return
    	}
    
        part := parts[height]
        child := n.matchChild(part) // 由于返回的不是 nil , 而是  node.part == ":id" 这个节点。
    	if child == nil {
    		child = &node{part: part, isWild: part[0] == ':' || part[0] == '*'}
    		n.children = append(n.children, child)
    	}
        
    	child.insert(pattern, parts, height+1)
    }
    

    要想达到预期 结果 n == nil, ps == nil , 就只能修改注册顺序。

    先注册静态路由, 确保 matchChildren(part string) []*node  方法返回的切片 包含 child.part="b" 的这个节点。
    r.addRoute("GET", "/hello/b/c", nil)
    r.addRoute("GET", "/hello/:id", nil)
    
  • geecache day4 一致性哈希疑问

    geecache day4 一致性哈希疑问

    尊敬的作者,您好: 在学习您的geecache day4 一致性哈希章节中针对该算法有个疑问,“环上有 peer2,peer4,peer6 三个节点,key11,key2,key27 均映射到 peer2,key23 映射到 peer4。” 这个部分不太理解,当前环上有三个节点的话,key11映射的节点为 11%3=2顺时针映射peer2,key2映射的节点应该 2%3=2顺时针映射peer2,key27映射的节点为27%3=0顺时针映射peer2,key23映射的节点不也应该为23%3=2 peer2 吗? 这个地方一直没搞明白,望您有空点拨下。万分感谢。

  • http,tcp,rpc

    http,tcp,rpc

    想问下http的rpc和tcp的rpc有啥区别。 看完http rpc,感觉流程是通过向server(给server发一个connect的http请求),得到client(包含了tcp conn),然后就是发rpc数据。 服务端:实现serveHTTP方法,然后就是handleHTTP,进入serveHTTP,先检查http请求是不是connect,然后就到了原来的serveConn, 就感觉这http和tcp的rpc没有啥区别,希望大佬可以解答一下。

TinySQL is a course designed to teach you how to implement a distributed relational database in Go

TinySQL TinySQL is a course designed to teach you how to implement a distributed relational database in Go. TinySQL is also the name of the simplifed

Nov 7, 2021
100 days of Go learning
100 days of Go learning

This repository is a journal of my path to learning GO. By the end of the 100 days, you should be able to follow along by day and learn Go as well.

Oct 27, 2022
Golang - A collection of small Go programs used as learning exercises

This repo is a collection of small Go programs used as learning exercises Some o

Dec 31, 2021
A complete guide to undersatnd golang programming language, web requests, JSON and creating web APIs with mongodb

Golang series A complete guide to undersatnd golang programming language, web requests, JSON and creating web APIs with mongodb LearnCodeonline.in 01

Jan 1, 2023
📁 Examples for 🚀 Fiber - Express inspired web framework written in Go

?? Examples for ?? Fiber - Express inspired web framework written in Go

Dec 29, 2022
A course to build distributed key-value service based on TiKV model
A course to build distributed key-value service based on TiKV model

The TinyKV Course This is a series of projects on a key-value storage system built with the Raft consensus algorithm

Jan 7, 2023
Labs from MIT's graduate-level Distributed Systems course

Labs from MIT's graduate-level Distributed Systems course Course website here Lab 1: MapReduce Lab 2: Raft Consensus Algorithm Lab 2A: Raft Leader Ele

Jun 20, 2022
Web programming tutorial with Golang

Tutorial de programação Web com Golang Para rodar o servidor Entre na pasta web_app, onde está o main.go cd caminho/para/pasta/web_app Agora execute

Jan 19, 2022
A basic helloworld golang web program , just for personal use

Standard Go Project Layout Overview This is a basic helloworld golang web progra

Aug 6, 2022
This is my first golang project. The main reason for its existence is the need for practice. I will be studying golang while writing this project

My first GoLang project Project Aim The goal of this project is to develop the most simple golang bot to learn how to work with this programming langu

Jan 7, 2022
Learn how to write webapps without a framework in Go.

This is an easy to understand example based tutorial aimed at those who know a little of Go and nothing of webdev and want to learn how to write a webserver in Go. You will create a to do list application as you advance the chapters.

Dec 28, 2022
roguelike tutorial in Go using the framework gruid

Gruid Go Roguelike Tutorial This tutorial follows the overall structure of the TCOD Python Tutorial, but makes use of the Go programming language and

Jul 25, 2022
Boilerplate for writing Go applications without framework using hexagonal application development approach
Boilerplate for writing Go applications without framework using hexagonal application development approach

Boilerplate for writing Go applications without framework using hexagonal application development approach

Dec 2, 2022
This project is a GO Restful API service with Gin framework and Gorm SQLite with authorization

GO Restful API service with Gin framework and Gorm SQLite Template Structure Gin is a web framework written in Go (Golang). It features a martini-like

Oct 24, 2022
Golang 设计模式
Golang 设计模式

Golang设计模式思想 前言 一切设计模式都是灵活应用struct的组合模式,以及go隐形继承接口的特性 go中的interface就是一些方法装饰, 而struct并不依赖于接口 设计模式类型 创建模式 建造者模式(Builder Pattern) 将一个复杂对象的构建与它的表示分离, 使得同样

Dec 29, 2022
Examples of Golang compared to Node.js for learning
Examples of Golang compared to Node.js for learning

This guide full of examples is intended for people learning Go that are coming from Node.js, although the vice versa can work too. This is not meant to be a complete guide and it is assumed that you've gone through the Tour of Go tutorial. This guide is meant to be barely good enough to help you at a high level understand how to do X in Y and doing further learning on your own is of course required.

Jan 9, 2023
Golang && Back-end Stack , Continually updated

Stack Here Golang OS Net Note Tool Delve Authentication Paper Paper ?? Link GO官方文档 GO Official Doc GO官方博客 GO Official Blog GO官方仓库

Nov 8, 2022
The Ultimate Workshop Track for #golang Developer
The Ultimate Workshop Track  for #golang Developer

layout title nav_order description permalink default An Ultimate GopherLabs Hands-on Labs 1 An Ultimate GopherLabs Hands-on Labs / Join GopherLabs Com

Dec 7, 2022
high performance coding with golang(Go 语言高性能编程,Go 语言陷阱,Gotchas,Traps)
high performance coding with golang(Go 语言高性能编程,Go 语言陷阱,Gotchas,Traps)

Go 语言高性能编程 订阅 最新动态可以关注:知乎 Go语言 或微博 极客兔兔 订阅方式:watch geektutu/blog ,每篇文章都能收到邮件通知,或通过 RSS 订阅。

Dec 28, 2022