Database access layer for golang

grimoire

GoDoc Build Status Go Report Card Maintainability Test Coverage FOSSA Status

⚠️ Grimoire V2 is available as REL and Changeset package.

Grimoire is a database access layer inspired by Ecto. It features a flexible query API and built-in validation. It currently supports MySQL, PostgreSQL, and SQLite3 but a custom adapter can be implemented easily using the Adapter interface.

Features:

  • Query Builder
  • Association Preloading
  • Struct style create and update
  • Changeset Style create and update
  • Builtin validation using changeset
  • Multi adapter support
  • Logger

Motivation

Common go ORM accepts struct as a value for modifying records which has a problem of unable to differentiate between an empty, nil, or undefined value. It's a tricky problem especially when you want to have an endpoint that supports partial updates. Grimoire attempts to solve that problem by integrating Changeset system inspired from Elixir's Ecto. Changeset is a form like entity which allows us to not only solve that problem but also help us with casting, validations, and constraints check.

Install

go get github.com/Fs02/grimoire

Quick Start

package main

import (
	"time"

	"github.com/Fs02/grimoire"
	"github.com/Fs02/grimoire/adapter/mysql"
	"github.com/Fs02/grimoire/changeset"
	"github.com/Fs02/grimoire/params"
)

type Product struct {
	ID        int
	Name      string
	Price     int
	CreatedAt time.Time
	UpdatedAt time.Time
}

// ChangeProduct prepares data before database operation.
// Such as casting value to appropriate types and perform validations.
func ChangeProduct(product interface{}, params params.Params) *changeset.Changeset {
	ch := changeset.Cast(product, params, []string{"name", "price"})
	changeset.ValidateRequired(ch, []string{"name", "price"})
	changeset.ValidateMin(ch, "price", 100)
	return ch
}

func main() {
	// initialize mysql adapter.
	adapter, err := mysql.Open("root@(127.0.0.1:3306)/db?charset=utf8&parseTime=True&loc=Local")
	if err != nil {
		panic(err)
	}
	defer adapter.Close()

	// initialize grimoire's repo.
	repo := grimoire.New(adapter)

	var product Product

	// Inserting Products.
	// Changeset is used when creating or updating your data.
	ch := ChangeProduct(product, params.Map{
		"name":  "shampoo",
		"price": 1000,
	})

	if ch.Error() != nil {
		// handle error
	}

	// Changeset can also be created directly from json string.
	jsonch := ChangeProduct(product, params.ParseJSON(`{
		"name":  "soap",
		"price": 2000,
	}`))

	// Create products with changeset and return the result to &product,
	if err = repo.From("products").Insert(&product, ch); err != nil {
		// handle error
	}

	// or panic when insertion failed
	repo.From("products").MustInsert(&product, jsonch)

	// Querying Products.
	// Find a product with id 1.
	repo.From("products").Find(1).MustOne(&product)

	// Updating Products.
	// Update products with id=1.
	repo.From("products").Find(1).MustUpdate(&product, ch)

	// Deleting Products.
	// Delete Product with id=1.
	repo.From("products").Find(1).MustDelete()
}

Examples

Documentation

Guides: https://fs02.github.io/grimoire

API Documentation: https://godoc.org/github.com/Fs02/grimoire

License

Released under the MIT License

FOSSA Status

Owner
Comments
  • [Potential Bug] Changeset.ValidateRequired doesn't produce error for required field

    [Potential Bug] Changeset.ValidateRequired doesn't produce error for required field

    Bug Description

    ValidateRequired doesn't produce error even though required field is empty

    Steps to reproduce

    https://play.golang.org/p/9soJDMn9B8t

    Expected result

    ch.Errors() should be [ktp_direksi is required, tdp is required, sk_domisili is required, akta_pendirian is required, akta_perubahan is required]

    Actual result

    ch.Errors() is [] (empty)

    Possible root cause

    • For this to happen, we need to set some fields in the entity. If some fields in the entity has been set to non empty value, ch.zero will be false.
    • If the required field is not in the changeset and ch.zero is false, it will check if the required field exists in ch.values
    • Required fields exists in ch.values, but it only checks for empty string and nil value.
    • Since the required fields (e.g sk_domisili) has fichaURL as its type, it will check for nil value. Since it's not nil, it will continue and skip the loop without adding error to the changeset https://github.com/Fs02/grimoire/blob/master/changeset/validate_required.go#L27

    Possible solution

    • Change nil check to zero value check with IsZero https://github.com/Fs02/grimoire/blob/master/changeset/validate_required.go#L27
  • Quering Relationships

    Quering Relationships

    Thanks guys for sharing this package, very intuitive api in most cases with exception for relationships...

    More of the question than issue I hope:

    So I have relationship one to many:

    Tests Table:

    CREATE TABLE tests (
        id INT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
        title TEXT NOT NULL,
        created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
        updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
    );
    

    Tags Table:

    CREATE TABLE tags (
        id INT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
        test_id INT UNSIGNED,
        title TEXT NOT NULL,
        created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
        updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
        FOREIGN KEY (test_id)
            REFERENCES tests(id)
            ON DELETE CASCADE
    );
    

    Here are the structs:

    type Test struct {
    	ID        int
    	Title     string
    	Tags      []Tag `references:"ID" foreign_key:"TestID"`
    	CreatedAt string
    	UpdatedAt string
    }
    
    type Tag struct {
    	ID        int
    	TestID    int
    	Title     string
    	CreatedAt string
    	UpdatedAt string
    }
    

    My question is how do I query all the tests with all assosiated tags. I tried all posibilities and I can not figure it out how to query relationships. Always geting null for the tags... Any help would be appriciated.

    {
    "status": "OK",
    "data": [
    {
    "ID": 1,
    "Title": "test 1",
    "Tags": null,
    "CreatedAt": "2019-08-19 23:18:24",
    "UpdatedAt": "2019-08-19 23:18:24"
    },
    {
    "ID": 1,
    "Title": "test 1",
    "Tags": null,
    "CreatedAt": "2019-08-19 23:18:24",
    "UpdatedAt": "2019-08-19 23:18:24"
    },
    {
    "ID": 1,
    "Title": "test 1",
    "Tags": null,
    "CreatedAt": "2019-08-19 23:18:24",
    "UpdatedAt": "2019-08-19 23:18:24"
    },
    etc.
    
  • Bump github.com/tidwall/gjson from 1.1.3 to 1.6.5

    Bump github.com/tidwall/gjson from 1.1.3 to 1.6.5

    Bumps github.com/tidwall/gjson from 1.1.3 to 1.6.5.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

  • Many To Many Relationship (pivot table)

    Many To Many Relationship (pivot table)

    Hello,

    I would like to say thank you for helping out last time. Unfortunetly I got stuck again.

    I am trying to query many to many realtionship.

    1. Post table

    id title

    1. Pivot Table

    id post_id tag_id

    1. Tag Table id title

    Tables in the db are named as: posts, post_tag and tags. As far, this naming convention was working great for preloading has many, has one or belongs to. Can't figure it out how do I query posts with all the tags. Tried joins with preload but always get a panic mentioning that PostID or TagID field is not found...

  • Add license scan report and status

    Add license scan report and status

    Your FOSSA integration was successful! Attached in this PR is a badge and license report to track scan status in your README.

    Below are docs for integrating FOSSA license checks into your CI:

  • Bump github.com/tidwall/gjson from 1.6.5 to 1.9.3

    Bump github.com/tidwall/gjson from 1.6.5 to 1.9.3

    Bumps github.com/tidwall/gjson from 1.6.5 to 1.9.3.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

  • Bypass Validation for Setting Changes with Empty Values

    Bypass Validation for Setting Changes with Empty Values

    Issue: The validation is still running when setting a field with empty values using changeset.EmptyValues(..) options, giving a validation error.

    This is an issue when a field is intentionally set to an empty value.

  • Fix ValidateRequired: Don't check empty value with isZero

    Fix ValidateRequired: Don't check empty value with isZero

    There's an issue in ValidateRequired when using changeset with default value. See the example here: https://play.golang.org/p/IFXG3A6s9ey

    Since it checks changeset.changes with isZero, some_bool field is considered empty because its value is false.

    The solution is to revert zero value check with isZero. To compare with default value, implement isZeroer interface instead.

    note: This changes has been tested in papyrus repository and all tests pass.

  • Fix PutDefault

    Fix PutDefault

    After https://github.com/Fs02/grimoire/pull/57 has merged, I found that there's still an mishandled case in put default, which is when the existing value is zero and the input for that field is also zero value. (Ex: A field that hasn't been filled previously and want to be filled with a zero value)

    That field gets a default value put into it, when the input parameters tell it to fill it with zero

  • Update PutDefault

    Update PutDefault

    Update PutDefault Method Put default value only if there's no change and no existing value.

    Problem:

    • changeset.Cast will give empty changes when given input with same value as existing value, triggering the PutDefault
  • Internal improvements

    Internal improvements

    • Cache some functions that use reflection.
    • Ability to avoid reflection on some cases by implementing inferfaces:
    // Defines fields in a struct, used mainly in changeset.
    type fields interface {
    	Fields() map[string]int
    }
    
    // Defines type of fields in a struct, used mainly in changeset.
    type types interface {
    	fields
    	Types() []reflect.Type
    }
    
    // Defines values of fields in a struct, used mainly in changeset.
    type values interface {
    	fields
    	Values() []interface{}
    }
    
    // Defines database scanners, used when scanning result from database rows.
    type scanners interface {
    	Scanners([]string) []interface{}
    }
    
igor is an abstraction layer for PostgreSQL with a gorm like syntax.

igor igor is an abstraction layer for PostgreSQL, written in Go. Igor syntax is (almost) compatible with GORM. When to use igor You should use igor wh

Jan 1, 2023
Go Postgres Data Access Toolkit

dat GoDoc dat (Data Access Toolkit) is a fast, lightweight Postgres library for Go. Focused on Postgres. See Insect, Upsert, SelectDoc, QueryJSON Buil

Dec 20, 2022
A Go (golang) package that enhances the standard database/sql package by providing powerful data retrieval methods as well as DB-agnostic query building capabilities.

ozzo-dbx Summary Description Requirements Installation Supported Databases Getting Started Connecting to Database Executing Queries Binding Parameters

Dec 31, 2022
Go database query builder library for PostgreSQL

buildsqlx Go Database query builder library Installation Selects, Ordering, Limit & Offset GroupBy / Having Where, AndWhere, OrWhere clauses WhereIn /

Dec 23, 2022
Zero boilerplate database operations for Go
Zero boilerplate database operations for Go

(Now compatible with MySQL and PostgreSQL!) Everyone knows that performing simple DATABASE queries in Go takes numerous lines of code that is often re

Dec 17, 2022
open source training courses about distributed database and distributed systemes
open source training courses about distributed database and distributed systemes

Welcome to learn Talent Plan Courses! Talent Plan is an open source training program initiated by PingCAP. It aims to create or combine some open sour

Jan 3, 2023
Bulk query SQLite database over the network

SQLiteQueryServer Bulk query SQLite database over the network. Way faster than SQLiteProxy!

May 20, 2022
LBADD: An experimental, distributed SQL database
LBADD: An experimental, distributed SQL database

LBADD Let's build a distributed database. LBADD is an experimental distributed SQL database, written in Go. The goal of this project is to build a dat

Nov 29, 2022
Mocking your SQL database in Go tests has never been easier.

copyist Mocking your SQL database in Go tests has never been easier. The copyist library automatically records low-level SQL calls made during your te

Dec 19, 2022
Additions to Go's database/sql for super fast performance and convenience. (fork of gocraft/dbr)

dbr (fork of gocraft/dbr) provides additions to Go's database/sql for super fast performance and convenience. Getting Started // create a connection (

Dec 31, 2022
Go library for accessing multi-host SQL database installations

hasql hasql provides simple and reliable way to access high-availability database setups with multiple hosts. Status hasql is production-ready and is

Dec 28, 2022
Document-oriented, embedded SQL database

Genji Document-oriented, embedded, SQL database Table of contents Table of contents Introduction Features Installation Usage Using Genji's API Using d

Jan 1, 2023
Generate a Go ORM tailored to your database schema.
Generate a Go ORM tailored to your database schema.

SQLBoiler is a tool to generate a Go ORM tailored to your database schema. It is a "database-first" ORM as opposed to "code-first" (like gorm/gorp). T

Jan 9, 2023
A Golang library for using SQL.

dotsql A Golang library for using SQL. It is not an ORM, it is not a query builder. Dotsql is a library that helps you keep sql files in one place and

Dec 27, 2022
a golang library for sql builder

Gendry gendry is a Go library that helps you operate database. Based on go-sql-driver/mysql, it provides a series of simple but useful tools to prepar

Dec 26, 2022
SQL builder and query library for golang

__ _ ___ __ _ _ _ / _` |/ _ \ / _` | | | | | (_| | (_) | (_| | |_| | \__, |\___/ \__, |\__,_| |___/ |_| goqu is an expressive SQL bu

Dec 30, 2022
Fluent SQL generation for golang

sqrl - fat-free version of squirrel - fluent SQL generator for Go Non thread safe fork of squirrel. The same handy fluffy helper, but with extra lette

Dec 16, 2022
Fluent SQL generation for golang

Squirrel is "complete". Bug fixes will still be merged (slowly). Bug reports are welcome, but I will not necessarily respond to them. If another fork

Jan 6, 2023
golang orm and sql builder

gosql gosql is a easy ORM library for Golang. Style: var userList []UserModel err := db.FetchAll(&userList, gosql.Columns("id","name"), gosql.

Dec 22, 2022