Database Access Layer for Golang - Testable, Extendable and Crafted Into a Clean and Elegant API

REL

GoDoc Build Status Go Report Card Maintainability Codecov Gitter chat

Modern Database Access Layer for Golang.

REL is golang orm-ish database layer for layered architecture. It's testable and comes with its own test library. REL also features extendable query builder that allows you to write query using builder or plain sql.

Features

  • Testable repository with builtin reltest package.
  • Seamless nested transactions.
  • Elegant, yet extendable query builder with mix of syntactic sugar.
  • Supports Eager loading.
  • Composite Primary Key.
  • Multi adapter.
  • Soft Deletion.
  • Pagination.
  • Schema Migration.

Install

go get github.com/go-rel/rel

Getting Started

Examples

License

Released under the MIT License

Owner
REL
Database Access Layer for Golang - Testable, Extendable and Crafted Into a Clean and Elegant API
REL
Comments
  • Execute arbitrary sql statements

    Execute arbitrary sql statements

    I have an update statement that looks like:

    update matches m
        set viewed = true
        where m.request_id in (
            select r.id
            from requests r
                        join users u on r.user_id = u.id
            where u.username = $1)
    

    And I need to execute via rel but I couldn't. ( and I don't want to use rel dsl here 😄 )

    If it is a select query, I could use SQL function and pass it to FindAll function. I tried the Update function but doesn't seem to work.

    I had to use the underlaying *sql/DB Exec method.

  • why invoke

    why invoke "SHOW VARIABLES LIKE 'auto_increment_increment'" after repo.InsertAll?

    step 1

    	if err = repo.InsertAll(ctx, &beInsertedAnswers); err != nil {
    		return
    	}
    
    

    step 2

    2022/07/06 18:19:17 [duration: 1.442817ms op: adapter-exec] INSERT INTO `answers` (`answer_text`,`attempt_id`,....) VALUES (?,?,?,?,?,?,?);
    2022/07/06 18:19:17 [duration: 1.398397ms op: adapter-query] SHOW VARIABLES LIKE 'auto_increment_increment';
    

    why invoke "SHOW VARIABLES LIKE 'auto_increment_increment'" after repo.InsertAll?

  • Support embeddable structs

    Support embeddable structs

    Hi! I've added support for embedded structs as discussed in this issue.

    Both pointers and plain fields are supported.

    type Person struct {
    	ID   int
    	Name string
    }
    
    type Employee struct {
    	Person
    	Salary int
    }
    

    By default embedded field names are added without prefixes. Name prefixes can be set like for other fields:

    type Employee struct {
    	Person   `db:"person_"`
    	Salary int
    }
    

    If two fields have the same name, REL panics during construction:

    What I've changed

    • documentData indices now store full field paths as []int
    • documentDatas are merged for embedded fields during creation
    • Embedded pointers are initialized during document creation

    Tests

    I've added tests for all main cases

  • Add alternative soft delete scheme

    Add alternative soft delete scheme

    Allows to use alternative soft delete scheme with boolean field Deleted. When record is deleted it will update Deleted to true and if there is UpdatedAt field it will also update it with deletion timestamp.

    Sample:

    type Test struct {
      ID int64
      Name string
      CreatedAt time.Time
      UpdatedAt *time.Time
      Deleted bool
    }
    
  • Add option for filtered/conditional index option

    Add option for filtered/conditional index option

    Add option that implements rel.KeyOption for index WHERE condition.

    As far as I know only MySQL does not support this.

    PostgreSQL: https://www.postgresql.org/docs/9.1/sql-createindex.html SQLite3: https://www.sqlite.org/partialindex.html MSSQL: https://docs.microsoft.com/en-us/sql/t-sql/statements/create-index-transact-sql?view=sql-server-ver15

    so that it could be specified for example:

    t.Unique([]string{"column"}, rel.KeyCondition("NOT deleted"))
    

    That would generate:

    CREATE UNIQUE INDEX ON table (column) WHERE NOT deleted;
    

    technically this could be easily done by simple type but I don't know how to implement this conditionally so not to break MySQL provider that does not support this

    // KeyCondition filter option for unique key.
    type KeyCondition string
    
    func (kc KeyCondition) applyKey(key *Key) {
    	key.Options += fmt.Sprintf(" WHERE (%s)", kc)
    }
    
  • [Subquery?] How to query user in any of groups.

    [Subquery?] How to query user in any of groups.

    table User {
      id int
      name varchar(32)
    }
    
    table GroupUser struct {
      user_id int
      group_id int
    }
    
    table Group {
      id int
      name varchar(32)
    }
    

    How to find unique user in groups [1,2,3] ? Follow code does not work:

    groups := []int{1,2,3}
    subQuery := rel.From("group_users").Select("DISTINCT user_id").Where(where.InInt("group_id", groups))
    repo.FindAll(ctx, &users, where.In("id",  subQuery))
    
  • Problem migrating

    Problem migrating

    Hello,

    I am attempting to run rel migrate on a blank, fresh database. If you observe the screenshot below you'll see I have a migration file, but I am getting an error:

    go: updates to go.sum needed, disabled by -mod=readonly
    

    This is in a simple, test project with a freshly initialized go.mod. Any thoughts? Thanks!

    Screen Shot 2020-10-14 at 11 32 00 AM
  • Cascade delete support?

    Cascade delete support?

    hi!

    just found your project on reddit and I'm very interested in trying it out. i've used gorm before but couldn't get myself to like it, this seems like a viable alternative.

    as for my question, does rel support cascade delete as of this point?

  • Generic type Repository wrapper

    Generic type Repository wrapper

    • Simple wrapper to provide type safety to repository
    • Rename record/records to be entity/entities, because the new generic repository is named as EntityRepository
  • Add support for embedded no anonymous fields

    Add support for embedded no anonymous fields

    Change

    // struct without primary key is a field
    // TODO: test by scanner/valuer instead?
    if pk, _ := searchPrimary(typ); len(pk) == 0 {
    	data.fields = append(data.fields, name)
    	continue
    }
    

    If struct implement scanner/valuer - is field, else is embedded

  • Optimistic locking

    Optimistic locking

    Hi! I've implemented optimistic locking as discussed here.

    It's automatically enabled by including a LockVersion int field into the struct.

    It adds a where lock_version = $ clause to select only valid records and uses set lock_version = lock_version + 1 for updating locks.

    What I've changed

    Mainly repository.update() and repository.delete(). They now include custom logic for applying locks.

    Because using SQL expressions like += requires reloading the whole struct, I've added a NoReload flag to Mutates to surpress triggering reloads for lock updates. In case no reload is required, the locks are incremented manually.

    Tests

    I've added simple tests for updating and deleting actual and stale records.

  • Transactions doc please clarify

    Transactions doc please clarify

    Hi everyone.

    I am struggling with understanding the documentation about transactions: https://go-rel.github.io/transactions/

    The example shows the line Process(ctx, transaction). Where is that transaction value coming from, and why it passed withing the Update() method?

    Also another question: Should I create a new context.Context for every transaction or I can use the same one for all my transactions?

  • Add method to alter table column

    Add method to alter table column

    TODO:

    • [x] Way to detect what constraints are applied

    For discussion:

    • ~~How to detect that type have changed?~~
    • ~~How to detect what constraints need to be changed?~~
    • ~~Should SQLite just return error as it does not support alter column?~~
  • ERROR: duplicate key value violates unique constraint

    ERROR: duplicate key value violates unique constraint "pg_class_relname_nsp_index"

    ERROR:  duplicate key value violates unique constraint "pg_class_relname_nsp_index"
    DETAIL:  Key (relname, relnamespace)=(devices_schema_version_id_seq, 2200) already exists.
    STATEMENT:  CREATE TABLE IF NOT EXISTS "devices_schema_version" ("id" SERIAL NOT NULL PRIMARY KEY, "version" BIGINT UNIQUE, "created_at" TIMESTAMPTZ, "updated_at" TIMESTAMPTZ);
    

    This issue was found by starting two concurrent services, both running an auto-migration as part of its startup in which they were making sure that the devices_schema_version table existed, however: Postgres has a known bug with its CREATE TABLE IF NOT EXIST implementation that may cause the above cited error.

    The remedy is to acquire a lock using LOCK TABLE pg_catalog.pg_namespace IN SHARE ROW EXCLUSIVE MODE; within the same transaction according to source 1 and source 2.

    Perhaps this issue should have been written in https://github.com/go-rel/postgres instead?

  • [Bug] Change `assoc many` slice and `Update`/`Insert` with `Changeset`

    [Bug] Change `assoc many` slice and `Update`/`Insert` with `Changeset`

    Append

    Example

    type Main struct {
        ID        int `db:",primary"`
        Childs []*Child `ref:"id" fk:"main_id" autosave:"true" autoload:"true"`
    }
    
    type Child struct {
        ID        int `db:",primary"`
        MainID int
        Name string
    }
    
    main := new(Main)
    repo.Find(ctx, main, rel.Eq("id", id))
    changeset := rel.NewChangeset(main)
    main.Childs = append(main.Childs, &Child{...})
    repo.Update(ctx, main, changeset)
    

    Actual result

    panic("rel: invalid mutator")
    

    Change

    Example

    main := new(Main)
    repo.Find(ctx, main, rel.Eq("id", id))
    changeset := rel.NewChangeset(main)
    main.Childs[0].Name = "foo"
    main.Childs[1].Name = "bar"
    repo.Update(ctx, main, changeset)
    

    Actual result

    generate invalid queries

    DELETE FROM "child" WHERE "child"."main_id"=$1;
    INSERT INTO "child" ("name","main_id") VALUES ($1,$2),($3,$4) RETURNING "id"
    
  • Stable column order for generated SQL statements

    Stable column order for generated SQL statements

    Currently looks like because of map usage for storing column data each time insert/update/select SQL statements are generated column order is different, that makes grouping of SQL statements for perf stats in logs impossible and also this makes it unusable for prepared statement caching being polluted and unusable that hurts performance.

Golang REST Layer SQL Storage Handler

This REST Layer resource storage backend stores data in a SQL Database using database/sql.

Feb 15, 2022
Sqlair - SQLite Query Layer With Golang

sqlair SQLite Query Layer Creates an abstract over the go sql package to provide

Feb 18, 2022
Library for scanning data from a database into Go structs and more

scany Overview Go favors simplicity, and it's pretty common to work with a database via driver directly without any ORM. It provides great control and

Jan 9, 2023
[mirror] the database client and tools for the Go vulnerability database

The Go Vulnerability Database golang.org/x/vulndb This repository is a prototype of the Go Vulnerability Database. Read the Draft Design. Neither the

Dec 29, 2022
Database - Example project of database realization using drivers and models

database Golang based database realization Description Example project of databa

Feb 10, 2022
A Go rest API project that is following solid and common principles and is connected to local MySQL database.
A Go rest API project that is following solid and common principles and is connected to local MySQL database.

This is an intermediate-level go project that running with a project structure optimized RESTful API service in Go. API's of that project is designed based on solid and common principles and connected to the local MySQL database.

Dec 25, 2022
CRUD API example is written in Go using net/http package and MySQL database.
CRUD API example is written in Go using net/http package and MySQL database.

GoCrudBook CRUD API example is written in Go using net/http package and MySQL database. Requirements Go MySQL Code Editor Project Structure GoCrudBook

Dec 10, 2022
Database migrations. CLI and Golang library.

migrate Database migrations written in Go. Use as CLI or import as library. Migrate reads migrations from sources and applies them in correct order to

Jan 9, 2023
database to golang struct
database to golang struct

中文文档 mysql database to golang struct conversion tools base on gorm(v1/v2),You can automatically generate golang sturct from mysql database. big Camel-

Jan 9, 2023
Constant Database native golang implementation

CDB golang implementation cdb is a fast, reliable, simple package for creating and reading constant databases see docs for more details Advantages Ite

Jul 15, 2022
Golang restAPI crud project with mySql database.
 Golang restAPI crud project with mySql database.

Golang RestAPI using gorilla/mux Golang restAPI crud project with mySql database. Test Api with Thunder Client vs code beautiful Extension. and use Be

Mar 26, 2022
A simple Golang-based application that queries a PostgreSQL database

Qwik-E-Mart Demo App A simple Golang-based application that queries a PostgreSQL database named qwikemart to read and return customer data stored in t

Nov 6, 2021
Implementasi database oracle kedalam golang

Go with Oracle database Implementasi database oracle kedalam golang How to using swagger Install generator swagger menggunakan perintah : go get -u gi

Nov 20, 2021
Golang database driver for SQLite

go-sqlite Golang database driver for SQLite. Does not use cgo. This driver is based on pure-Go SQLite implementation (https://gitlab.com/cznic/sqlite)

Dec 30, 2022
BigQuery database/sql golang driver

BigQuery SQL Driver This library is compatible with Go 1.17+ Please refer to CHA

Dec 7, 2022
CloudQuery extracts, transforms, and loads your cloud assets into normalized PostgreSQL tables.
CloudQuery extracts, transforms, and loads your cloud assets into normalized PostgreSQL tables.

The open-source cloud asset inventory backed by SQL. CloudQuery extracts, transforms, and loads your cloud assets into normalized PostgreSQL tables. C

Dec 31, 2022
Sync MySQL data into elasticsearch
Sync MySQL data into elasticsearch

go-mysql-elasticsearch is a service syncing your MySQL data into Elasticsearch automatically. It uses mysqldump to fetch the origin data at first, the

Dec 30, 2022
An experimental toolkit for injecting alternate authentication strategies into a PostgreSQL-compatible wire format.

PG Auth Proxy This is an experimental toolkit for injecting alternate authentication strategies into a PostgreSQL-compatible wire format. This is a pr

Jan 20, 2022