SQL builder and query library for golang

  __ _  ___   __ _ _   _
 / _` |/ _ \ / _` | | | |
| (_| | (_) | (_| | |_| |
 \__, |\___/ \__, |\__,_|
 |___/          |_|

GitHub tag Test GoDoc codecov Go Report Card

goqu is an expressive SQL builder and executor

If you are upgrading from an older version please read the Migrating Between Versions docs.

Installation

If using go modules.

go get -u github.com/doug-martin/goqu/v9

If you are not using go modules...

NOTE You should still be able to use this package if you are using go version >v1.10 but, you will need to drop the version from the package. import "github.com/doug-martin/goqu/v9 -> import "github.com/doug-martin/goqu"

go get -u github.com/doug-martin/goqu

Migrating Between Versions

Features

goqu comes with many features but here are a few of the more notable ones

  • Query Builder
  • Parameter interpolation (e.g SELECT * FROM "items" WHERE "id" = ? -> SELECT * FROM "items" WHERE "id" = 1)
  • Built from the ground up with multiple dialects in mind
  • Insert, Multi Insert, Update, and Delete support
  • Scanning of rows to struct[s] or primitive value[s]

While goqu may support the scanning of rows into structs it is not intended to be used as an ORM if you are looking for common ORM features like associations, or hooks I would recommend looking at some of the great ORM libraries such as:

Why?

We tried a few other sql builders but each was a thin wrapper around sql fragments that we found error prone. goqu was built with the following goals in mind:

  • Make the generation of SQL easy and enjoyable
  • Create an expressive DSL that would find common errors with SQL at compile time.
  • Provide a DSL that accounts for the common SQL expressions, NOT every nuance for each database.
  • Provide developers the ability to:
    • Use SQL when desired
    • Easily scan results into primitive values and structs
    • Use the native sql.Db methods when desired

Docs

  • Dialect - Introduction to different dialects (mysql, postgres, sqlite3, sqlserver etc)
  • Expressions - Introduction to goqu expressions and common examples.
  • Select Dataset - Docs and examples about creating and executing SELECT sql statements.
  • Insert Dataset - Docs and examples about creating and executing INSERT sql statements.
  • Update Dataset - Docs and examples about creating and executing UPDATE sql statements.
  • Delete Dataset - Docs and examples about creating and executing DELETE sql statements.
  • Prepared Statements - Docs about interpolation and prepared statements in goqu.
  • Database - Docs and examples of using a Database to execute queries in goqu
  • Working with time.Time - Docs on how to use alternate time locations.

Quick Examples

Select

See the select dataset docs for more in depth examples

sql, _, _ := goqu.From("test").ToSQL()
fmt.Println(sql)

Output:

SELECT * FROM "test"
sql, _, _ := goqu.From("test").Where(goqu.Ex{
	"d": []string{"a", "b", "c"},
}).ToSQL()
fmt.Println(sql)

Output:

SELECT * FROM "test" WHERE ("d" IN ('a', 'b', 'c'))

Insert

See the insert dataset docs for more in depth examples

ds := goqu.Insert("user").
	Cols("first_name", "last_name").
	Vals(
		goqu.Vals{"Greg", "Farley"},
		goqu.Vals{"Jimmy", "Stewart"},
		goqu.Vals{"Jeff", "Jeffers"},
	)
insertSQL, args, _ := ds.ToSQL()
fmt.Println(insertSQL, args)

Output:

INSERT INTO "user" ("first_name", "last_name") VALUES ('Greg', 'Farley'), ('Jimmy', 'Stewart'), ('Jeff', 'Jeffers') []
ds := goqu.Insert("user").Rows(
	goqu.Record{"first_name": "Greg", "last_name": "Farley"},
	goqu.Record{"first_name": "Jimmy", "last_name": "Stewart"},
	goqu.Record{"first_name": "Jeff", "last_name": "Jeffers"},
)
insertSQL, args, _ := ds.ToSQL()
fmt.Println(insertSQL, args)

Output:

INSERT INTO "user" ("first_name", "last_name") VALUES ('Greg', 'Farley'), ('Jimmy', 'Stewart'), ('Jeff', 'Jeffers') []
type User struct {
	FirstName string `db:"first_name"`
	LastName  string `db:"last_name"`
}
ds := goqu.Insert("user").Rows(
	User{FirstName: "Greg", LastName: "Farley"},
	User{FirstName: "Jimmy", LastName: "Stewart"},
	User{FirstName: "Jeff", LastName: "Jeffers"},
)
insertSQL, args, _ := ds.ToSQL()
fmt.Println(insertSQL, args)

Output:

INSERT INTO "user" ("first_name", "last_name") VALUES ('Greg', 'Farley'), ('Jimmy', 'Stewart'), ('Jeff', 'Jeffers') []
ds := goqu.Insert("user").Prepared(true).
	FromQuery(goqu.From("other_table"))
insertSQL, args, _ := ds.ToSQL()
fmt.Println(insertSQL, args)

Output:

INSERT INTO "user" SELECT * FROM "other_table" []
ds := goqu.Insert("user").Prepared(true).
	Cols("first_name", "last_name").
	FromQuery(goqu.From("other_table").Select("fn", "ln"))
insertSQL, args, _ := ds.ToSQL()
fmt.Println(insertSQL, args)

Output:

INSERT INTO "user" ("first_name", "last_name") SELECT "fn", "ln" FROM "other_table" []

Update

See the update dataset docs for more in depth examples

sql, args, _ := goqu.Update("items").Set(
	goqu.Record{"name": "Test", "address": "111 Test Addr"},
).ToSQL()
fmt.Println(sql, args)

Output:

UPDATE "items" SET "address"='111 Test Addr',"name"='Test' []
type item struct {
	Address string `db:"address"`
	Name    string `db:"name" goqu:"skipupdate"`
}
sql, args, _ := goqu.Update("items").Set(
	item{Name: "Test", Address: "111 Test Addr"},
).ToSQL()
fmt.Println(sql, args)

Output:

UPDATE "items" SET "address"='111 Test Addr' []
sql, _, _ := goqu.Update("test").
	Set(goqu.Record{"foo": "bar"}).
	Where(goqu.Ex{
		"a": goqu.Op{"gt": 10}
	}).ToSQL()
fmt.Println(sql)

Output:

UPDATE "test" SET "foo"='bar' WHERE ("a" > 10)

Delete

See the delete dataset docs for more in depth examples

ds := goqu.Delete("items")

sql, args, _ := ds.ToSQL()
fmt.Println(sql, args)
sql, _, _ := goqu.Delete("test").Where(goqu.Ex{
		"c": nil
	}).ToSQL()
fmt.Println(sql)

Output:

DELETE FROM "test" WHERE ("c" IS NULL)

Contributions

I am always welcoming contributions of any type. Please open an issue or create a PR if you find an issue with any of the following.

  • An issue with Documentation
  • You found the documentation lacking in some way

If you have an issue with the package please include the following

  • The dialect you are using
  • A description of the problem
  • A short example of how to reproduce (if applicable)

Without those basics it can be difficult to reproduce your issue locally. You may be asked for more information but that is a good starting point.

New Features

New features and/or enhancements are great and I encourage you to either submit a PR or create an issue. In both cases include the following as the need/requirement may not be readily apparent.

  1. The use case
  2. A short example

If you are issuing a PR also also include the following

  1. Tests - otherwise the PR will not be merged
  2. Documentation - otherwise the PR will not be merged
  3. Examples - [If applicable] see example_test.go for examples

If you find an issue you want to work on please comment on it letting other people know you are looking at it and I will assign the issue to you.

If want to work on an issue but dont know where to start just leave a comment and I'll be more than happy to point you in the right direction.

Running tests

The test suite requires a postgres, mysql and sqlserver databases. You can override the connection strings with the MYSQL_URI, PG_URI, SQLSERVER_URI environment variables*

go test -v -race ./...

You can also run the tests in a container using docker-compose.

GO_VERSION=latest docker-compose run goqu

License

goqu is released under the MIT License.

Owner
Doug Martin
Enjoys solving hard problems and learning new languages. Actively developing with Scala, Node and Golang. Husband to a great wife and VP of Engineering at C2FO.
Doug Martin
Comments
  • Do not interpolate literals when prepared=true.

    Do not interpolate literals when prepared=true.

    This addresses https://github.com/doug-martin/goqu/issues/132

    It is a reset of closed PR: https://github.com/doug-martin/goqu/pull/151

    The gist of this fix is to modify the builders to not interpolate nil and boolean values so that prepared statements can work properly. This makes prepared=true work more consistently across all types.

    The scope of this change:

    (1) Modify literalNil to use a placeholder if prepared=true similar to literalBool and literalInt

    (2) Modify checkBoolExpType to not turn (in)equality on nil or bool values to an IS expression.

    (3) LOTS of tests and examples updated to reflect the new behavior.

    (4) Fix bugs from https://github.com/doug-martin/goqu/pull/151 found by @doug-martin to preserve correct behavior with Is/IsNot and not incorrectly map them to Eq/NotEq.

  • SetError

    SetError

    Is there a way to set an error on a SelectDataset (or any other dataset for that matter) ? It would be super convenient if my code which is building up a query using goqu could leverage its internal error tracking so a subsequent ToSQL would give me back the error I set (or any other internal goqu error) instead of me having to add another layer on top of goqu to track this.

    Here's some pseudocode example of what I'd like to be able to do:

    var sel *goqu.SelectDataset = ....
    if (filter != Equal) {
        return sel.SetError(fmt.Errorf("Only Equal operation is supported"))
    }
    

    Then, later on when I go to execute the resulting SQL, this would provide a really nice idiomatic way to detect and propagate the error:

        query, args, err := sqlizer.ToSQL()
        if err != nil {
            return err
        }   
    

    Thoughts? Better ideas on how to handle this ?

    Thanks!!

  • Add support for

    Add support for "FOR UPDATE" and "SKIP LOCKED"

    This addresses #43. It's a first stab, but hopefully close.

    The public interface only exposes a .ForUpdate() method on the dataset for now, but under the hood it's built to support .ForNoKeyUpdate(), .ForShare(), and .ForKeyShare() as well.

  • NULL literals in prepared statements

    NULL literals in prepared statements

    goqu rocks :-)

    So..I may be doing this wrong or have incorrect expectations and if so please set me straight :-).

    I'm trying to use goqu to create prepared statements. I saw https://github.com/doug-martin/goqu/issues/127 which was super helpful in getting me going. However, it seems that UpdateDataset is interpolating NULL values even though I used .Prepared(true) which as I understand it should cause it to not interpolate the values so the expression can be reused with different parameters in the future. Here's a concrete example:

    func TestUpdatePreparedWithNullValue(t *testing.T) {
    
        dialect := goqu.Dialect("postgres")
    
        var value1 *int = nil 
    
        updateDataset := dialect.Update("foo").Prepared(true).Set(map[string]interface{}{
            "value1": value1,
        })  
    
        sql, _, _ := updateDataset.ToSQL()
        assert.Equal(t, `UPDATE "foo" SET "value1"=$1`, sql)
    }
    

    And here's the test output:

            	Error Trace:	user_test.go:95
            	Error:      	Not equal: 
            	            	expected: "UPDATE \"foo\" SET \"value1\"=$1"
            	            	actual  : "UPDATE \"foo\" SET \"value1\"=NULL"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-UPDATE "foo" SET "value1"=$1
            	            	+UPDATE "foo" SET "value1"=NULL
            	Test:       	TestUpdatePreparedWithNullValue
    
    
  • Mysql dialect TimeFormat strips location data

    Mysql dialect TimeFormat strips location data

    Describe the bug When connecting to mysql the default value of cfg.Loc is UTC which means reading datetime data from mysql results in a time.Time with Location set to UTC even if the inserted value was in another timezone.

    This happens because the TimeFormat of the mysql dialect strips the timezone. This became an issue after the removal of the utc conversion in literalTime here: https://github.com/doug-martin/goqu/pull/106

    Can the dialect handle this itself somehow? Perhaps defining TimeFormat func(t time.Time) string so it can be set per dialect?

    To Reproduce Insert a time.Time value with another timezone. The use ScanStruct to scan it in again.

    Expected behavior The scanned value should have the same point in time as the inserted value.

    Dialect:

    • [ ] postgres
    • [x] mysql
    • [ ] sqlite3

    Additional context Add any other context about the problem here.

  • Offline SQL building

    Offline SQL building

    As far as I can tell, there is currently no way to initialize a SQL builder without creating a database instance. Why is this, since I'd have thought the database connection is only necessary for querying after building?

    My potential use case simply involves generating SQL strings, not necessarily querying them.

  • with table as (insert ... returning ...)

    with table as (insert ... returning ...)

    Is your feature request related to a problem? Please describe. We are trying to convert a query that has a common table expression that is a result of insert...returning. We are getting the error goqu: unsupported expression type *goqu.InsertDataset from the sql generator.

    Describe the solution you'd like Adding a case for InsertExpression to https://github.com/doug-martin/goqu/blob/25020dc497ec58c3dbfe98190ac30ff72b0f59d2/sqlgen/expression_sql_generator.go#L140

    Describe alternatives you've considered N/A

    Dialect

    • [x] postgres
    • [ ] mysql
    • [ ] sqlite3

    Additional context Opening this issue. We think we know the fix, but need to fork and setup our dev environments to get you a PR. Thanks for all the awesome work lately! We would like to contribute where possible.

  • Scan into embedded structs

    Scan into embedded structs

    Is your feature request related to a problem? Please describe. Being able to use the built in goqu functionality like ScanStructContext and ScanStructsContext is great and makes for concise, readable code. However, I have not yet found a way to scan into a struct that has embedded structs within it.

    Describe the solution you'd like A simple example of something we would like to be able to do with goqu below.

    Let's say there are two tables (table1 and table2) and we write a query to join these tables and select all columns from both the tables. I want to scan into a struct that looks like this rows struct that contains structs that define all the columns for table1 and table 2.

    var rows []struct {
      Table1 table1Model `db:"table1"`
      Table2 table2Model `db:"table2"`
    }
    
    var table1Model struct {
      col1 datatype `db:"col1"`
      col2 datatype `db:"col2"`
    }
    
    var table2Model struct {
      col1 datatype `db:"col1"`
      col2 datatype `db:"col2"`
    }
    

    The query would look roughly like this.

    goqu.From("table1").
      Prepared(true).
      InnerJoin(
        goqu.T("table2"),
        goqu.On(goqu.T("table1).Col("col1")Eq(goqu.T("table2").Col("col2"))),
      ).
      Select(
        goqu.L("table1"."col1" as "table1.col1"),
        goqu.L("table1"."col2" as "table1.col2"),
        goqu.L("table2"."col1" as "table2.col1"),
        goqu.L("table2"."col2" as "table2.col2"),
      ).
      ScanStructsContext(ctx, &rows)
    

    Describe alternatives you've considered Right now I can accomplish this by using the ToSQL() method on my query and then executing the query with sqlx because the sqlx StructScan method supports scanning into nested structs.

    Dialect

    • [X] postgres
    • [ ] mysql
    • [ ] sqlite3

    Additional context This is for sure a nice to have in goqu as it is not blocking functionality at the moment, but could help clean up a lot of code by using only goqu to accomplish these queries.

  • Manually modifying queries before execution (v7)

    Manually modifying queries before execution (v7)

    With v6, using CrudExec, I was able to modify the queries before executing them using the exposed Sql field. I am using it with extra features of postgres like UPSERT and INSERT ... RETURNING.

    It doesn't look like that's possible with the new exec.QueryExecutor api.

    Please consider exposing its query field.

  • Issue 183: Propagate subselect errors properly.

    Issue 183: Propagate subselect errors properly.

    This addresses https://github.com/doug-martin/goqu/issues/183 by propagating subselect errors into newly returned SelectDataset so caller doesn't have to explicitly check for errors in subselects. Let me know what you think @doug-martin.

  • exec.Scanner: New exposed scanner supports iterative scanning

    exec.Scanner: New exposed scanner supports iterative scanning

    A Scanner that is modeled after sql.Rows Next and Scan. This can be used when scanning one item at a time for better memory efficiency on large result sets.

    I'm a bit unsure about the db.From().Select().Executor().Scanner() way of getting it when coming from a SelectDataset but I think that this wouldn't be used so often as to warrant a method directly on the SelectDataset.

    Let me know what you think!

Type safe SQL query builder and struct mapper for Go

sq (Structured Query) ?? ?? sq is a code-generated, type safe query builder and struct mapper for Go. ?? ?? Documentation • Reference • Examples This

Dec 19, 2022
SQL query builder for Go

GoSQL Query builder with some handy utility functions. Documentation For full documentation see the pkg.go.dev or GitBook. Examples // Open database a

Dec 12, 2022
Fast SQL query builder for Go

sqlf A fast SQL query builder for Go. sqlf statement builder provides a way to: Combine SQL statements from fragments of raw SQL and arguments that ma

Dec 23, 2022
gosq is a parsing engine for a simplicity-focused, template-based SQL query builder for Go.

gosq is a parsing engine for a simplicity-focused, template-based SQL query builder for Go.

Oct 24, 2022
Data-builder - Data builder with golang

databuilder import "github.com/go-coldbrew/data-builder" Index Variables func Is

Feb 5, 2022
Go database query builder library for PostgreSQL

buildsqlx Go Database query builder library Installation Selects, Ordering, Limit & Offset GroupBy / Having Where, AndWhere, OrWhere clauses WhereIn /

Dec 23, 2022
a golang library for sql builder

Gendry gendry is a Go library that helps you operate database. Based on go-sql-driver/mysql, it provides a series of simple but useful tools to prepar

Dec 26, 2022
sqlc implements a Dynamic Query Builder for SQLC and more specifically MySQL queries.

sqlc-go-builder sqlc implements a Dynamic Query Builder for SQLC and more specifically MySQL queries. It implements a parser using vitess-go-sqlparser

May 9, 2023
golang orm and sql builder

gosql gosql is a easy ORM library for Golang. Style: var userList []UserModel err := db.FetchAll(&userList, gosql.Columns("id","name"), gosql.

Dec 22, 2022
Simple query builder for MongoDB

?? greenleaf - simple, type safe and easy to use query builder for MongoDB Installation To install use: go get github.com/slavabobik/greenleaf Quick

Nov 27, 2022
Database Abstraction Layer (dbal) for Go. Support SQL builder and get result easily (now only support mysql)

godbal Database Abstraction Layer (dbal) for go (now only support mysql) Motivation I wanted a DBAL that No ORM、No Reflect、Concurrency Save, support S

Nov 17, 2022
An easy-use SQL builder.

EQL An easy-use SQL builder. Design We are not English native speaker, so we use Chinese to write the design documents. We plan to translate them to E

Dec 26, 2022
A Go (golang) package that enhances the standard database/sql package by providing powerful data retrieval methods as well as DB-agnostic query building capabilities.

ozzo-dbx Summary Description Requirements Installation Supported Databases Getting Started Connecting to Database Executing Queries Binding Parameters

Dec 31, 2022
Query git repositories with SQL. Generate reports, perform status checks, analyze codebases. 🔍 📊

askgit askgit is a command-line tool for running SQL queries on git repositories. It's meant for ad-hoc querying of git repositories on disk through a

Jan 5, 2023
SQL query helper

SQL query helper

Nov 7, 2021
Command line tool to generate idiomatic Go code for SQL databases supporting PostgreSQL, MySQL, SQLite, Oracle, and Microsoft SQL Server

About xo xo is a command-line tool to generate Go code based on a database schema or a custom query. xo works by using database metadata and SQL intro

Jan 8, 2023
Go fearless SQL. Sqlvet performs static analysis on raw SQL queries in your Go code base.

Sqlvet Sqlvet performs static analysis on raw SQL queries in your Go code base to surface potential runtime errors at build time. Feature highlights:

Dec 19, 2022
PirateBuilder - Pirate Builder For Golang
PirateBuilder - Pirate Builder For Golang

PirateBuilder Builder You need to extract the file "PirateBuilder.rar". Start "P

Jun 10, 2022
Nerdlight-firmwarebuilder - Nerdlight NodeMCU Firmware Builder CLI

nerdlight-firmwarebuilder ⚒ ⚡️ Nerdlight NodeMCU Firmware Builder CLI ⚒ Descript

Feb 12, 2022