Golang PostgreSQL schema migration tool

goerd

This is a New Era of migrations PostgreSQL databases with Golang.

We no longer want to ensure the database schema with incremental patches. Shema MUST match what the app wants! It doesn't matter what was in the database before that.

It could be empty, it could contain some data in any of its previous versions. It doesn't matter anymore. It is important that we carefully transform it into the target state. We will no longer be guessing about the target state of the database by patchset. This is a thing of the past.

This tool allows you to turn schemas into instructions for the database, including migrations between schemas. Create easy-to-read data models as contracts for agreement between architects, development teams, and team leaders. This tool provides agility to change the huge data-layered models.

Conceptual view

Install

go install github.com/covrom/goerd/cmd/goerd@latest

Features

  • Generate yaml short schema description from database
  • Generate plantuml view of schema
  • Create posgresql migrations as a set of SQL queries that apply changes between two schemas, a schema and a database, or two databases using a schema definition that is stored in a yaml or plantuml file.

Migration of the following objects is currently supported:

  • Tables
  • Columns
  • Indexes
  • Constraints and Foreign Keys
  • Views

This set covers 99% of PostgreSQL usecases in Golang services.

Example of generated plantuml:

Plantuml view

Usage

Save database schema to yaml schema or PlantUML:

goerd -from "postgres://username:password@url:port/dbName" -to ./schema1.yaml

goerd -from "postgres://username:password@url:port/dbName" -to ./schema1.puml

Print SQL queries migration between two schemas, dropping queries is commented out:

goerd -from schema1.yml -to schema2.yaml

goerd -from schema1.yml -to "postgres://username:password@url:port/dbName"

With dropping queries:

goerd -drop -from schema1.yml -to schema2.yml

goerd -drop -from schema1.yml -to "postgres://username:password@url:port/dbName"

Save PlantUML from schema:

goerd -from schema1.yml -to schema1.puml

goerd -from "postgres://username:password@url:port/dbName" -to schema1.puml

Apply schema from yaml to database:

goerd -c apply -from schema1.yml -to "postgres://username:password@url:port/dbName"

Apply schema from DB1 to DB2:

goerd -c apply -from "postgres://username:password@url:port/DB1" -to "postgres://username:password@url:port/DB2"

API

See docs.

testing and examples

docker-compose up and see ./output/schema.yaml

Owner
Roman Сovanyan
golang high load instructor
Roman Сovanyan
https://github.com/covrom/goerd
Similar Resources

A web application attack surface mapping tool. It takes in a list of urls then performs numerous probes

sigurlscann3r A web application attack surface mapping tool. It takes in a list of urls then performs numerous probes Resources Features Installation

Sep 24, 2022

A simple and privacy focused analytics tool written in go.

A simple and privacy focused analytics tool written in go.

Analytics Box Hello guys, this is privacy friendly analytics tool, Analytics Box for web written in go. It ensures user privacy but at the same time a

Nov 27, 2022

A simple application lifecycle management tool with multiple servers.

A simple application lifecycle management tool with multiple servers.

Aug 13, 2022

A tool to monitor and health check servers through plug-in scripts

A tool to monitor and health check servers through plug-in scripts

WatchDogClient - WDC WDC is a client application for WatchDog - wd, which is a tool to monitor and health check servers through plug-in scripts. WDC l

Jan 12, 2022

htf (Host That File) is a tool to make serving up your favorite pentest tools simpler and faster.

htf (Host That File) is a tool to make serving up your favorite pentest tools simpler and faster.

htf htf (Host That File) is a tool to make serving up your favorite pentest tools simpler and faster. All you need to do is populate the htf configura

Nov 28, 2021

A cross platform single binary tool to work with draw io files

SNIPO SNIPIO is a cross platform single binary tool to work with draw io files With snipio you can do the following operations: list all layers in a d

Jan 7, 2023

This application is a tool to rapidly create TFS tasks and synchronize them with wiki.

This application is a tool to rapidly create TFS tasks and synchronize them with wiki.

Jan 10, 2022

A simple command line tool using which you can skip phone number based SMS verification by using a temporary phone number that acts like a proxy

A simple command line tool using which you can skip phone number based SMS verification by using a temporary phone number that acts like a proxy

Fake-SMS A simple command line tool using which you can skip phone number based SMS verification by using a temporary phone number that acts like a pr

Dec 31, 2022

ging is a tool for create gin web framework development templates

ging ging is a tool for create gin web framework development templates This tool is for the freshmen who want to learn golang and gin web framework, i

Jan 15, 2022
Comments
  • fix: also use table name to find correct relation to alter

    fix: also use table name to find correct relation to alter

    Right now if there are multiple tables with "parent_id" relation to the same parent table, the diff returns the first match and the alter table fails.

    This fix adds a table name check to the relation finder so that the diffs don't find relations in other tables that have similar column names.

  • Indexes are not diff'ed correctly. Comparing pointers to index will always return false.

    Indexes are not diff'ed correctly. Comparing pointers to index will always return false.

    https://github.com/covrom/goerd/blob/a445386b7f689343cb5f8b2ff5f40cc0404ea28d/schema/diff.go#L496

    for _, v := range pt.indexes {
		if v.from == pi.from &&
			v.to == pi.to {
			fnd = true
			break
		}
	}

    The Indexes test is also failing because of this.
    https://github.com/covrom/goerd/blob/a445386b7f689343cb5f8b2ff5f40cc0404ea28d/schema/diff_test.go#L303

    --- FAIL: TestPatchSchema_BuildChangeIdx (0.00s)
    --- FAIL: TestPatchSchema_BuildChangeIdx/1 (0.00s)
        diff_test.go:365: ALTER TABLE table1 ADD COLUMN column3 uuid NOT NULL
            DROP INDEX IF EXISTS table1_col2
            CREATE INDEX table1_col2 ON table1 USING btree(column2,column3)
FAIL
FAIL	github.com/covrom/goerd/schema	0.002s
FAIL
Related tags
Web Applications go golang postgresql erd
A restful api's with Gin Framework with a structured project that defaults to PostgreSQL database
 A restful api's with Gin Framework with a structured project that defaults to PostgreSQL database

Welcome to Golang Gin boilerplate v2 The fastest way to deploy a restful api's with Gin Framework with a structured project that defaults to PostgreSQ

Oct 15, 2021
GO API with Gin Gonic with postgresql using gorp

GO API with Gin Gonic with postgresql using gorp Tips: Make sure you have project in src folder of $GOPATH Also, iniitalize go mod init project-name a

Jul 28, 2022
Writing an application using Go and PostgreSQL by Henrique Vicente

pgxtutorial This is an accompanying repository of the article Back to basics: Writing an application using Go and PostgreSQL by Henrique Vicente. Feel

Dec 29, 2022
REST API written in GO with PostgreSQL and Nginx Proxy + Certbot Let's Encrypt HTTPS certificates + Graphical Frontend in PHP. Deployed via docker-compose.

SSOA-PT REST APP Services Backend: REST API in Go Database: PostgreSQL Web Proxy: Nginx Let's Encrypt HTTPS certificates with certbot Frontend: PHP Ap

Mar 19, 2022
Implementación una base de datos que almacene la información relativa a una tarjeta de crédito utilizando PostgreSQL y BoltDB.

Bases de Datos I: Trabajo Práctico 1. Introducción El objetivo del trabajo práctico es implementar una base de datos que almacene la información relat

Dec 20, 2021
Kaastolon - A fork of solon PostgreSQL HA With our customizations
 Kaastolon - A fork of solon PostgreSQL HA With our customizations

kaastolon a fork of stolon - PostgreSQL cloud native High Availability stolon is

Dec 30, 2022
sigurlx a web application attack surface mapping tool.

sigurlx a web application attack surface mapping tool, it does ...:

Jul 24, 2021
The cider is a simple tool of building GitHub pages. It's fast and easy to use.
The cider is a simple tool of building GitHub pages. It's fast and easy to use.

The cider is a simple tool of building GitHub pages. It's fast and easy to use. See example: https://www.leyafo.com Install Compiling from source code

Feb 13, 2022
urlhunter is a recon tool that allows searching on URLs that are exposed via shortener services such as bit.ly and goo.gl.
 urlhunter is a recon tool that allows searching on URLs that are exposed via shortener services such as bit.ly and goo.gl.

a recon tool that allows searching on URLs that are exposed via shortener services

Jan 7, 2023
Analytics box a simple and privacy focused analytics tool written in go like google analytics
 Analytics box a simple and privacy focused analytics tool written in go like google analytics

Analytics box is analytics tool like google analytics but instead of ripping user's privacy off them like google it respects their privacy and doesn't collect any unnecessary information.

Nov 27, 2022