Thin clones of PostgreSQL to build powerful development, test, QA, staging environments

Database Lab Engine (DLE)

Blazing-fast cloning of PostgreSQL databases 🐘
Thin clones of PostgreSQL to build powerful development, test, QA, staging environments.
Available for any PostgreSQL, including AWS RDS, GCP CloudSQL, Heroku, Digital Ocean, and self-managed instances.

Latest release

CI pipeline status Go report

Contributor Covenant Community Slack Twitter Follow

Why DLE?

  • Build dev/QA/staging environments based on full-size production-like databases.
  • Provide temporary full-size database clones for SQL query analysis and optimization (see also: SQL optimization chatbot Joe).
  • Automatically test database changes in CI/CD pipelines to avoid incidents in production.

For example, cloning a 1 TiB PostgreSQL database takes ~10 seconds. Dozens of independent clones are up and running on a single machine, supporting lots of development and testing activities, not increasing costs for hardware.

How it works

Thin cloning is fast because it uses Copy-on-Write (CoW). DLE supports two technologies to enable CoW and thin cloning: ZFS (default) and LVM.

With ZFS, Database Lab Engine periodically creates a new snapshot of the data directory and maintains a set of snapshots, cleaning up the old and unused ones. When requesting a new clone, users can choose which snapshot to use.

Read more:

Where to start

Case studies

Features

  • Blazing-fast cloning of Postgres databases – a few seconds to create a new clone ready to accept connections and queries, regardless of the database size.
  • The theoretical maximum number of snapshots and clones is 264 (ZFS, default).
  • The theoretical maximum size of PostgreSQL data directory: 256 quadrillion zebibytes, or 2128 bytes (ZFS, default).
  • PostgreSQL major versions supported: 9.6–14.
  • Two technologies are supported to enable thin cloning (CoW): ZFS and LVM.
  • All components are packaged in Docker containers.
  • UI to make manual work more convenient.
  • API and CLI to automate the work with DLE snapshots and clones.
  • By default, PostgreSQL containers include many popular extensions (docs).
  • PostgreSQL containers can be customized (docs).
  • Source database can be located anywhere (self-managed Postgres, AWS RDS, GCP CloudSQL, Azure, Timescale Cloud, and so on) and does NOT require any adjustments. There are NO requirements to install ZFS or Docker to the source (production) databases.
  • Initial data provisioning can be at both physical (pg_basebackup, backup / archiving tools such as WAL-G or pgBackRest), or logical (dump/restore directly from the source or from files stored at AWS S3) levels.
  • For the logical mode, partial data retrieval is supported (specific databases, specific tables).
  • For the physical mode, a continuously updated state is supported ("sync container"), making DLE a specialized version of standby Postgres.
  • For the logical mode, periodical full refresh is supported, automated, and controlled by DLE. It is possible to use multiple disks containing different versions of the database, so full refresh won't require downtime.
  • Fast Point in Time Recovery (PITR) to the points available in DLE snapshots.
  • Unused clones are automatically deleted.
  • "Deletion protection" flag can be used to block automatic or manual deletion of clones.
  • Snapshot retention policies supported in DLE configuration.
  • Persistent clones: clones survive DLE restarts (including full VM reboots).
  • The "reset" command can be used to switch to a different version of data.
  • DB Migration Checker component collects various artifacts useful for DB testing in CI (docs).
  • SSH port forwarding for API and Postgres connections.
  • Docker container config parameters can be specified in the DLE config.
  • Resource usage quotas for clones: CPU, RAM (container quotas, supported by Docker)
  • Postgres config parameters can be specified in the DLE config (separately for clones, the "sync" container, and the "promote" container).
  • Monitoring: auth-free /healthz API endpoint, extended /status (requires auth), Netdata module.

How to contribute

Give the project a star

The easiest way to contribute is to give the project a GitHub/GitLab star:

Add a star

Mention that you use DLE

Please post a tweet mentioning @Database_Lab or share the link to this repo in your favorite social network.

If you are actively using DLE at work, think about where you could mention it. The best way of mentioning it is using graphics with a link. Brand assets can be found in the ./assets folder. Feel free to put them in your documents, slide decks, application, and website interfaces to show that you use DLE.

HTML snippet for lighter backgrounds:

">
<a href="http://databaselab.io">
  <img width="400" src="https://postgres.ai/assets/powered-by-dle-for-light-background.svg" />
a>

Fro darker backgrounds:

">
<a href="http://databaselab.io">
  <img width="400" src="https://postgres.ai/assets/powered-by-dle-for-dark-background.svg" />
a>

Propose an idea or report a bug

Check out our contributing guide for more details.

Participate in development

Check out our contributing guide for more details.

Reference guides

How-to guides

More you can found in the "How-to guides" section of the docs.

Miscellaneous

License

DLE source code is licensed under the OSI-approved open source license GNU Affero General Public License version 3 (AGPLv3).

Reach out to the Postgres.ai team if you want a trial or commercial license that does not contain the GPL clauses: Contact page.

FOSSA Status

Community & Support

Contributor Covenant

Owner
Postgres.ai
We eliminate database-related roadblocks on the way of developers, DBAs and QA engineers – Find us on GitLab.com
Postgres.ai
Comments
  • VM Freeze

    VM Freeze

    After some time of running up, Hyper-V VM starts up consuming CPU ("sy" around 90-100%), then it takes up to a few minutes to (re)start sync-instance container. Finally, VM hangs on completely; it may be impossible to restart/stop VM via Hyper-V commands.

    VM specs:

    • 10 GB RAM
    • 2 vCPU (Yeap, it's just a sandbox, I want to master this tool)
    • 32 GB system disk (VHDX over SATA SSD)
    • 500 GB ZFS disk (VHDX over SATA HDD)
    • Ubuntu 20.04 LTS (just Docker & ZFS & Database Lab & VS Code installed by its Windows UI).

    Host specs:

    • 64 GB RAM
    • Core i5 6400
    • Windows 10 Pro 20H2 & Hyper-V.

    As it was last time: Снимок экрана 2020-12-06 180828

  • chore(deps): bump moment from 2.24.0 to 2.29.4 in /ui/packages/platform

    chore(deps): bump moment from 2.24.0 to 2.29.4 in /ui/packages/platform

    Bumps moment from 2.24.0 to 2.29.4.

    Changelog

    Sourced from moment's changelog.

    2.29.4

    • Release Jul 6, 2022
      • #6015 [bugfix] Fix ReDoS in preprocessRFC2822 regex

    2.29.3 Full changelog

    • Release Apr 17, 2022
      • #5995 [bugfix] Remove const usage
      • #5990 misc: fix advisory link

    2.29.2 See full changelog

    • Release Apr 3 2022

    Address https://github.com/moment/moment/security/advisories/GHSA-8hfj-j24r-96c4

    2.29.1 See full changelog

    • Release Oct 6, 2020

    Updated deprecation message, bugfix in hi locale

    2.29.0 See full changelog

    • Release Sept 22, 2020

    New locales (es-mx, bn-bd). Minor bugfixes and locale improvements. More tests. Moment is in maintenance mode. Read more at this link: https://momentjs.com/docs/#/-project-status/

    2.28.0 See full changelog

    • Release Sept 13, 2020

    Fix bug where .format() modifies original instance, and locale updates

    2.27.0 See full changelog

    • Release June 18, 2020

    Added Turkmen locale, other locale improvements, slight TypeScript fixes

    2.26.0 See full changelog

    • Release May 19, 2020

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

  • chore(deps): bump follow-redirects from 1.14.0 to 1.14.8 in /ui/packages/platform

    chore(deps): bump follow-redirects from 1.14.0 to 1.14.8 in /ui/packages/platform

    Bumps follow-redirects from 1.14.0 to 1.14.8.

    Commits
    • 3d81dc3 Release version 1.14.8 of the npm package.
    • 62e546a Drop confidential headers across schemes.
    • 2ede36d Release version 1.14.7 of the npm package.
    • 8b347cb Drop Cookie header across domains.
    • 6f5029a Release version 1.14.6 of the npm package.
    • af706be Ignore null headers.
    • d01ab7a Release version 1.14.5 of the npm package.
    • 40052ea Make compatible with Node 17.
    • 86f7572 Fix: clear internal timer on request abort to avoid leakage
    • 2e1eaf0 Keep Authorization header on subdomain redirects.
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

  • chore(deps): bump d3-color, d3 and d3-flame-graph in /ui/packages/platform

    chore(deps): bump d3-color, d3 and d3-flame-graph in /ui/packages/platform

    Bumps d3-color to 3.1.0 and updates ancestor dependencies d3-color, d3 and d3-flame-graph. These dependencies need to be updated together.

    Updates d3-color from 1.4.0 to 3.1.0

    Release notes

    Sourced from d3-color's releases.

    v3.1.0

    v3.0.1

    • Make build reproducible.

    v3.0.0

    • Adopt type: module.

    This package now requires Node.js 12 or higher. For more, please read Sindre Sorhus’s FAQ.

    v2.0.0

    This release adopts ES2015 language features such as for-of and drops support for older browsers, including IE. If you need to support pre-ES2015 environments, you should stick with d3-color 1.x or use a transpiler.

    v1.4.1

    • Fix parsing of 4- and 8-digit hexadecimal transparent colors. #52
    Commits

    Updates d3 from 5.12.0 to 7.6.1

    Release notes

    Sourced from d3's releases.

    v7.6.1

    v7.6.0

    v7.5.0

    v7.4.5

    v7.4.4

    • Fix incorrect behavior of d3.bisector when given an asymmetric comparator.

    v7.4.3

    v7.4.2

    • Fix off-by-one bin assignment due to rounding error in d3.bin.

    v7.4.1

    • Significantly improve the performance of d3.bin.
    • Fix the implementation of d3.thresholdScott.
    • d3.pack and d3.packEnclose are now fully deterministic.
    • d3.pack and d3.packEnclose now handle certain floating point errors better.

    v7.4.0

    v7.3.0

    v7.2.1

    • Fix stratify.path when the top-level directory is only a single character.

    ... (truncated)

    Commits

    Updates d3-flame-graph from 2.1.3 to 4.1.3

    Commits
    • bb0037d task: bumping version
    • 70cabab Don't animate left & top properties
    • 815b265 task: bump version
    • a5e14bb Merge branch 'mhansen-tip'
    • c94f56d Update examples and README to use tooltip.text()
    • 8861889 Add tooltip.text(), deprecate tooltip.html()
    • db721ad Add some more tests, and clean up DOM
    • eeebc69 Add test for tooltips
    • 16e538f Remove references to d3-tip
    • 84ab4b7 fix: details handler vul
    • Additional commits viewable in compare view

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

  • chore(deps): bump terser from 5.12.1 to 5.14.2 in /ui/packages/platform

    chore(deps): bump terser from 5.12.1 to 5.14.2 in /ui/packages/platform

    Bumps terser from 5.12.1 to 5.14.2.

    Changelog

    Sourced from terser's changelog.

    v5.14.2

    • Security fix for RegExps that should not be evaluated (regexp DDOS)
    • Source maps improvements (#1211)
    • Performance improvements in long property access evaluation (#1213)

    v5.14.1

    • keep_numbers option added to TypeScript defs (#1208)
    • Fixed parsing of nested template strings (#1204)

    v5.14.0

    • Switched to @​jridgewell/source-map for sourcemap generation (#1190, #1181)
    • Fixed source maps with non-terminated segments (#1106)
    • Enabled typescript types to be imported from the package (#1194)
    • Extra DOM props have been added (#1191)
    • Delete the AST while generating code, as a means to save RAM

    v5.13.1

    • Removed self-assignments (varname=varname) (closes #1081)
    • Separated inlining code (for inlining things into references, or removing IIFEs)
    • Allow multiple identifiers with the same name in var destructuring (eg var { a, a } = x) (#1176)

    v5.13.0

    • All calls to eval() were removed (#1171, #1184)
    • source-map was updated to 0.8.0-beta.0 (#1164)
    • NavigatorUAData was added to domprops to avoid property mangling (#1166)
    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

  • chore(deps): bump terser from 4.8.0 to 4.8.1 in /ui

    chore(deps): bump terser from 4.8.0 to 4.8.1 in /ui

    Bumps terser from 4.8.0 to 4.8.1.

    Changelog

    Sourced from terser's changelog.

    v4.8.1 (backport)

    • Security fix for RegExps that should not be evaluated (regexp DDOS)
    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

  • chore(deps): bump eventsource from 1.1.0 to 1.1.2 in /ui

    chore(deps): bump eventsource from 1.1.0 to 1.1.2 in /ui

    Bumps eventsource from 1.1.0 to 1.1.2.

    Changelog

    Sourced from eventsource's changelog.

    1.1.2

    • Inline origin resolution, drops original dependency (#281 Espen Hovlandsdal)

    1.1.1

    • Do not include authorization and cookie headers on redirect to different origin (#273 Espen Hovlandsdal)
    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

  • chore(deps): bump moment from 2.29.1 to 2.29.4 in /ui

    chore(deps): bump moment from 2.29.1 to 2.29.4 in /ui

    Bumps moment from 2.29.1 to 2.29.4.

    Changelog

    Sourced from moment's changelog.

    2.29.4

    • Release Jul 6, 2022
      • #6015 [bugfix] Fix ReDoS in preprocessRFC2822 regex

    2.29.3 Full changelog

    • Release Apr 17, 2022
      • #5995 [bugfix] Remove const usage
      • #5990 misc: fix advisory link

    2.29.2 See full changelog

    • Release Apr 3 2022

    Address https://github.com/moment/moment/security/advisories/GHSA-8hfj-j24r-96c4

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

  • chore(deps): bump follow-redirects from 1.14.4 to 1.15.1 in /ui/packages/ce

    chore(deps): bump follow-redirects from 1.14.4 to 1.15.1 in /ui/packages/ce

    Bumps follow-redirects from 1.14.4 to 1.15.1.

    Commits
    • 62a551c Release version 1.15.1 of the npm package.
    • 7fe0779 Use for ... of.
    • 948c30c Fix redirecting to relative URL when using proxy
    • 22e81fc Release version 1.15.0 of the npm package.
    • 96a3947 Add headers to request details
    • 24dcb20 Pass status & request details to beforeRedirect (#198)
    • 7abae9b Test on Node 18.
    • 79d0d8d refactor: replace deprecated String.prototype.substr() (#196)
    • 13136e9 Release version 1.14.9 of the npm package.
    • 2ec9b0b Keep headers when upgrading from HTTP to HTTPS.
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

  • chore(deps): bump nanoid from 3.1.30 to 3.3.4 in /ui/packages/ce

    chore(deps): bump nanoid from 3.1.30 to 3.3.4 in /ui/packages/ce

    Bumps nanoid from 3.1.30 to 3.3.4.

    Changelog

    Sourced from nanoid's changelog.

    3.3.4

    3.3.3

    • Reduced size (by Anton Khlynovskiy).

    3.3.2

    • Fixed enhanced-resolve support.

    3.3.1

    • Reduced package size.

    3.3

    • Added size argument to function from customAlphabet (by Stefan Sundin).

    3.2

    • Added --size and --alphabet arguments to binary (by Vitaly Baev).

    3.1.32

    • Reduced async exports size (by Artyom Arutyunyan).
    • Moved from Jest to uvu (by Vitaly Baev).

    3.1.31

    • Fixed collision vulnerability on object in size (by Artyom Arutyunyan).
    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

  • chore(deps): bump async from 2.6.3 to 3.2.3 in /ui/packages/ce

    chore(deps): bump async from 2.6.3 to 3.2.3 in /ui/packages/ce

    Bumps async from 2.6.3 to 3.2.3.

    Changelog

    Sourced from async's changelog.

    v3.2.3

    • Fix bugs in comment parsing in autoInject. (#1767, #1780)

    v3.2.2

    • Fix potential prototype pollution exploit

    v3.2.1

    v3.2.0

    • Fix a bug in Safari related to overwriting func.name
    • Remove built-in browserify configuration (#1653)
    • Varios doc fixes (#1688, #1703, #1704)

    v3.1.1

    • Allow redefining name property on wrapped functions.

    v3.1.0

    • Added q.pushAsync and q.unshiftAsync, analagous to q.push and q.unshift, except they always do not accept a callback, and reject if processing the task errors. (#1659)
    • Promises returned from q.push and q.unshift when a callback is not passed now resolve even if an error ocurred. (#1659)
    • Fixed a parsing bug in autoInject with complicated function bodies (#1663)
    • Added ES6+ configuration for Browserify bundlers (#1653)
    • Various doc fixes (#1664, #1658, #1665, #1652)

    v3.0.1

    Bug fixes

    • Fixed a regression where arrays passed to queue and cargo would be completely flattened. (#1645)
    • Clarified Async's browser support (#1643)

    v3.0.0

    The async/await release!

    There are a lot of new features and subtle breaking changes in this major version, but the biggest feature is that most Async methods return a Promise if you omit the callback, meaning you can await them from within an async function.

    const results = await async.mapLimit(urls, 5, async url => {
        const resp = await fetch(url)
        return resp.body
    })
    

    Breaking Changes

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

  • CVE-2022-46175 (High) detected in json5-2.2.1.tgz, json5-1.0.1.tgz

    CVE-2022-46175 (High) detected in json5-2.2.1.tgz, json5-1.0.1.tgz

    CVE-2022-46175 - High Severity Vulnerability

    Vulnerable Libraries - json5-2.2.1.tgz, json5-1.0.1.tgz

    json5-2.2.1.tgz

    JSON for humans.

    Library home page: https://registry.npmjs.org/json5/-/json5-2.2.1.tgz

    Dependency Hierarchy:

    • @postgres.ai/ce-1.0.0.tgz (Root Library)
      • core-7.19.0.tgz
        • :x: json5-2.2.1.tgz (Vulnerable Library)
    json5-1.0.1.tgz

    JSON for humans.

    Library home page: https://registry.npmjs.org/json5/-/json5-1.0.1.tgz

    Dependency Hierarchy:

    • @postgres.ai/platform-1.0.0.tgz (Root Library)
      • react-scripts-5.0.0.tgz
        • babel-loader-8.2.3.tgz
          • loader-utils-1.4.0.tgz
            • :x: json5-1.0.1.tgz (Vulnerable Library)

    Found in HEAD commit: b3ac62d12e3d43994ff7ad836e34da801ed665fb

    Found in base branch: master

    Vulnerability Details

    JSON5 is an extension to the popular JSON file format that aims to be easier to write and maintain by hand (e.g. for config files). The parse method of the JSON5 library before and including version 2.2.1 does not restrict parsing of keys named __proto__, allowing specially crafted strings to pollute the prototype of the resulting object. This vulnerability pollutes the prototype of the object returned by JSON5.parse and not the global Object prototype, which is the commonly understood definition of Prototype Pollution. However, polluting the prototype of a single object can have significant security impact for an application if the object is later used in trusted operations. This vulnerability could allow an attacker to set arbitrary and unexpected keys on the object returned from JSON5.parse. The actual impact will depend on how applications utilize the returned object and how they filter unwanted keys, but could include denial of service, cross-site scripting, elevation of privilege, and in extreme cases, remote code execution. JSON5.parse should restrict parsing of __proto__ keys when parsing JSON strings to objects. As a point of reference, the JSON.parse method included in JavaScript ignores __proto__ keys. Simply changing JSON5.parse to JSON.parse in the examples above mitigates this vulnerability. This vulnerability is patched in json5 version 2.2.2 and later.

    Publish Date: 2022-12-24

    URL: CVE-2022-46175

    CVSS 3 Score Details (7.1)

    Base Score Metrics:

    • Exploitability Metrics:
      • Attack Vector: Network
      • Attack Complexity: High
      • Privileges Required: Low
      • User Interaction: None
      • Scope: Unchanged
    • Impact Metrics:
      • Confidentiality Impact: High
      • Integrity Impact: Low
      • Availability Impact: High

    For more information on CVSS3 Scores, click here.

    Suggested Fix

    Type: Upgrade version

    Origin: https://www.cve.org/CVERecord?id=CVE-2022-46175

    Release Date: 2022-12-24

    Fix Resolution: json5 - 2.2.2


    Step up your Open Source Security Game with Mend here

  • CVE-2022-23471 (Medium) detected in github.com/containerd/containerd-v1.6.1

    CVE-2022-23471 (Medium) detected in github.com/containerd/containerd-v1.6.1

    CVE-2022-23471 - Medium Severity Vulnerability

    Vulnerable Library - github.com/containerd/containerd-v1.6.1

    An open and reliable container runtime

    Library home page: https://proxy.golang.org/github.com/containerd/containerd/@v/v1.6.1.zip

    Dependency Hierarchy:

    • github.com/docker/docker-v20.10.3-0.20220207145910-4b3471ddc064+incompatible (Root Library)
      • :x: github.com/containerd/containerd-v1.6.1 (Vulnerable Library)

    Found in HEAD commit: b3ac62d12e3d43994ff7ad836e34da801ed665fb

    Found in base branch: master

    Vulnerability Details

    containerd is an open source container runtime. A bug was found in containerd's CRI implementation where a user can exhaust memory on the host. In the CRI stream server, a goroutine is launched to handle terminal resize events if a TTY is requested. If the user's process fails to launch due to, for example, a faulty command, the goroutine will be stuck waiting to send without a receiver, resulting in a memory leak. Kubernetes and crictl can both be configured to use containerd's CRI implementation and the stream server is used for handling container IO. This bug has been fixed in containerd 1.6.12 and 1.5.16. Users should update to these versions to resolve the issue. Users unable to upgrade should ensure that only trusted images and commands are used and that only trusted users have permissions to execute commands in running containers.

    Publish Date: 2022-12-07

    URL: CVE-2022-23471

    CVSS 3 Score Details (6.5)

    Base Score Metrics:

    • Exploitability Metrics:
      • Attack Vector: Network
      • Attack Complexity: Low
      • Privileges Required: Low
      • User Interaction: None
      • Scope: Unchanged
    • Impact Metrics:
      • Confidentiality Impact: None
      • Integrity Impact: None
      • Availability Impact: High

    For more information on CVSS3 Scores, click here.

    Suggested Fix

    Type: Upgrade version

    Origin: https://www.cve.org/CVERecord?id=CVE-2022-23471

    Release Date: 2022-12-07

    Fix Resolution: v1.6.12


    Step up your Open Source Security Game with Mend here

  • CVE-2022-30633 (High) detected in golang.org/x/net-v0.0.0-20211216030914-fe4d6282115f

    CVE-2022-30633 (High) detected in golang.org/x/net-v0.0.0-20211216030914-fe4d6282115f

    CVE-2022-30633 - High Severity Vulnerability

    Vulnerable Library - golang.org/x/net-v0.0.0-20211216030914-fe4d6282115f

    Library home page: https://proxy.golang.org/golang.org/x/net/@v/v0.0.0-20211216030914-fe4d6282115f.zip

    Dependency Hierarchy:

    • github.com/golang/oauth2-v0.1.0 (Root Library)
      • :x: golang.org/x/net-v0.0.0-20211216030914-fe4d6282115f (Vulnerable Library)

    Found in HEAD commit: b3ac62d12e3d43994ff7ad836e34da801ed665fb

    Found in base branch: master

    Vulnerability Details

    Uncontrolled recursion in Unmarshal in encoding/xml before Go 1.17.12 and Go 1.18.4 allows an attacker to cause a panic due to stack exhaustion via unmarshalling an XML document into a Go struct which has a nested field that uses the 'any' field tag.

    Publish Date: 2022-08-10

    URL: CVE-2022-30633

    CVSS 3 Score Details (7.5)

    Base Score Metrics:

    • Exploitability Metrics:
      • Attack Vector: Network
      • Attack Complexity: Low
      • Privileges Required: None
      • User Interaction: None
      • Scope: Unchanged
    • Impact Metrics:
      • Confidentiality Impact: None
      • Integrity Impact: None
      • Availability Impact: High

    For more information on CVSS3 Scores, click here.

    Suggested Fix

    Type: Upgrade version

    Origin: https://security-tracker.debian.org/tracker/CVE-2022-30633

    Release Date: 2022-05-13

    Fix Resolution: go1.17.12,go1.18.4


    Step up your Open Source Security Game with Mend here

  • CVE-2022-41717 (Medium) detected in golang.org/x/sys-v0.0.0-20211216021012-1d35b9e2eb4e

    CVE-2022-41717 (Medium) detected in golang.org/x/sys-v0.0.0-20211216021012-1d35b9e2eb4e

    CVE-2022-41717 - Medium Severity Vulnerability

    Vulnerable Library - golang.org/x/sys-v0.0.0-20211216021012-1d35b9e2eb4e

    Library home page: https://proxy.golang.org/golang.org/x/sys/@v/v0.0.0-20211216021012-1d35b9e2eb4e.zip

    Dependency Hierarchy:

    • github.com/Shirou/gopsutil-v2.20.9+incompatible (Root Library)
      • :x: golang.org/x/sys-v0.0.0-20211216021012-1d35b9e2eb4e (Vulnerable Library)

    Found in HEAD commit: b3ac62d12e3d43994ff7ad836e34da801ed665fb

    Found in base branch: master

    Vulnerability Details

    An attacker can cause excessive memory growth in a Go server accepting HTTP/2 requests. HTTP/2 server connections contain a cache of HTTP header keys sent by the client. While the total number of entries in this cache is capped, an attacker sending very large keys can cause the server to allocate approximately 64 MiB per open connection.

    Publish Date: 2022-12-08

    URL: CVE-2022-41717

    CVSS 3 Score Details (5.3)

    Base Score Metrics:

    • Exploitability Metrics:
      • Attack Vector: Network
      • Attack Complexity: Low
      • Privileges Required: None
      • User Interaction: None
      • Scope: Unchanged
    • Impact Metrics:
      • Confidentiality Impact: None
      • Integrity Impact: None
      • Availability Impact: Low

    For more information on CVSS3 Scores, click here.

    Suggested Fix

    Type: Upgrade version

    Release Date: 2022-12-08

    Fix Resolution: go1.19.4


    Step up your Open Source Security Game with Mend here

  • CVE-2022-28131 (High) detected in golang.org/x/net-v0.0.0-20211216030914-fe4d6282115f

    CVE-2022-28131 (High) detected in golang.org/x/net-v0.0.0-20211216030914-fe4d6282115f

    CVE-2022-28131 - High Severity Vulnerability

    Vulnerable Library - golang.org/x/net-v0.0.0-20211216030914-fe4d6282115f

    Library home page: https://proxy.golang.org/golang.org/x/net/@v/v0.0.0-20211216030914-fe4d6282115f.zip

    Dependency Hierarchy:

    • github.com/golang/oauth2-v0.1.0 (Root Library)
      • :x: golang.org/x/net-v0.0.0-20211216030914-fe4d6282115f (Vulnerable Library)

    Found in HEAD commit: b3ac62d12e3d43994ff7ad836e34da801ed665fb

    Found in base branch: master

    Vulnerability Details

    In Decoder.Skip in encoding/xml in Go before 1.17.12 and 1.18.x before 1.18.4, stack exhaustion and a panic can occur via a deeply nested XML document.

    Publish Date: 2022-08-10

    URL: CVE-2022-28131

    CVSS 3 Score Details (7.5)

    Base Score Metrics:

    • Exploitability Metrics:
      • Attack Vector: Network
      • Attack Complexity: Low
      • Privileges Required: None
      • User Interaction: None
      • Scope: Unchanged
    • Impact Metrics:
      • Confidentiality Impact: None
      • Integrity Impact: None
      • Availability Impact: High

    For more information on CVSS3 Scores, click here.

    Suggested Fix

    Type: Upgrade version

    Origin: https://security-tracker.debian.org/tracker/CVE-2022-28131

    Release Date: 2022-03-29

    Fix Resolution: go1.17.12,go1.18.4


    Step up your Open Source Security Game with Mend here

pREST (PostgreSQL REST), simplify and accelerate development, ⚡ instant, realtime, high-performance on any Postgres application, existing or new

pREST pREST (PostgreSQL REST), simplify and accelerate development, instant, realtime, high-performance on any Postgres application, existing or new P

Jan 9, 2023
a powerful mysql toolset with Go
a powerful mysql toolset with Go

go-mysql A pure go library to handle MySQL network protocol and replication. Call for Committer/Maintainer Sorry that I have no enough time to maintai

Dec 28, 2022
A flexible and powerful SQL string builder library plus a zero-config ORM.

SQL builder for Go Install Usage Basic usage Pre-defined SQL builders Build SQL for MySQL, PostgreSQL or SQLite Using Struct as a light weight ORM Nes

Dec 30, 2022
pg_timetable: Advanced scheduling for PostgreSQL
pg_timetable: Advanced scheduling for PostgreSQL

pg_timetable: Advanced scheduling for PostgreSQL pg_timetable is an advanced job scheduler for PostgreSQL, offering many advantages over traditional s

Dec 29, 2022
Cross-platform client for PostgreSQL databases

pgweb Web-based PostgreSQL database browser written in Go. Overview Pgweb is a web-based database browser for PostgreSQL, written in Go and works on O

Dec 30, 2022
PostgreSQL style Parser splitted from CockroachDB

What's this PostgreSQL style Parser splitted from CockroachDB See: Complex SQL format example

Jan 5, 2023
Enhanced PostgreSQL logical replication

pgcat - Enhanced postgresql logical replication Why pgcat? Architecture Build from source Install Run Conflict handling Table mapping Replication iden

Dec 21, 2022
Interactive client for PostgreSQL and MySQL
Interactive client for PostgreSQL and MySQL

dblab Interactive client for PostgreSQL and MySQL. Overview dblab is a fast and lightweight interactive terminal based UI application for PostgreSQL a

Jan 8, 2023
WAL-G is an archival restoration tool for PostgreSQL, MySQL/MariaDB, and MS SQL Server (beta for MongoDB and Redis).

WAL-G is an archival restoration tool for PostgreSQL, MySQL/MariaDB, and MS SQL Server (beta for MongoDB and Redis).

Jan 1, 2023
Worker failover support for PostgreSQL Citus extension using pg_auto_failover.

citus-failover Worker failover support for citus community version using pg_auto_failover. What is this? This is a simple service to monitor changes i

Dec 7, 2022
Interactive terminal user interface and CLI for database connections. MySQL, PostgreSQL. More to come.
Interactive terminal user interface and CLI for database connections. MySQL, PostgreSQL. More to come.

?? dbui dbui is the terminal user interface and CLI for database connections. It provides features like, Connect to multiple data sources and instance

Jan 5, 2023
Modify orca-zhang/borm in order to use in PostgreSQL

borm ??️ 针对 orca-zhang/borm 进行了修改,暂时只能兼容PostgreSQL 原因 在b站时候用过borm,用起来感觉非常简洁 自己学校里用PostgreSQL比较多 可变条件真的非常好用 问题 首先需要注意的是,这是写给PG的 PG 根本不存在某些 MySQL 独有的函数

Aug 24, 2022
Cross-platform client for PostgreSQL databases

pgweb Web-based PostgreSQL database browser written in Go. Overview Pgweb is a web-based database browser for PostgreSQL, written in Go and works on O

Dec 30, 2022
PolarDB Cluster Manager is the cluster management component of PolarDB for PostgreSQL, responsible for topology management, high availability, configuration management, and plugin extensions.

What is PolarDB Cluster Manager PolarDB Cluster Manager is the cluster management component of PolarDB for PostgreSQL, responsible for topology manage

Nov 9, 2022
A simple Golang-based application that queries a PostgreSQL database

Qwik-E-Mart Demo App A simple Golang-based application that queries a PostgreSQL database named qwikemart to read and return customer data stored in t

Nov 6, 2021
CloudQuery extracts, transforms, and loads your cloud assets into normalized PostgreSQL tables.
CloudQuery extracts, transforms, and loads your cloud assets into normalized PostgreSQL tables.

The open-source cloud asset inventory backed by SQL. CloudQuery extracts, transforms, and loads your cloud assets into normalized PostgreSQL tables. C

Dec 31, 2022
Implemented PostgreSQL with Golang
Implemented PostgreSQL with Golang

Customer Information Web Api Implemented PostgreSQL with Golang docker run --name postgresql-container -p 5432:5432 -e POSTGRES_PASSWORD=Password! -d

Nov 15, 2021
Typescript type declaration to PostgreSQL CREATE TABLE converter

ts2psql NOTE: This is WIP. Details in this readme are ideal state. Current usage: go build && ./ts2psql (or go build && ts2psql if on Windows OS). A s

Jan 13, 2022
An experimental toolkit for injecting alternate authentication strategies into a PostgreSQL-compatible wire format.

PG Auth Proxy This is an experimental toolkit for injecting alternate authentication strategies into a PostgreSQL-compatible wire format. This is a pr

Jan 20, 2022