Octillery replace import statement of database/sql to go.knocknote.io/octillery/database/sql. Therefore, OR Mapping library call Octillery's interface. and it can capture all queries.

We don't import the database/sql in project directly but in xorm, so should we replace the xorm pkg? if so how to solve the dependency problem with go mod?

If CSV includes "abcde\nfg" , we expect to that it is inserted the following.

abcde
fg

But, currently inserted value is the following.

abcde\nfg

So, I fixed this by unquoting with double quotation value.

As a default, I will support distributed transaction.

How to use

Set before/after callback for commit

octillery.BeforeCommitCallback(func(tx *sql.Tx, writeQueries []*sql.QueryLog) error {
  // write all writeQueries to log (WAL) in application side
  return nil
}))
octillery.AfterCommitCallback(func(tx *sql.Tx) error {}, func(tx *sql.Tx, isCriticalError bool, failureWriteQueries []*sql.QueryLog) error {
  if isCriticalError {
    // distributed transaction error
    // record failureWriteQueries and should execute after.
  }
}))

Also, callback function can set at *sql.Tx scope too.

tx, _ := db.Begin()
tx.BeforeCommitCallback(func(writeQueries []*sql.QueryLog) error {
  // write all writeQueries to log (WAL) in application side
  return nil
}))
tx.AfterCommitCallback(func() error {}, func(isCriticalError bool, failureWriteQueries []*sql.QueryLog) error {
  if isCriticalError {
    // distributed transaction error
    // record failureWriteQueries and should execute after.
  }
}))

Exec for multiple databases

You can use *sql.Tx to multiple databases.

(tableA and tableB are stored other databases)

tx, _ := db.Begin()
tx.Exec("insert into tableA ...") // (1)
tx.Exec("insert into tableB ...") // (2)

// tx.Commit() commit (1) before it commit (2)
// if (1) and (2) are success too, call AfterCommitSuccessCallback
// if (1) and (2) are failure too, call AfterCommitFailureCallback with NOT CriticalError
// if (1) is failure, instantly call AfterCommitFailureCallback with NOT CriticalError
// if (1) is OK but (2) is failure, call AfterCommitFailureCallback with CriticalError
if err := tx.Commit(); err != nil {
  ...
}

Recover from CriticalError

writeQueryLogs := queryLogsfromWAL()
tx, _ := db.Begin()
for _, writeQueryLog := range writeQueryLogs {
    isCommittedQuery, err := tx.IsAlreadyCommittedQueryLog(writeQueryLog)
    if err != nil {
       ....
    }
    if !isCommittedQuery {
        result, err := tx.ExecWithQueryLog(writeQueryLog)
        ....
    }
}

If disable distributed transaction support

set distributed_transaction: false in databases.yml

If application does not call tx.Query() or tx.Exec(), *sql.Tx.tx instance is nil . In this case, invalid memory address occur at WriteQueries or ReadQueries.
So I add nil check for that.

• Add go.mod and go.sum files to support Go Modules ( GO111MODULE=on )
• octillery install automatically put source of plugin to [$HOME/go|$GOPATH]/pkg/mod/go.knocknote.io/octillery@*/plugin if module-aware mode is enabled

connection.SetConfig calls setupDBFromConfig and creates database automatically.

In migration phase this function is really useful. But online application I use a strictly restricted database user ( don't have create database or any needless priviledges.

So I propose the function for skip auto create database.

default: &default
  adapter: mysql
  encoding: utf8mb4
  username: appuser
  password: ${appuser_password}

skip_auto_setup: ${skip_auto_setup}

[ Changed Summary ]

• Remove default plugins ( mysql.go and sqlite3.go ) from plugin directory
• Add .gitignore
• Fix typo of document
• Fix coding style ( it suggested by golint )

[Changed Summary]

• Add document for exported function suggested by golint
• Fix coding style ( it suggested by golint )
• Fix some codes by using gocyclo

[Changed Summary]

Add badge of goreportcard to README.md, and Fix badge style of CircleCI. Also, I fixed some codes by https://goreportcard.com/report/go.knocknote.io/octillery .

See https://github.com/knocknote/octillery/blob/feature/improve-goreportcard/README.md .

this PR resolves the following MySQL 8 problems .

• https://github.com/schemalex/schemalex/pull/85
• https://www.pixelstech.net/article/1531316568-Fix-this-authentication-plugin-is-not-supported-issue-while-using-Go-to-connect-MySQL-8

Since host and password change depending on the environment such as docker, local, GCP, I want to use the environment variable of OS for yml. ex)

default: &default
  adapter: mysql
  encoding: utf8mb4
  username: user
  password: ${DB_PASSWORD}
  database: dbname

tables:
  table1:
    shard: true
    shard_key: user_id
    shards:
      - table1_shard_1:
          <<: *default
          master:
            - ${DB_SHARD_1}:3306
      - table2_stone_shard_2:
          <<: *default
          master:
            - ${DB_SHARD_2}:3306

Can I modify config.Load to use ExpandEnv? ?

Refs: https://github.com/blastrain/octillery/issues/25

I'd like to add PostgreSQL support for octillery. I have added an adapter to support the new DB driver, but please let me know if there is anything else I need to do.

Call defer in a function scope instead of in a loop because a defer statement defers the execution of a function until the surrounding function returns.

I'd like to add PostgreSQL support for octillery. If it is ok to work for it, let me and my colleague make pull request .

It seems that only we need is adding adapter for supporting new DB driver. But please let me know if I'm missing something to add a a driver. https://github.com/blastrain/octillery/tree/master/connection/adapter/plugin

For diversity reasons, it would be nice to try to avoid 'master' and 'slave' terminology in this repository which can be associated to slavery. The master-slave terminology could be problematic for people in several countries which has the history of slavery like Romania, USA and many others. Thank you for considering the proposal. Let me know if any changes in the PR are needed, I would be happy to implement them.