Simple Gopher

Introduction

You'd probably want to check out ARCHITECTURE.md file to learn more on the architecture of the project layout.

Partial example of a service using hexagonal architecture approach with Go package oriented design. This serves as an example how to design Go services without a framework, high decoupling and easy testability. The example isn't 100% completed or fully covered, but good enough example on how it should look like. It will probably gain more polishing as there is always stuff to be added or updated, but for now, lets start with this.

Bellow is an example of how the README.md file for the service should look like.

Description

Won't get too much into the service itself, it's for you to discover and isn't too important, but to keep it shot, the Simple Gopher is a web API service that integrates with image resize service for resizing images and storing that data and integrates with AWS Cognito for authentication.

Table of contents

• Configuration
• Developing
  • Development requirements
  • Running locally
  • Dependency management
• Open Api 3 documentation
• Testing
  • Unit testing
  • Integration testing
  • Test configuration
• Migrations
  • Migration requirements
  • Running migrations
  • Migrations in CI/CD
• CI/CD
  • Deploying to CI/CD
• Troubleshooting
• Helpful materials

Configuration

Configuration of the application is done through the environment variables, which are following:

Required environment variables:

Developing

Development requirements

• Go v1.17
• PostgreSQL
• Docker and docker-compose

You will need to create a new aws cli profile locally and use it with credentials for this API. To create it execute aws configure --profile your-profile then export AWS_PROFILE=your-profile

Running locally

1. Set the environment variables. It's also recommended storing them in .env file which is ignored for ease of management. Export variables in each row like export MY_VARIABLE=1234, then load the env variables with source .env
2. Ensure that you start the database and all the other required services by running: docker-compose up -d
3. After setting the environment variables, execute the make start command to build and start the server

Dependency management

• Remove dependency by removing all occurrences of the library in imports and execute:

  go mod tidy

Open Api 3 documentation

Golang implementation of OpenApi3 specification aka Swagger through dynamic configuration with Swagger UI.

We use Swagger-UI with some small changes in order for it to fetch changes from our API where we can set the redirection url for OAuth2.

Dependencies

• github.com/getkin/kin-openapi/openapi3
• github.com/getkin/kin-openapi/openapi3gen

Documentation is added/updated in src/api/http-transport/openapi3.go, served via src/api/http-transport/handler_openapi3.go with swagger UI located in src/api/http-transport/docs.

Testing

Test configuration

Ensure that you set the DATABASE_TEST_URL environment variable, reason for new one is for safety reasons.

export DATABASE_TEST_URL=postgresql://postgres:example@localhost/db?sslmode=disable

# Also, disable the pooling, we don't need it for testing
export SQS_POST_AUTH_CONSUMER_DISABLED=true

Unit testing

• make verify

Integration testing

Unit testing with integration testing

• export TEST_INTEGRATION=true
• make verify

Vetting

Go provides a great tool for checking out code and detecting possible bugs, think of it as a linter. To run it execute:

make vet

Testing in CI/CD

Ensure that you run the migrations before running end-to-end tests. TBD: create separate e2e test execution

Migrations

Migration requirements

To run the migrations, you must set the database connection url through environment variable:

export DATABASE_URL=postgresql://postgres:example@localhost/db?sslmode=disable

Note that the sslmode=disable is for local development only, for production you should use ssl encryption.

Running migrations

BE CAREFUL WHEN REVERTING! Migrations are done with Go library golang-migrate/migrate and are executed with following commands:

• make migrate_up to update the schema to the latest
• make migrate_up_step to update the migration one step up
• make migrate_down to revert the schema one step down

It's paramount to follow best practices to ensure you don't break anything.

Migrations in CI/CD

For CI/CD ensure that you always run migrations before deploying your new app, like a pre-run action. For rollbacks, also ensure that you first close the application before running the migration down command. As always, for specific cases you will need to pay attention if you need code that supports both versions, so it doesn't crash if the change is drastic.

CI/CD

CI/CD is currently on the Heroku and additional options that were added for it are located in go.mod file as:

1. Build phase

// +heroku goVersion 1.17
// +heroku install ./cmd/...

2. Run phase in Procfile

They specify the Go version and build location. Builds will end up in bin/ directory. Other thing to note is that the bin/go-post-compile and bin/go-pre-compile will execute if they exist, so use them for pre and post actions.

Deploying to CI/CD

It is recommended to execute both unit and integration tests, as well as building the app before pushing to git.

# Test
export TEST_INTEGRATION=true
make test
make

More about Heroku options can be found at the Go Buildpack.

Troubleshooting

• Error when building: open /usr/local/go/pkg/darwin_amd64/runtime/cgo.a: permission denied

  sudo chmod -R 777 /usr/local/go

• How to clear test cache? Execute go clean -testcache

Helpful materials

• Go cheat sheet
• Learn Go with tests
• Go by example
• Effective Go
• Makefile tutorial

Advanced materials

• Network Programming with Go