go-waitgroup
How to use
An package that allows you to use the constructs of a sync.WaitGroup to create a pool of goroutines and control the concurrency.
Using it is just like a normal sync.WaitGroup. The only difference is the initialisation. When you use waitgroup.NewWaitGroup, you have the option to specify it's size.
Any int which is bigger than 0 will limit the number of concurrent goroutines. If you specify -1 or 0, all goroutines will run at once (just like a plain sync.WaitGroup).
package main
import (
"fmt"
"net/http"
"github.com/pieterclaerhout/go-waitgroup"
)
func main() {
urls := []string{
"https://www.easyjet.com/",
"https://www.skyscanner.de/",
"https://www.ryanair.com",
"https://wizzair.com/",
"https://www.swiss.com/",
}
wg := waitgroup.NewWaitGroup(3)
for _, url := range urls {
wg.BlockAdd()
go func(url string) {
defer wg.Done()
fmt.Printf("%s: checking\n", url)
res, err := http.Get(url)
if err != nil {
fmt.Println("Error: %v")
} else {
defer res.Body.Close()
fmt.Printf("%s: result: %v\n", url, err)
}
}(url)
}
wg.Wait()
fmt.Println("Finished")
}
Using closures
There is also a way to use function closures to make it even more readable:
package main
import (
"fmt"
"net/http"
"github.com/pieterclaerhout/go-waitgroup"
)
func main() {
urls := []string{
"https://www.easyjet.com/",
"https://www.skyscanner.de/",
"https://www.ryanair.com",
"https://wizzair.com/",
"https://www.swiss.com/",
}
wg := waitgroup.NewWaitGroup(3)
for _, url := range urls {
urlToCheck := url
wg.Add(func() {
fmt.Printf("%s: checking\n", urlToCheck)
res, err := http.Get(urlToCheck)
if err != nil {
fmt.Println("Error: %v")
} else {
defer res.Body.Close()
fmt.Printf("%s: result: %v\n", urlToCheck, err)
}
})
}
wg.Wait()
fmt.Println("Finished")
}
Handling errors
If you want to handle errors, there is also an ErrorGroup. This uses the same principles as a normal WaitGroup with a small twist.
First of all, you can only add functions which returns just an error.
Second, as soon as one of the queued items fail, the rest will be cancelled:
package main
import (
"context"
"fmt"
"os"
"github.com/pieterclaerhout/go-waitgroup"
)
func main() {
ctx := context.Background()
wg, ctx := waitgroup.NewErrorGroup(ctx, tc.size)
if err != nil {
fmt.Println("Error: %v")
os.Exit(1)
}
wg.Add(func() error {
return nil
})
wg.Add(func() error {
return errors.New("An error occurred")
})
if err := wg.Wait(); err != nil {
fmt.Println("Error: %v")
os.Exit(1)
}
}
You can also add multiple functions in one step:
package main
import (
"context"
"errors"
"fmt"
"os"
"github.com/pieterclaerhout/go-waitgroup"
)
func main() {
ctx := context.Background()
wg, ctx := waitgroup.NewErrorGroup(ctx, tc.size)
if err != nil {
fmt.Println("Error: %v")
os.Exit(1)
}
wg.Add(
func() error {
return nil
},
func() error {
return errors.New("An error occurred")
},
)
if err := wg.Wait(); err != nil {
fmt.Println("Error: %v")
os.Exit(1)
}
}
