I recently wrote a couple programs that relied on a separate process to continuously do some work. Whenever I’ve written concurrent programs in Go I usually go through the same process of reminding myself how channels work, what gets blocked, and when we need to rely on a WaitGroup.
This blog is written with future me in mind when I need to remind myself how Go handles concurrency (also writing it like this I’m more likely to remember). This does not cover concurrent data access (i.e. mutexes and semaphores).
Goroutines
Goroutines are Go’s method of concurrency. They operate simultaneously alongside other routines. Every Go program has at least one goroutine – which is kicked off through main.
To start another goroutine you can use the keyword go in front of a method or function call.
Example:
func hello() {
fmt.Print(“hello ”)
}
func main(){
go Hello()
go func(){
fmt.Println(“world”)
}()
}
Channels
A channel is one of the main ways goroutines talk to each other. A huge advantage of it is that the communication is lock free.
Channels come in two forms: Buffered and unbuffered.
Since Go is a typed language, you need to specify what type of object the channel will pass around (interface{} works here too if your program needs to be type agnostic).
Unbuffered Channels
This is the default channel creation. You pass values from one goroutine to the other one at a time. The sending goroutine will block until the channel is read from.
unbufferedChannel := make(chan int)
Buffered Channels
Are particularly useful when you know how many goroutines you have launched or want to impose better limits on your program. You can create a channel that contains a specified length, and up until the buffer is full the go routines won’t block when they write to it.
// creates a channel with a buffer size of 10
bufferedChannel := make(chan int, 10)
Further reading:
- How to build an unbounded channel: https://medium.com/capital-one-tech/building-an-unbounded-channel-in-go-789e175cd2cd
- Buffered Channels in Go – What Are They Good For? https://medium.com/capital-one-tech/buffered-channels-in-go-what-are-they-good-for-43703871828
Writing to a channel
Writing to a channel is incredibly easy, all you have to do is the following
c := make(chan int, 10)
c <- x
Note that on some channels you can’t write to them because they are saved as “read only” channels. For example, the following would NOT compile:
func main() {
c := make(chan int, 1)
cantWriteToReadChannel(c)
}
func cantWriteToReadChannel(c <-chan int){
c <- 2
}
c <- chan int specifies a read only channel. To specify a write only channel you can move the arrow like so c chan<- int. Thus the following would compile:
func main() {
c := make(chan int, 1)
cantWriteToReadChannel(c)
}
func cantWriteToReadChannel(c chan<- int){
c <- 2
}
Reading from a channel
The simplest way to read from a channel is the following:
package main
import "fmt"
func main() {
c := make(chan string)
go sendString(c)
msg := <-c
fmt.Println(msg)
}
func sendString(c chan<- string) {
c <- "Read from single channel"
}
Output:
Read from single channel
It’s rare however we only want to send one message between goroutines. To continuously read we can set up a for loop and range over the channel.
for i := range c {
fmt.Println(i)
}
Example usage:
package main
import "fmt"
func main() {
c := make(chan int)
go sendInt(c)
for i := range c {
fmt.Println(i)
if i == 2 {
return
}
}
}
func sendInt(c chan<- int) {
for x := 0; x < 5; x++ {
c <- x
}
}
Outputs:
0
1
2
Closing a channel
In the above example of reading a channel, we break out of the for loop when we reach a certain input. This obviously isn’t what we want in real life. What if we want to read everything put on a channel and only exit when we’re done? We can close the channel, and the for loop will then be able to iterate what’s left in the channel and exit.
package main
import "fmt"
func main() {
c := make(chan int)
go sendInt(c)
for i := range c {
fmt.Println(i)
}
}
func sendInt(c chan<- int) {
for x := 0; x < 10; x++ {
c <- x
}
close(c)
}
Outputs:
0
1
2
3
4
Reading from multiple channels
Oftentimes we want to read from multiple channels, for that we can use the select statement inside of a for loop.
package main
import "fmt"
func main() {
cInt := make(chan int)
cStr := make(chan string)
go sendInt(cInt)
go sendStr(cStr)
for {
select {
case i := <-cInt:
fmt.Print(i, " ")
case m := <-cStr:
fmt.Print(m, " ")
}
}
}
func sendInt(c chan<- int) {
for x := 0; x < 5; x++ {
c <- x
}
}
func sendStr(c chan<- string) {
msg := []string{"Hello", "World", "!"}
for _, m := range msg {
c <- m
}
}
Outputs:
Hello World 0 ! 1 2 3 4
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [select]:
main.main()
//threadfun/main.go:12 +0x168
exit status 2
Notice that we enter deadlock after. If we were to try and close the channels similar to what we did before, we get the following output (and the program never exits):
Hello World ! 0 1 2 3 4 0 0 0 0 0 0 ……. (continuously and never ends)
Thus we need something a little different. Luckily we can use go’s built in functionality that let’s us know when a channel is closed. We can change the above to the following:
package main
import "fmt"
func main() {
cInt := make(chan int)
cStr := make(chan string)
go sendInt(cInt)
go sendStr(cStr)
c1, c2 := true, true
var i int
var m string
for c1 || c2 {
select {
case i, c1 = <-cInt:
if c1 {
fmt.Print(i, " ")
}
case m, c2 = <-cStr:
if c2 {
fmt.Print(m, " ")
}
}
}
}
func sendInt(c chan<- int) {
for x := 0; x < 5; x++ {
c <- x
}
close(c)
}
func sendStr(c chan<- string) {
msg := []string{"Hello", "World", "!"}
for _, m := range msg {
c <- m
}
close(c)
}
Outputs:
Hello 0 World ! 1 2 3 4
Waiting on Done
Another common way to stop reading on a goroutine is to wait on an explicit channel that will exit when called.
package main
import "fmt"
func main() {
c := make(chan int)
done := make(chan bool)
go readOnChannel(c, done)
for x := 0; x < 5; x++ {
c <- x
}
done <- true
}
func readOnChannel(c <-chan int, done <-chan bool) {
for {
select {
case i := <-c:
fmt.Print(i, " ")
case <-done:
return
}
}
}
Context with cancel
A common practice for waiting on a done channel is to use a context with a cancel set on it. Contexts are useful because they allow the program to store information within it, so using the context to let a program know when to finish is a useful added abstraction.
import (
"context"
"fmt"
)
func main() {
c := make(chan int)
ctx, cancel := context.WithCancel(context.Background())
go readOnChannel(ctx, c)
for x := 0; x < 5; x++ {
c <- x
}
cancel()
}
func readOnChannel(ctx context.Context, c <-chan int) {
for {
select {
case i := <-c:
fmt.Print(i, " ")
case <-ctx.Done():
return
}
}
}
Ticker
We can also use a select statement to run something every X amount of time using a ticker.
func main() {
t := time.NewTicker(time.Second * 1)
for {
select {
case <-t.C:
fmt.Println("Tock")
}
}
}
Outputs:
Tock
Tock
Tock
...etc...
WaitGroups
WaitGroups make it easier to wait for different goroutines to finish.
Imagine you work in a grocery store and are in charge of overseeing three employees taking inventory. You give each one a clipboard to take inventory, and when they’re done they give the clipboard back. Only when you’ve received the very last clipboard are you allowed to say that inventory is done. This is essentially what a WaitGroup is in Go.
Typically a WaitGroup is incremented at the beginning a go routine, and once that goroutine is finished, it calls Done. We can wait on that WaitGroup until every goroutine has called Done
package main
import (
"fmt"
"sync"
"time"
)
func worker(id int, wg *sync.WaitGroup) {
defer wg.Done()
fmt.Printf("Worker %d starting\n", id)
time.Sleep(time.Second)
fmt.Printf("Worker %d done\n", id)
}
func main() {
var wg sync.WaitGroup
for i := 1; i <= 5; i++ {
wg.Add(1)
go worker(i, &wg)
}
wg.Wait()
}
Outputs:
Worker 2 starting
Worker 4 starting
Worker 5 starting
Worker 1 starting
Worker 3 starting
Worker 1 done
Worker 3 done
Worker 4 done
Worker 5 done
Worker 2 done