I will be talking about benchmarking in Go in this article.
But first, what is benchmarking in general?
Simply put, it means running a set of programs to assess the relative performance against a set of standards.
In our case, we want to judge whether Go codes that we wrote are performing well or not.
How to do benchmarking in Go
Go included a default benchmarking tool inside testing package.
To differentiate benchmarking from testing, you need to prefix the function with BenchmarkXxx.
func BenchmarkXxx(*testing.B)
If you happen to have other testing functions and want to run only all benchmarks, we can do so by:
go test -bench .
Or if you want to run individual benchmark:
go test -bench FUNCTION_NAME
go test -bench BenchmarkUseConcatOperator
go test -bench Use
-bench argument uses regex pattern so if you define -bench Use it will run all benchmarks that contain the string "Use" on their names.
Benchmark tests setup
Let's set up some simple benchmark tests. I will explain the thought process but if you just want to see the source code you can check my GitHub Gist here.
First, we want to decide on a theme for testing. String concatenation sounds like an interesting topic since in Go we could do it in various ways.
We'll create a main.go file and setup three functions for concatenating strings:
- First is using the simple concat operator "+"
- Second is utilizing
bytes.Bufferpackage - Third is utilizing
strings.Builderpackage
package main
import (
"bytes"
"math/rand"
"strings"
)
func main() {
// This main function does nothing since we only want to test benchmarking
}
func UseConcatOperator(slice []string) string {
var s string
for _, val := range slice {
s = s + val
}
return s
}
func UseBytesBuffer(slice []string) string {
var b bytes.Buffer
for _, val := range slice {
b.WriteString(val)
}
return b.String()
}
func UseStringsBuilder(slice []string) string {
var sb strings.Builder
for _, val := range slice {
sb.WriteString(val)
}
return sb.String()
}
Now that we have three functions to concatenate strings, we want to see which one actually performs the best when concatenating a slice of strings.
Obviously, we need to create that slice of strings for testing, adding another function below the three.
func GetSliceOfStrings() []string {
size := 1000
slice := make([]string, size)
for i := 0; i < size; i++ {
s := "abcdedfghijklmnopqrstuvwxyz"
slice[i] = s
}
return slice
}
There you have it. We got our functions to test. Now on to the test file, we shall name it main_test.go.
package main
import (
"testing"
)
var slice = GetSliceOfStrings()
func BenchmarkUseConcatOperator(b *testing.B) {
for i := 0; i < b.N; i++ {
UseConcatOperator(slice)
}
}
func BenchmarkUseBytesBuffer(b *testing.B) {
for i := 0; i < b.N; i++ {
UseBytesBuffer(slice)
}
}
func BenchmarkUseStringsBuilder(b *testing.B) {
for i := 0; i < b.N; i++ {
UseStringsBuilder(slice)
}
}
It's clear at a glance that we have a benchmarking function for each of our string concatenation functions. And we provide the parameter for each testing with a slice of 1000 strings.
We can start benchmarking with a minimal command:
go test -bench .
Which will yield these results:
C:\Users\user\playground> go test -bench .
goos: windows
goarch: amd64
pkg: playground
BenchmarkUseConcatOperator-8 276 4009217 ns/op
BenchmarkUseBytesBuffer-8 25038 45427 ns/op
BenchmarkUseStringsBuilder-8 26460 50240 ns/op
PASS
ok playground 5.024s
It's a little bit hard to read but each of our benchmark functions ran the actual function they are assigned to. For instance, BenchmarkUseConcatOperator ran UseConcatOperator 276 times at a speed of 4,009,217ns per operation.
Since each benchmark is run for a minimum of 1 second by default, compared to the other concatenate functions, UseConcatOperator is the slowest.
If we want to modify the parameters a bit we can do so.
Perhaps we want to set each benchmarking test to do x repetition...
go test -bench . -benchtime 10000x
Result:
C:\Users\user\playground> go test -bench . -benchtime 10000x
goos: windows
goarch: amd64
pkg: playground
BenchmarkUseConcatOperator-8 10000 4301341 ns/op
BenchmarkUseBytesBuffer-8 10000 45964 ns/op
BenchmarkUseStringsBuilder-8 10000 42709 ns/op
PASS
ok playground 43.958s
Or set each benchmarking test to last x seconds instead of default 1s...
go test -bench . -benchtime 5s
Result:
C:\Users\user\playground> go test -bench . -benchtime 10000x
goos: windows
goarch: amd64
pkg: playground
BenchmarkUseConcatOperator-8 1365 4324163 ns/op
BenchmarkUseBytesBuffer-8 125238 48508 ns/op
BenchmarkUseStringsBuilder-8 134504 44652 ns/op
PASS
ok playground 19.419s
From these benchmarks, we can safely conclude that using strings.Builder gives the best performance in string concatenation.
More test flags
What can we do to get more information from our benchmarks? We can use flags from Go's official documentation.
-
-benchmem
Gives out details on memory allocation.
BenchmarkUseConcatOperator-8 256 4292437 ns/op 14227566 B/op 1000 allocs/op
type BenchmarkResult struct {
N int // The number of iterations.
T time.Duration // The total time taken.
Bytes int64 // Bytes processed in one iteration.
MemAllocs uint64 // The total number of memory allocations; added in Go 1.1
MemBytes uint64 // The total number of bytes allocated; added in Go 1.1
}
-cpuprofile FILE_NAME (ex: cpu.prof)
Write a CPU profile to the specified file (cpu.prof) before exiting.
Use go tool pprof to read the file.
C:\Users\user\playground> go tool pprof cpu.prof
Type: cpu
Time: Mar 26, 2021 at 2:46pm (JST)
Duration: 5.42s, Total samples = 10.15s (187.21%)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top10
-memprofile FILE_NAME (ex: mem.prof)
Write a memory allocation profile to the specified file (mem.prof) before exiting.
Use go tool pprof to read the file.
-count
Set up the number of repetitions for each benchmarking test.
C:\Users\user\playground> go test -bench . -count 5
goos: windows
goarch: amd64
pkg: playground
BenchmarkUseConcatOperator-8 289 3667618 ns/op
BenchmarkUseConcatOperator-8 291 3874053 ns/op
BenchmarkUseConcatOperator-8 307 3939308 ns/op
BenchmarkUseConcatOperator-8 328 3883875 ns/op
BenchmarkUseConcatOperator-8 327 3869996 ns/op
-
-cpu
Define the number of CPUs (a list of GOMAXPROCS) used during benchmarking.
Closing remarks
Benchmarking is one of the important points when testing your program. Knowing how well your program performs helps a lot in fine-tuning small details.
However, benchmarking should not be the very focus of work in the early stage of development. It's better to make sure business requirements are met first, before mulling over program optimization.