Unit testing using mocks in Go
Why mocks are needed?
I generally prefer testing my code without using mocks. For example, if the code is integrated with Elastic Search, I prefer to run a elastic search container locally and test the Go code against that container. But there are some cases where creating a local environment for testing is not possible. This usually happens when integrating with cloud providers. Let’s take the case where the code is creating a AWS s3 bucket. To create a S3 bucket, the user needs access to create S3 buckets in AWS. The company you are working for might enforce organization policies which allows the creation of S3 buckets only from a whitelisted set of EC2 VMs. This prevents the ability to test code which creates S3 buckets locally. There is also a need to test error cases such as handling of bucket creation failures. Simulating failure cases is difficult without the use of mocks. In the current software development era with majority of the development happening in cloud, learning how to unit test using mocks become essential.
Unit testing a function to create a S3 bucket
Let’s first write a function to create a S3 bucket. We will then learn how to test this function using mocks.
1package aws
2
3import (
4 "context"
5 "errors"
6 "log"
7 "time"
8
9 "github.com/aws/aws-sdk-go-v2/aws"
10 "github.com/aws/aws-sdk-go-v2/service/s3"
11 "github.com/aws/aws-sdk-go-v2/service/s3/types"
12)
13
14func createS3Bucket(s3Client s3.Client, name string, region string) error {
15 if _, err := s3Client.CreateBucket(context.TODO(), &s3.CreateBucketInput{
16 Bucket: aws.String(name),
17 CreateBucketConfiguration: &types.CreateBucketConfiguration{
18 LocationConstraint: types.BucketLocationConstraint(region),
19 },
20 }); err != nil {
21 return err
22 }
23 if err := s3.NewBucketExistsWaiter(&s3Client).Wait(
24 context.TODO(), &s3.HeadBucketInput{Bucket: aws.String(name)}, time.Minute); err != nil {
25 log.Printf("Failed attempt to wait for bucket %s to exist.\n", name)
26 return err
27 }
28 return nil
29}
The above function takes a s3.Client struct as a parameter and creates a bucket. It returns error if the bucket creation fails. Most of the AWS API calls are asynchronous and hence there is code added in line no. 23 to check whether the bucket exists. The code waits for the bucket to be created and returns any error that occurs when waiting to check whether the bucket exists. Save the above code to a file name s3client.go and run go mod init github.com/golangbot/s3mock followed by go mod tidy to initialize the Go module.
Creating the interface needed to unit test
The possible test cases for the createS3Bucket function include
- Testing whether the bucket creation is successful
- Testing whether the function returns error if any when creating buckets
- Testing whether the function successfully waits for the bucket to be finished creating
- Testing whether the function returns errors if any when waiting for the bucket to be created
Testing the above cases using the real s3 Client is near impossible. It’s difficult to simulate error cases with the real s3 client. This is where mocking becomes essential.
In Go, it is not possible to mock using structs. With this I mean, it’s not possible to provide a mock version of s3.Client struct which can simulate error conditions as a parameter to the createS3Bucket function.
The way to mock functionality in Go is using interfaces. The first step is to refactor the createS3Bucket function to accept an interface instead of the s3.Client struct. The interface will have all the methods needed for bucket creation. Once this interface is created, it’s possible to create a mock type that implements this interface and unit test the function.
To convert the s3.Client parameter to interface, we first need to find the methods from the s3.Client struct that are used in the createS3Bucket function. In line no. 15 of the above code snippet, the createS3Bucket method of s3.Client is used to create the bucket. So this is one of the methods that needs to be present in the interface. In line no. 23, the s3.Client is passed as a parameter to NewBucketExistsWaiter function. The NewBucketExistsWaiter function needs a s3.HeadBucketAPIClient parameter which by itself is an interface with the following definition,
1type HeadBucketAPIClient interface {
2 HeadBucket(context.Context, *HeadBucketInput, ...func(*Options)) (*HeadBucketOutput, error)
3}
The above analysis concludes that, we need to create an interface with two methods namely CreateBucket and NewBucketExistsWaiter. This interface can then be used as a parameter in the createS3Bucket function. Let’s define the interface first.
1type s3Client interface {
2 CreateBucket(ctx context.Context, params *s3.CreateBucketInput, optFns ...func(*s3.Options)) (*s3.CreateBucketOutput, error)
3 HeadBucket(context.Context, *s3.HeadBucketInput, ...func(*s3.Options)) (*s3.HeadBucketOutput, error)
4}
The next step is to refactor the createS3Bucket function to use this interface as parameter.
1func createS3Bucket(s3Client s3Client, name string, region string) error {
2 if _, err := s3Client.CreateBucket(context.TODO(), &s3.CreateBucketInput{
3 Bucket: aws.String(name),
4 CreateBucketConfiguration: &types.CreateBucketConfiguration{
5 LocationConstraint: types.BucketLocationConstraint(region),
6 },
7 }); err != nil {
8 return err
9 }
10
11 if err := s3.NewBucketExistsWaiter(s3Client).Wait(
12 context.TODO(), &s3.HeadBucketInput{Bucket: aws.String(name)}, time.Minute); err != nil {
13 log.Printf("Failed attempt to wait for bucket %s to exist.\n", name)
14 return err
15 }
16 return nil
17}
The createS3Bucket defined above now uses the s3Client interface as parameter instead of the s3.Client struct. The interface must also be passed to the NewBucketExistsWaiter in line no. 11 of the above function.
Now this function can be unit tested by creating a mock type which implements the s3Client interface. The complete code is provided below.
1package aws
2
3import (
4 "context"
5 "log"
6 "time"
7
8 "github.com/aws/aws-sdk-go-v2/aws"
9 "github.com/aws/aws-sdk-go-v2/service/s3"
10 "github.com/aws/aws-sdk-go-v2/service/s3/types"
11)
12
13type s3Client interface {
14 CreateBucket(ctx context.Context, params *s3.CreateBucketInput, optFns ...func(*s3.Options)) (*s3.CreateBucketOutput, error)
15 HeadBucket(context.Context, *s3.HeadBucketInput, ...func(*s3.Options)) (*s3.HeadBucketOutput, error)
16}
17
18func createS3Bucket(s3Client s3Client, name string, region string) error {
19 if _, err := s3Client.CreateBucket(context.TODO(), &s3.CreateBucketInput{
20 Bucket: aws.String(name),
21 CreateBucketConfiguration: &types.CreateBucketConfiguration{
22 LocationConstraint: types.BucketLocationConstraint(region),
23 },
24 }); err != nil {
25 return err
26 }
27
28 if err := s3.NewBucketExistsWaiter(s3Client).Wait(
29 context.TODO(), &s3.HeadBucketInput{Bucket: aws.String(name)}, time.Minute); err != nil {
30 log.Printf("Failed attempt to wait for bucket %s to exist.\n", name)
31 return err
32 }
33 return nil
34}
Mocking the s3Client interface and testing successful s3 bucket creation
Before implementing full fledged table driven unit tests for the createS3Bucket function, let’s write a single unit test which mocks the interface and understand how interfaces helps with unit testing.
The first step in mocking the s3Client interface is creating a type that implements the interface. We can create a mockS3Client struct type that implements the interface.
1type mockS3Client struct {
2}
3
4func (m mockS3Client) CreateBucket(ctx context.Context, params *s3.CreateBucketInput, optFns ...func(*s3.Options)) (*s3.CreateBucketOutput, error) {
5 return &s3.CreateBucketOutput{}, nil
6}
7
8func (m mockS3Client) HeadBucket(ctx context.Context, params *s3.HeadBucketInput, optFns ...func(*s3.Options)) (*s3.HeadBucketOutput, error) {
9 return &s3.HeadBucketOutput{}, nil
10}
The above code snippet creates a mockS3Client struct type which has the CreateBucket and HeadBucket methods and hence it implements the s3Client interface. These methods in the mockS3Client don’t return any errors and hence they simulate the successful creation of the s3 bucket. Let’s use the mockS3Client to test the successful creation of the s3 bucket.
1package aws
2
3import (
4 "context"
5 "testing"
6
7 "github.com/aws/aws-sdk-go-v2/service/s3"
8)
9
10type mockS3Client struct {
11}
12
13func (m mockS3Client) CreateBucket(ctx context.Context, params *s3.CreateBucketInput, optFns ...func(*s3.Options)) (*s3.CreateBucketOutput, error) {
14 return &s3.CreateBucketOutput{}, nil
15}
16
17func (m mockS3Client) HeadBucket(ctx context.Context, params *s3.HeadBucketInput, optFns ...func(*s3.Options)) (*s3.HeadBucketOutput, error) {
18 return &s3.HeadBucketOutput{}, nil
19}
20
21func Test_createS3BucketSuccess(t *testing.T) {
22 mockS3Client := mockS3Client{}
23 wantErr := false
24 if err := createS3Bucket(mockS3Client, "test", "us-west-2"); (err != nil) != wantErr {
25 t.Errorf("createBucket() error = %v, wantErr %v", err, wantErr)
26 }
27}
Save the above code in a file named s3client_test.go. In line no. 22 of the above code in the Test_createS3BucketSuccess function, the mockS3Client is created and it is then passed to the createS3Bucket function in line no. 24. Since the mockS3Client implements the s3Client interface, it is possible to use the mockS3Client as a parameter to the createS3Bucket function. Since the CreateBucket and HeadBucket methods don’t return any error, they provide mock implementation for a successful s3 bucket creation. Run the Test_createBucketSuccess using the command go test ./... -v. The test case will pass and it will print the following output.
1➜ go test ./... -v
2=== RUN Test_createS3BucketSuccess
3--- PASS: Test_createS3BucketSuccess (0.00s)
4PASS
5ok github.com/golangbot/s3mock 0.006s
Great! We have just written our first test case by using mocks in Go. The next step is to mock the case where the s3 bucket creation fails and test whether createS3Bucket returns an error.
Mocking error cases using the interface
To test whether createS3Bucket returns an error when the bucket creation fails, we need to make sure that the createBucket method in the s3Client interface returns an error. To do that we will add a parameter to the mockS3Client struct which determines whether to return an error or not from the CreateBucket method.
The mockS3Client struct above has been changed and a createBucketError field has been added. This field will be set to an error when CreateBucket method must return an error. It will be clear what I mean once the test case is refactored.
1package aws
2
3import (
4 "context"
5 "errors"
6 "testing"
7
8 "github.com/aws/aws-sdk-go-v2/service/s3"
9)
10
11type mockS3Client struct {
12 createBucketError error
13}
14
15func (m mockS3Client) CreateBucket(ctx context.Context, params *s3.CreateBucketInput, optFns ...func(*s3.Options)) (*s3.CreateBucketOutput, error) {
16 return &s3.CreateBucketOutput{}, m.createBucketError
17}
18
19func (m mockS3Client) HeadBucket(ctx context.Context, params *s3.HeadBucketInput, optFns ...func(*s3.Options)) (*s3.HeadBucketOutput, error) {
20 return &s3.HeadBucketOutput{}, nil
21}
22
23func Test_createS3BucketSuccess(t *testing.T) {
24 mockS3Client := mockS3Client{}
25 wantErr := false
26 if err := createS3Bucket(mockS3Client, "test", "us-west-2"); (err != nil) != wantErr {
27 t.Errorf("createBucket() error = %v, wantErr %v", err, wantErr)
28 }
29}
30
31func Test_createBucketFailure(t *testing.T) {
32 mockS3Client := &mockS3Client{
33 createBucketError: errors.New("mock error"),
34 }
35 wantErr := true
36 if err := createS3Bucket(mockS3Client, "test", "us-west-2"); (err != nil) != wantErr {
37 t.Errorf("createBucket() error = %v, wantErr %v", err, wantErr)
38 }
39}
The CreateBucket function in line no. 15 of the above code now uses returns m.createBucketError. If this field contains an error, then CreateBucket will return an error. This field will be nil by default and hence no changes were needed in the Test_createS3BucketSuccess test case.
The Test_createBucketFailure test case tests whether createS3Bucket returns an error when the bucket creation fails. So createBucketError is set to an error when creating the mockS3Client in line no. 33. Because of this CreateBucket returns an error in the Test_createBucketFailure test case and hence the test case passes ensuring that createS3Bucket returns an error when bucket creation fails.
Thus we have used the mock interface to test both the successful and failures in s3 bucket creation.
Refactor the code and use table driven tests
As we can see from the tests above, most of the code is repeated in the two test cases. So it’s possible to refactor the code using table driven tests. This will make adding new tests cases easier too. The refactored test cases using table driven tests are provided below.
1package aws
2
3import (
4 "context"
5 "errors"
6 "testing"
7
8 "github.com/aws/aws-sdk-go-v2/service/s3"
9)
10
11type mockS3Client struct {
12 createBucketError error
13}
14
15func (m mockS3Client) CreateBucket(ctx context.Context, params *s3.CreateBucketInput, optFns ...func(*s3.Options)) (*s3.CreateBucketOutput, error) {
16 return &s3.CreateBucketOutput{}, m.createBucketError
17}
18
19func (m mockS3Client) HeadBucket(ctx context.Context, params *s3.HeadBucketInput, optFns ...func(*s3.Options)) (*s3.HeadBucketOutput, error) {
20 return &s3.HeadBucketOutput{}, nil
21}
22
23func Test_createS3Bucket(t *testing.T) {
24 type args struct {
25 s3Client s3Client
26 name string
27 region string
28 }
29 tests := []struct {
30 name string
31 args args
32 wantErr bool
33 }{
34 {
35 name: "successful bucket creation",
36 args: args{
37 s3Client: &mockS3Client{
38 createBucketError: nil,
39 },
40 name: "test",
41 region: "us-west-2",
42 },
43 wantErr: false,
44 },
45 {
46 name: "failed bucket creation",
47 args: args{
48 s3Client: &mockS3Client{
49 createBucketError: errors.New("mock error"),
50 },
51 name: "test",
52 region: "us-west-2",
53 },
54 wantErr: true,
55 },
56 }
57 for _, tt := range tests {
58 t.Run(tt.name, func(t *testing.T) {
59 if err := createS3Bucket(tt.args.s3Client, tt.args.name, tt.args.region); (err != nil) != tt.wantErr {
60 t.Errorf("createS3Bucket() error = %v, wantErr %v", err, tt.wantErr)
61 }
62 })
63 }
64}
In this tutorial we only tested whether the bucket creation succeeds or fails. It is possible to add more checks such as validating the bucket name, region and so on using the mocks. I will leave it as an exercise to add these checks.
It is also possible to generate the mock types using pre-existing libraries such as https://github.com/vektra/mockery. They also provide prebuilt functionality to return different values from the mocked interface methods. For example, we used createBucketError to determine whether to return an error or not from the CreateBucket method. This kind of functionality is already built in https://github.com/vektra/mockery. I have used this library in several of my projects and I highly recommend it.
Thanks for reading. I hope you liked this tutorial. Please leave your feedback and comments. Please consider sharing this tutorial on twitter and LinkedIn. Have a good day.
If you’re searching for job opportunities in Go programming, explore gojobs.run, a specialized search engine I developed that focuses on indexing and curating Go related job listings.
