Mock programs to test os/exec in Go

This post explores a pattern you can use in Go to write unit tests for code that invokes external programs with the os/exec package. Using this pattern, you can mock the external program’s behavior and test that your code,

calls the external program with the correct arguments, working directory, and environment variables
behaves as expected when the external program returns a specific output or fails

📢 Important

Tests that use this pattern will not invoke the external program. The validity of these tests is limited by the validity of the assumptions you make in your mocked behaviors.

1. Problem

Let’s write a function that reports the hash of the current Git commit.

We can do this in git with the rev-parse command.

$ git rev-parse HEAD
2b238cbdbbfdcae5e5d029ef99c74f7ff6287c1f

exec.Command makes it easy to turn this into a Go function.

// HeadCommit reports the commit hash of the HEAD commit.
func HeadCommit(dir string) (string, error) {
  cmd := exec.Command("git", "rev-parse", "HEAD")
  cmd.Dir = dir
  out, err := cmd.Output()
  if err != nil {
    return "", err
  }
  return string(bytes.TrimSpace(out)), nil
}

How can we test this?

The problem arises once we start trying to test this function.

Let’s try writing an integration test that sets up a temporary Git repository, commits to it, runs HeadCommit inside it, and verifies the result.

Initialize an empty repository

func TestHeadCommitEndToEnd(t *testing.T) {
  repoRoot := t.TempDir()

  cmd := exec.Command("git", "init")
  cmd.Dir = repoRoot
  require.NoError(t, cmd.Run(), "git init failed")

Commit to it

  cmd = exec.Command("git", "commit",
    "--allow-empty", "-m", "initial commit")
  cmd.Dir = repoRoot
  require.NoError(t, cmd.Run(), "git commit failed")

Run HeadCommit inside it

  result, err := HeadCommit(repoRoot)
  require.NoError(t, err, "HeadCommit failed")

💡 Tip

Use the testify library to keep test assertions concise.

And now we verify… what?

This is where our integration test runs into trouble.

Non-deterministic

We cannot hard-code the expected commit hash because it’s not deterministic. The commit hash changes based on the commit message, contents of the tree, time of the day, author of the commit, phase of the moon, hash of the parent commit (if any), and more.

require.Equal(t,
  "ea09bff03b3894b79e335f77a0f71d75adf263e9", result) // FAIL

Even if we could verify the hash, or tested a behavior that was more deterministic, we run into other problems.

Non-hermetic

The git command is not hermetic. It depends on the system configuration and can break on different machines.

For example, the initial git commit above will fail if the current user’s Git configuration does not set the user.name or user.email keys — which is often the case on test machines.

$ git commit --allow-empty -m 'initial commit'
Author identity unknown

*** Please tell me who you are.

Run

  git config --global user.email "[email protected]"
  git config --global user.name "Your Name"

to set your account's default identity.

We can can work around this by setting up a fake $HOME with a fake Git configuration for our test, but it’s getting pretty messy now. Some systems cannot be easily isolated for testing.

Slow

Each such workaround adds to the work the tests have to do before the actual test logic, and all that extra work slows down the test suite.

Inflexible

Finally, the strategy is inflexible and limited in the test scenarios it can produce. The test does not control Git’s behavior so it cannot easily make Git fail in a specific way or return a specific invalid output.

A better approach

What if we could specify mock behaviors for external programs like Git — similar to how we mock interfaces?

Our tests will become more hermetic, deterministic, and performant. The added flexibility will allow us to manufacture more test scenarios, making the code we’re testing more robust.

One way to achieve this is to have HeadCommit invoke a program controlled by the test.

2. A test is just a program

To get to the solution, we first need a high-level understanding of how the go test command works.

When you run go test, the command generates, compiles, and runs a full-fledged Go program containing the code under test, and all the Test* functions.

In fact, you can ask go test to only build the program and not run it with the -c flag. It will place the compiled program in the current directory with the name ${name}.test where ${name} is the name of the current directory.

$ go help test
# ...
  -c
      Compile the test binary to pkg.test but do not run it
      (where pkg is the last element of the package's import path).
      The file name can be changed with the -o flag.
# ...

Try it out:

$ go test -c
$ ./mypackage.test --help
# ...
$ ./mypackage.test
--- FAIL: TestHeadCommitEndToEnd (0.02s)
# ...

2.1. Hijack it with `TestMain`

go test provides TestMain as a hook for controlling the test binary. This is typically used to run setup or cleanup code before or after the tests.

As an example, write a function named TestMain accepting a testing.M as an argument, and go test will run that function in the generated program. The function should call M.Run to have Go run the tests.

func TestMain(m *testing.M) {
  fmt.Println("inside TestMain")
  os.Exit(m.Run())
}

// $ go test
// inside TestMain
// PASS

🚨 Warning

If you write a TestMain function, and you don’t call m.Run(), your tests will not run.

func TestMain(m *testing.M) {
  fmt.Println("success!")
}

func TestBad(t *testing.T) {
  assert.Equal(t, 1, 2, "universe is broken")
}

// $ go test
// success!

The fact that TestMain can have arbitrary logic is the key to solving our problem.

We can make our code invoke the test binary instead of Git, and have that binary run our mocked behaviors.

3. Mocking external programs

We know that go test runs our test inside just another Go binary, and that we can control its behavior with TestMain. Those are the pieces we need to mock the behavior of an external program.

Roughly, we want this control flow:

go test runs the test
the test calls HeadCommit via TestHeadCommit
HeadCommit uses os/exec to run the test binary
the test binary pretends to be git rev-parse

3.1. Inject the binary path

First, we’ll refactor HeadCommit so that we can specify the path to the Git executable. This will let the test easily specify the path to the test binary.

📝 Note

If you cannot modify the code that uses os/exec, or you cannot change the path to the binary for other reasons, you can still use this method. Read the rest of the article to understand the technique, and then skip on to Dealing with fixed binary paths.

Introduce a Git struct that lets us specify the path of the git executable. This struct can expand to become the entry point for all Git commands in the future.

// Git provides access to git commands.
type Git struct {
  // Exe is the path to the git executable.
  Exe string
}

Next, turn HeadCommit into a method on this struct, and use the new Exe field instead of hard-coding "git".

 // HeadCommit reports the commit hash of the HEAD commit.
-func HeadCommit(dir string) (string, error) {
- cmd := exec.Command("git", "rev-parse", "HEAD")
+func (g *Git) HeadCommit(dir string) (string, error) {
+ cmd := exec.Command(g.Path, "rev-parse", "HEAD")
  cmd.Dir = dir
  out, err := cmd.Output()
  if err != nil {

💡 Tip

Exe is currently a required field. You can make it optional for ease of use by giving it a default value of "git" if it’s empty.

exe := g.Exe
if path == "" {
  exe = "git"
}
cmd := exec.Command(exe, ...)

This will search the system $PATH for the git executable if the field is unset.

Now we can test HeadCommit. Going back to the Control Flow Diagram, we need the following:

a function that specifies the mocked behavior
a TestMain that invokes the mocked behavior or the tests
a test that drives the whole machinery

3.2. Implement mock behavior

In your test file, add a function that implements the mock behavior. This function behaves like an independent main(). It can,

access arguments with os.Args
exit the program with os.Exit or log.Fatal
access environment variables with os.Getenv
etc.

For example, add a headCommitSuccess function that pretends to be git rev-parse: it validates its arguments and prints a fixed commit hash.

func headCommitSuccess() {
  want := []string{"rev-parse", "HEAD"}               (1)
  if args := os.Args[1:]; !slices.Equal(want, args) { (2)
    log.Fatalf(`expected arguments %q, got %q`, want, args)
  }
  fmt.Println("ea09bff03b3894b79e335f77a0f71d75adf263e9") (3)
}

Expected command line arguments.
Compare to actual command line arguments or exit with a non-zero exit code.
Print a fixed string for the test to verify if the arguments matched.

3.3. Hijack the test binary

TestMain needs to determine whether it should simulate git rev-parse by calling headCommitSuccess, or run tests as usual.

We introduce a TEST_GIT_BEHAVIOR environment variable that we’ll set from our test to accomplish that.

func TestMain(m *testing.M) {
  behavior := os.Getenv("TEST_GIT_BEHAVIOR")
  switch behavior {
  case "":
    os.Exit(m.Run()) (1)
  case "headCommitSuccess":
    headCommitSuccess() (2)
  default:
    log.Fatalf("unknown behavior %q", behavior) (3)
  }
}

If TEST_GIT_BEHAVIOR is unset, this is a go test command. Run the tests as usual with M.Run.
If it’s set to headCommitSuccess — name of the fake behavior — then run that behavior. We can add more fake behaviors in the future.
Fail for unrecognized behaviors. This will prevent debugging sessions caused by typos.

💡 Tip
Use %q for strings in error messages.

3.4. Write the test

Finally, write a test for HeadCommit that passes the test binary as the git executable. Inside the test, set TEST_GIT_BEHAVIOR to the name of the previously defined behavior for TestMain.

func TestHeadCommit(t *testing.T) {
  testExe, err := os.Executable() (1)
  require.NoError(t, err, "can't determine current exectuable")

  git := Git{Exe: testExe}
  t.Setenv("TEST_GIT_BEHAVIOR", "headCommitSuccess") (2)

  got, err := git.HeadCommit(t.TempDir()) (3)
  require.NoError(t, err, "HeadCommit failed")
  assert.Equal(t,
    "ea09bff03b3894b79e335f77a0f71d75adf263e9", got)
}

os.Executable reports the path of the current binary. Use it to get the path of the test binary.
Use T.Setenv to set TEST_GIT_BEHAVIOR to headCommitSuccess. go test will clear this environment variable when the test finishes, ensuring that other tests are not affected by this behavior.
Call Git.HeadCommit inside a temporary directory and verify its output.

💡 Tip
Use T.TempDir to create temporary directories that get cleaned up automatically when the test finishes.

If you now run the test, it will pass as expected.

$ go test -v
=== RUN   TestHeadCommit
--- PASS: TestHeadCommit (0.01s)
PASS
# ...

A quick summary of what we’ve done:

TestHeadCommit sets TEST_GIT_BEHAVIOR to headCommitSuccess, uses the path of the test binary as git, and runs HeadCommit
HeadCommit runs the test binary it thinks is git, which calls TestMain
TestMain inspects TEST_GIT_BEHAVIOR, runs the headCommitSuccess function, and exits
HeadCommit reads the output of headCommitSuccess and treats it as the output of git rev-parse
TestHeadCommit verifies the return value

Although this machinery takes a minute to understand, it’s fairly extensible and robust. You can easily add new fake behaviors to the switch statement in TestMain to test other functions that use os/exec, and to simulate failure scenarios that would otherwise be difficult to replicate.

4. Caveats

There are a few limitations to this pattern:

It works and is deterministic, but it’s still a bit of a hack job. Try not to litter your codebase with it. Organize your code so that you use this pattern in very few places, and preferably far from the business logic.
If your code, or any package you import has a non-trivial init() function, that will run twice. (For this and other reasons, you should Avoid init() when possible.)
If the mocked external program calls back into your own code, that will not contribute to the unit test coverage of your test.
Tests that use T.Setenv cannot run in parallel with other tests. If refactoring is an option, you can work around this; see Re-enabling parallel tests.

Appendix A: Dealing with fixed binary paths

Sometimes, you cannot refactor the code to inject the binary path. For example,

you may not own the code that uses os/exec
refactoring is a breaking API change
the executable you want to mock is not run directly by you, but by another program

You can still use this method in those cases with a small change: place a copy of the test binary on the $PATH with a name matching the program you’re mocking.

As a demonstration, let’s go back to the version of HeadCommit before we refactored it, and pretend we’re not allowed to change it.

// HeadCommit reports the commit hash of the HEAD commit.
func HeadCommit(dir string) (string, error) {
  cmd := exec.Command("git", "rev-parse", "HEAD")
  cmd.Dir = dir
  out, err := cmd.Output()
  if err != nil {
    return "", err
  }
  return string(bytes.TrimSpace(out)), nil
}

To test this, we still need to implement the mock behavior and hijack the test binary like before. However, TestHeadCommit looks a bit different:

func TestHeadCommit(t *testing.T) {
  testExe, err := os.Executable()
  require.NoError(t, err, "can't determine current exectuable")

  binDir := t.TempDir() (1)
  newPath := binDir + string(filepath.ListSeparator) + os.Getenv("PATH")
  t.Setenv("PATH", newPath)

  gitExe := filepath.Join(binDir, "git")
  copyExecutable(t, testExe, gitExe) (2)

  t.Setenv("TEST_GIT_BEHAVIOR", "headCommitSuccess") (3)

  got, err := HeadCommit(t.TempDir())
  require.NoError(t, err, "HeadCommit failed")
  assert.Equal(t,
    "ea09bff03b3894b79e335f77a0f71d75adf263e9", got)
}

Create a temporary directory and prepend it to the $PATH environment variable.

📝 Note
filepath.ListSeparator is the OS-specific path list separator — ":" on Linux. This helps keep our test platform-agnostic.

Copy the test executable into this directory with the name "git". The copyExecutable function is implemented like so:

// copyExecutable copies the executable at src to dst.
func copyExecutable(t *testing.T, src, dst string) {
  i, err := os.Open(src)
  require.NoError(t, err, "cannot open %q", src)
  defer i.Close()

  o, err := os.Create(dst)
  require.NoError(t, err, "cannot create %q", dst)
  defer o.Close()

  _, err = io.Copy(o, i)
  require.NoError(t, err, "cannot copy binary")

  require.NoError(t, os.Chmod(dst, 0o755), "cannot update permissions")
}

Set TEST_GIT_BEHAVIOR and test HeadCommit like before.

This test now passes.

$ go test -v
=== RUN   TestHeadCommit
--- PASS: TestHeadCommit (0.01s)
PASS
# ...

Appendix B: Re-enabling parallel tests

We used T.Setenv to manipulate environment variables temporarily for tests.

Unfortunately, changing environment variables from tests means that those tests cannot run in parallel with other tests, because if they did, they would clobber each other’s settings.

Go helps us avoid this mistake by disallowing use of T.Setenv inside tests that are marked as parallel with T.Parallel.

Try it out:

Mark the test parallel

 func TestHeadCommit(t *testing.T) {
+ t.Parallel()
+
  testExe, err := os.Executable()

Attempt to run it

$ go test
--- FAIL: TestHeadCommit (0.00s)
panic: testing: t.Setenv called after t.Parallel; cannot set environment variables in parallel tests [recovered]

However, if refactoring our code similar to Inject the binary path is an option, we can resolve this issue by also injecting environment variables.

Injecting environment variables

Start at the Git struct from Inject the binary path, add an env field to it.

 type Git struct {
  // Exe is the path to the git executable.
  Exe string
+
+ // env specifies additional environment variables
+ // for git commands.
+ env []string
 }

Next, in the HeadCommit method, set Cmd.Env if this list is set.

 func (g *Git) HeadCommit(dir string) (string, error) {
  cmd := exec.Command(g.Exe, "rev-parse", "HEAD")
  cmd.Dir = dir
+ if len(g.env) > 0 {
+   cmd.Env = append(os.Environ(), g.env...)
+ }
  out, err := cmd.Output()
  if err != nil {
    return "", err
  }
  return string(bytes.TrimSpace(out)), nil
 }

We now have a means of injecting environment variables into the git invocations.