Check out the latest superpixel benchmark — Superpixel Benchmark (2016) — and let me know your opinion! @david_stutz


M. Fowler. Mocks Aren't Stubs. 2004.

Fowler's example implemented in C++ using GoogleTest and GoogleMock can be found on GitHub.

Fowler provides an excellent discussion on the difference between classical test-driven development (TDD) and mockist TDD. As the title suggests, the argumentation is based on the difference between mocks and stubs:

  • Stub: provide pre-programmed responses to call made during the test; often stubs avoid calling components out-of-scope for the test or record specific calls for later verification.
  • Mock: objects set up with expectations about the calls made when running the test.

While these definitions may be ambiguous for developers not familiar with the mockist approach to TDD, Fowler elaborates on the difference between state verification and behavior verification:

  • State verification: the success of a test is defined by investigating the state of the system-under-test and its collaborators after exercising it (assuming the general four-phase testing approach of setup, exercise, verify and teardown).
  • Behavior verification: the success of a test is determined by verifying that the system-under-test made the correct calls to the involved collaborators.

Only mocks, through expectations about function calls, insists on behavior verification. Stubs usually use state verification - as do all other kinds of test doubles, e.g. dummies or fakes. Overall, the difference between classical TDD and mockist TDD is made explicit through the following definitions:

The classical TDD style is to use real objects if possible and a double if it's awkward to use the real thing. So a classical TDDer would use a real warehouse and a double for the mail service. The kind of double doesn't really matter that much.

A mockist TDD practitioner, however, will always use a mock for any object with interesting behavior. In this case for both the warehouse and the mail service.

It remains the question which approach to use. To this end, Fowler discusses several advantages and disadvantages of mockist TDD compared to classical TDD. Advantages include:

  • Mockist TDD supports outside-in development - given a user story, one first writes test for an outside-facing interface which implicitly defines the interface of all collaborators through the expectations set in the used mocks.
  • Complex test setups can be avoided as only the system-under-test needs to be fully setup while all collaborators are mocked.
  • Mockist TDD improves test isolation as tests cannot fail due to changed collaborators.

The disadvantages include:

  • Tests are inherently coupled to the implementation as the expected calls are tested explicitly.
  • The integration of the system-under-test with its collaborators is not tested; as a result, the individual components may work as expected, while the interaction between them may still be flawed.

What is your opinion on the summarized work? Or do you know related work that is of interest? Let me know your thoughts in the comments below or using the following platforms: