To Kill a Mockingtest

Please don’t use mocks or stubs in tests.  While they are seemingly ubiquitous in enterprise development, they have serious drawbacks, and typically mask easily fixable deficiencies in the underlying code.

Even their most ardent defenders concede that mocks and stubs have flaws. These include:

Dependence on fragile implementation details

Mocks and stubs require intimate knowledge of how code interacts with other modules.  Even if the implementation is correctly refactored without altering public contracts, these tests will tend to break, and draw your attention away from more productive tasks.

Testing incidental properties with no bearing on correctness

What is the point of this code?  This is an essential question to ask, in order to understand it.  Tests have a story to tell here, and mocks invariably tell the wrong one.  Is the point of makeCoffee() that we made a coffee, or that we opened the fridge to get the milk?  When we payShopkeeper(), do we care that we completed a transaction, or that we rummaged though our wallet for change?  When mocking tests fail, the poor maintainer is left to reconstruct the real intent from a trail of indirect clues and anecdotes.

Web of lies

It is good practice to write data structures that are correct-by-construction; any constructor or sequence of method calls is guaranteed to leave the data in a meaningful state.  Stubs introduce test-only fictions that are stripped of any of the safety latches and guarantees that may have been built in; they introduce fresh sources of error that are not present in the codebase.  There is no value in detecting any failure that arises in such a way.

As time goes on, lies beget more lies.  It is not unusual for a stubbed input in one place to result in another here, and another there; the fiction leaks and spreads into some kind of evil facsimile of the original code, but with more bulk, complexity and defects.

Continue reading