Comments on SandyWalsh.com: Effective Units Tests and Integration Tests

Good post, Sandy. I disagree on a couple of points...

2011-11-22T12:08:44.574-04:00

Good post, Sandy. I disagree on a couple of points, though.

I'm much, much, much more comfortable using fakes that maintain some amount of state than relying on the continued correctness of thousands of ad-hoc mocks objects. If the fake exposes more than a couple(!) of methods, I just add tests for the fake. I can run the tests against the real thing and against my fake, and see that they do the same thing. This pattern has served me quite well a number of times now. The amount of ad-hoc mock objects we have in Nova right now gives me the heebie jeebies. A method that mocks out a call to the db layer returning an object with an attribute that has since been moved could go unnoticed for months until some almost entirely unrelated change reveals that something has changed incompatibly. This is pain to clean up after.

I disagree about your 80%/20% thing. I think unit tests are even *more* important for the things that only 20% of your users use. If the stuff that everyone uses breaks, you'll find out in a jiffy and someone will always be up for contributing tests for that. The exotic stuff is exactly the sort of stuff you need help verifying.

I think your example of a method that doesn't need unit tests is much too general. Sure, I can think of many examples where such a method could acceptably be untested by unit tests, but in many cases, the correct, expected behaviour of that foo() method is exactly to execute those three methods in that exact order. If your mock library of choice doesn't let you verify this easily, I'd certainly whip up an ad-hoc mock that does.

Great post Sandy! You have a great way of breakin...

2011-07-14T15:00:23.197-03:00

Great post Sandy!

You have a great way of breaking this stuff down...

On the mocking side, I've come to love mocking frameworks for doing stubbing instead of trying to create crufty mock implementations to maintain...mockito is my favorite. I find that inline mock definition (ie when(mock.getSomething()).thenReturn("something")) much easier to use and maintain.

Great post, Sandy! One of the things I would ment...

2011-06-30T18:02:01.748-03:00

Great post, Sandy!

One of the things I would mention here is that the terminology around all of this is so loosely defined as to be stupid.

In a previous life, we simply referred to tests as blackbox versus whitebox. The former was more like integration/functional whereas the latter was more like unit tests. In that same vein, the last couple months have in turn told me that the things I thought of as "integration" tests were truly functional tests, and vice versa. As such, I've personally come to define things "unit tests" and "not unit tests" for simplicity's sake.

Regarding mocks, I personally believe their value should be evaluated on a case-by-case basis, as they can lead to some incredibly brittle tests, especially if you're enforcing the order of method calls. As for stubs, I've come to believe the only real difference between these and mocks is whether or not you verify the mock was called explicitly, and what parameters the mock actually received. One of the testing frameworks I've used in Ruby made this same distinction: stubs only returned a value necessary for a test, whereas a mock would vet the input and place a hard requirement on whether or not it was called.