Martin Fowler called them sociable tests. The only way to properly test your units’ behavior is to pull in their dependencies. Isolated tests are useless, brittle and slow to write.
Honestly, I don’t think unit tests are a useful name. Everyone has a different idea of what a unit is and the line between unit tests and integrations tests is IMO not very useful. As long as your tests are
isolated from external factors (ie they completely control the test environment),
fast to run
repeatable, aka not flaky
can identify problems easily
Then where you draw the line between unit and integration is meaningless. It was meant to be that ingratiation tests were slow, so you wanted to shrink them down to make them faster to run. But I have not had a problem with the speed of more integration style tests in a long time.
I also don’t think interacting with the OS is such a bad idea. For instance the filesystem (what everyone always points to as an example) IMO is fine if done right. The big issue with interacting with the FS is keeping your tests isolated - too many people end up reading/writing the same file locations and thus breaking isolation. But you can always create a unique tmp dir for each test and do what ever you want inside that. Interacting with the filesystem on modern system is fast, and reliable - especially given that tmp locations are generally in ram these days.
I think the better term you are looking for is mocks and mocking. IMO these should be kept to a minimum. Like the above - you dont need to mock out the filesystem API when you can just use the filesystem in an isolated way. Same with network services - I really like gos httptest module, it lets you easily spin up a webserver that you can respond with whatever you need to. No need to create a mockable API when you can spin up a fast and reliable http endpoint to respond how you need it to.
Which leads to fakes (ie fake, simple implementations of a real external API). IMO these are far more useful than mocks and should be your first resort with mocks being your last resort. Such as things like gofakes3 an in memory s3 implementation in go that you can use any s3 client to talk to. Things like this let you create tests that you spin up the server (a unique one for each test), put objects into it to set things up how you need them, run your function and assert the contents are what you expect. Makes your tests more complete (and that you are not just testing your mock implementation rather than your actual logic) while keeping them isolated and fast - all the benefits of a small unit test combined with the wider scope of an integration test.
Couldn’t agree more with this comment and the thread in general, it’s a relief to see. I get so frustrated as so many of my colleagues seem to cling to this very old concept of the testing pyramid and associated definitions. It’s completely meaningless in a modern setting. We should mock as little and as far back as possible, yet others seem to delight in locking huge chunks of functionally out of the test base just ‘because’.
I believe we should have a new word that differentiates between ultra-basic tiny unit tests, and bigger unit tests that are still not integration tests.
E.g. rust and some other newer languages have a way to write basically an inline test for a function — that would constitute my former category. These make sense during development as a reality check. “Yes, this ad hoc stack I need inside this class should have two elements if I push two elems” sort of thing. That implementation may not even be accessible from the outside in case of an OOP language so you can’t even properly test it. Also, these are the ones that should change with the code and removing them is no big deal.
The other kind should work against the public APIs of libs/classes and they should not be rewritten on internal changes at all.
This is the correct comment.
Martin Fowler called them sociable tests. The only way to properly test your units’ behavior is to pull in their dependencies. Isolated tests are useless, brittle and slow to write.
Yeah I’m of the opinion that unit tests are usually a waste of time and people should only write integration tests.
The only time I think unit tests are valuable is for checking edge cases when e.g. interacting with the operating system.
Honestly, I don’t think unit tests are a useful name. Everyone has a different idea of what a unit is and the line between unit tests and integrations tests is IMO not very useful. As long as your tests are
Then where you draw the line between unit and integration is meaningless. It was meant to be that ingratiation tests were slow, so you wanted to shrink them down to make them faster to run. But I have not had a problem with the speed of more integration style tests in a long time.
I also don’t think interacting with the OS is such a bad idea. For instance the filesystem (what everyone always points to as an example) IMO is fine if done right. The big issue with interacting with the FS is keeping your tests isolated - too many people end up reading/writing the same file locations and thus breaking isolation. But you can always create a unique tmp dir for each test and do what ever you want inside that. Interacting with the filesystem on modern system is fast, and reliable - especially given that tmp locations are generally in ram these days.
I think the better term you are looking for is mocks and mocking. IMO these should be kept to a minimum. Like the above - you dont need to mock out the filesystem API when you can just use the filesystem in an isolated way. Same with network services - I really like gos httptest module, it lets you easily spin up a webserver that you can respond with whatever you need to. No need to create a mockable API when you can spin up a fast and reliable http endpoint to respond how you need it to.
Which leads to fakes (ie fake, simple implementations of a real external API). IMO these are far more useful than mocks and should be your first resort with mocks being your last resort. Such as things like gofakes3 an in memory s3 implementation in go that you can use any s3 client to talk to. Things like this let you create tests that you spin up the server (a unique one for each test), put objects into it to set things up how you need them, run your function and assert the contents are what you expect. Makes your tests more complete (and that you are not just testing your mock implementation rather than your actual logic) while keeping them isolated and fast - all the benefits of a small unit test combined with the wider scope of an integration test.
Couldn’t agree more with this comment and the thread in general, it’s a relief to see. I get so frustrated as so many of my colleagues seem to cling to this very old concept of the testing pyramid and associated definitions. It’s completely meaningless in a modern setting. We should mock as little and as far back as possible, yet others seem to delight in locking huge chunks of functionally out of the test base just ‘because’.
I believe we should have a new word that differentiates between ultra-basic tiny unit tests, and bigger unit tests that are still not integration tests.
E.g. rust and some other newer languages have a way to write basically an inline test for a function — that would constitute my former category. These make sense during development as a reality check. “Yes, this ad hoc stack I need inside this class should have two elements if I push two elems” sort of thing. That implementation may not even be accessible from the outside in case of an OOP language so you can’t even properly test it. Also, these are the ones that should change with the code and removing them is no big deal.
The other kind should work against the public APIs of libs/classes and they should not be rewritten on internal changes at all.