I actually love that they’re so brittle because it quickly catches problems that need fixing.
Tests are not brittle when they catch actual problems. Tests are brittle when they break for no good reason. And really when you are refactoring something tests should not break - you are not changing behaviour, just reorganising things. If you need to do big changes, rewrite or even delete tests when you refactor something then your tests IMO are brittle and much less likely to catch regressions in behaviour. Yeah you will need to tweak them some times, but these should be kept to a minimum and not be happening every time you refactor anything.
When writing new code small tests can feel good, they are easy to write and you can test things quickly in isolation. But after that point how much value do they give you? Tests do have a cost - beyond the original time you spent writing them, you have to maintain them, and keep them uptodate, they take time to run etc… If they cannot properly catch a regression when it happens then IMO they are not worth the cost of keeping them around. Larger tests tend to be better at catching regressions and are less prone to need to be tweaked when you refactor the internals of something.
So, generally speaking I tend to prefer testing public APIs of something and ignore internal helper functions (unless some helper is sufficiently large/complex enough that it warrants extra dedicated tests, which is not common). Note that this does not mean public to downstream users of your library/script/binary, but also larger internal modules API that the rest of the application can use. Though I do find quite a few smaller applications you only need to test the actual public API from an end users perspective.
A lot of that has to do with type checking and a lot of the methods would have huge consequences if they were off.
Uhg, This is a big reason I don’t like loosely typed languages. Yeah that might be one case where smaller tests are needed but IMO testing input types should not be required at all - that is something a compiler can do for you for free. At least assuming you have a strongly typed language. People always say loosely typed languages are faster to code in - but this benefit is completely lost when you spend ages writing tests for different inputs types that a compiler in a stronger typed language would just reject.
deleted by creator
Tests are not brittle when they catch actual problems. Tests are brittle when they break for no good reason. And really when you are refactoring something tests should not break - you are not changing behaviour, just reorganising things. If you need to do big changes, rewrite or even delete tests when you refactor something then your tests IMO are brittle and much less likely to catch regressions in behaviour. Yeah you will need to tweak them some times, but these should be kept to a minimum and not be happening every time you refactor anything.
When writing new code small tests can feel good, they are easy to write and you can test things quickly in isolation. But after that point how much value do they give you? Tests do have a cost - beyond the original time you spent writing them, you have to maintain them, and keep them uptodate, they take time to run etc… If they cannot properly catch a regression when it happens then IMO they are not worth the cost of keeping them around. Larger tests tend to be better at catching regressions and are less prone to need to be tweaked when you refactor the internals of something.
So, generally speaking I tend to prefer testing public APIs of something and ignore internal helper functions (unless some helper is sufficiently large/complex enough that it warrants extra dedicated tests, which is not common). Note that this does not mean public to downstream users of your library/script/binary, but also larger internal modules API that the rest of the application can use. Though I do find quite a few smaller applications you only need to test the actual public API from an end users perspective.
Uhg, This is a big reason I don’t like loosely typed languages. Yeah that might be one case where smaller tests are needed but IMO testing input types should not be required at all - that is something a compiler can do for you for free. At least assuming you have a strongly typed language. People always say loosely typed languages are faster to code in - but this benefit is completely lost when you spend ages writing tests for different inputs types that a compiler in a stronger typed language would just reject.