Unit Testing: Time Consuming but Product Saving
“Today i finally recognized that unit tests are a critical part of my programming flow” — Ashley Williams, Twitter.
Long-time Node.js advocate Ashley Williams kicked off her recent Twitter thread in the same sort of stream of consciousness that many devs go through when they are coming to terms with the truth: Unit testing is annoying but necessary.
Unit testing is an essential part of developing software applications. Also called component testing, it is all about isolating one unit of code to verify that it’s working as it should be. And unlike many types of testing, this is usually done by the developer of the code itself.
A unit test differs from an integration test because an integration test, as its name suggests, focuses on the interaction between these units, modules, or components, as a unit test focus on one specific piece. The unit test also mocks behavior while the integration test runs on the actual code or in production.
Unit testing works best in conjunction with integration testing, but there are unique benefits from unit testing. This includes faster development because typically you write the unit test even before you write the code and then test your code against said test. And it catches errors at the unit level earlier on so the cost of fixing them is dramatically reduced.
Plus, since you have to make your code more modular to perform unit testing, you end up seeing an increase in reusable code. And of course the whole point is that unit tested code is more reliable, and with unit testing, you are able to know where the errors are coming from, from the start.
As Software Testing Fundamentals website puts it: “Unit testing is often neglected but it is, in fact, the most important level of testing.”
The Basics of Unit Testing
An example of a unit test could be of a specific feature or module, like the login of the app. In that case, you would test for the typical use cases of that module, like what happens when the login is correct, when it has the incorrect password or email, and when someone clicks “Login” but hasn’t actually filled in any information. Unit tests are particularly useful at simulating errors.
Brian Voong of Let’s Build That App online dev courses also gives the example of timestamps within the iOS Instagram app. What if someone mistyped “mims” instead of “3 mins ago”? This doesn’t break the code but it’s a pretty embarrassing error that your regular automated testing suite probably wouldn’t pick up. But a unit test would.
One way to perform unit testing is to write a test that will run every time any new code is released. If something is awry and a test has failed, your developers will be notified as to which line/s of code were changed and who made that change. It’s a great way to learn from your mistakes and it makes it easy to identify who should fix the error, and — especially convenient when compared with other types of tests — where that error is.
Most unit testing kits for both iOS and Android can be uploaded within your app and even include a test case you can use to model your test cases after and then delete.
“To write your very, very first test case, you have to just declare a new function and prefix it with the word of ‘test’,” Voong explains. You can then add variables in brackets behind it and follow through with creating your tests like the screenshot below. Typically, successful tests will show green checkmarks and failed tests will have red alerts.
In the following example, Voong is unit testing an app with a square root function.
Then, in order to make sure the unit test you’ve written actually works, you can command-click on the expected value — in the above example, on “square” — and modify the code to be incorrect. When you run the test again, it should alert you to an error with a red X.
Finally, when writing unit tests, like all testing automation, don’t neglect to include version control.
Is Unit Testing Really All That Important?
“Developing without unit tests is like being a trapeze artists without a safety net.” — Marjan Venema
Sadly in the rush of biweekly sprints and other deadlines, unit testing is often skipped — after all, the results are usually nothing, right? But by skipping unit testing upfront, you not only risk errors in the code, you also risk defects that distract from the business objectives.
When you first build an app and it is small, you may just perform manual regression testing, to make sure that your code hasn’t regressed to a previous version. You may have these manual tests in a spreadsheet and be running through them, but as your app progresses, that list starts to become insurmountable. This may even become someone’s job.
Unit tests pass, no integration tests pic.twitter.com/8geAsHgSBY
— Ryan Stortz (@withzombies) February 9, 2017
“And then to make everything more complicated, you realize that your customers may have different environments where they run your app. For instance, they might have different operating systems or different devices, different kinds of mobile phones, or they might be running them in different web browsers. And this affects your tests,” says Mattias Johansson, host of the Fun, Fun Function programming series.
“So this list of your hundred regression tests is actually multiplied by the number of operating systems you support and then multiplied by the number of devices you support, creating this…explosion,” he continues.
It becomes overwhelming fast.
“And it’s very rare that you will remove things from software, at least when compared with how often you add things to software,” Johansson pointed out.
This is why you need to automate your regression testing and keep your code simple with units.
Also even though unit testing is important, you don’t necessarily have to create test cases for everything, but rather you can focus on those that affect the behavior of the system.
How Unit Tests Fit in with the Rest of Your Tests
Kind of like the atom, unit testing is the smallest form of testing and thus means that it is testing your code at its core. But you have to test larger pieces too, so it’s important to unit test in conjunction with other testing. Below is API strategy consultant James Higginbotham’s API testing pyramid. He argues that unit testing is important but not the end all be all.
“Unit testing is primarily focused on ensuring that your code modules are behaving properly. For testing APIs, the most important focus should be on the acceptance tests, which verify that your API solves real problems and use cases,” Higginbotham told The New Stack.
“Functional testing may then be used to verify that each endpoint meets the expected behavior and honors the API’s defined contract for the consumer — ‘black box testing’. Finally, unit tests can be used to prevent internal regression of bugs by isolating portions of the API implementation — aka ‘white box testing’,” he continued.
Johansson agrees unit tests aren’t perfect finding the following two faults with them:
Unit Testing Drawback #1: They Do not Test the Contract.
By contract, he is referring to how that unit is expected to behave and with other units. The unit test will test that Component A will call a certain piece of data. And another unit test will check that Component B will return that data. But it doesn’t test the interaction together. He also says that if you make a mistake in the test itself, the test won’t capture those errors. An integration test is necessary to accomplish all this.
Unit Testing Drawback #2: You Have to Write the Contract.
It can be annoying to write boilerplate code simply for the act of testing that code. He says there has to be a clear separation between the interactions in order to write unit tests. On the other hand, an integration test doesn’t care if your pieces aren’t sufficiently “modularized.”
A major benefit to unit testing is a consequence of prioritizing unit tests. It naturally helps you expose spaghetti code and forces you to write cleaner, more modularized code from then on.
Yay, all unit tests passing! pic.twitter.com/ax2uxPsZqv
— Dave Hulbert (@dave1010) June 24, 2015
But probably the favorite for most devs is that unit tests actually let you know where your error is, while integration is just pass or fail. It’s also a faster manner of testing since integration testing has you running a real database — Johansson says a unit test could be hundreds of times faster. And since an integration covers many moving parts, he says that can make them “brittle” and significantly harder to write.
That doesn’t mean all teams are, can, or should be performing unit testing because of time and time again one drawback of a unit test is clear: time. They take time, particularly in a codebase that’s not set up to be unit tested.
Unit Testing in the Wild
Developer with more than 30 years experience and aspiring agile coach Marjan Venema told The New Stack about her experience with unit testing.
“Unit tests are an early warning system. If you have code that is complicated or complex, you may change something here, but it causes something to fail in another area of an application, and if you aren’t focused on that, you don’t see it,” she said.
On one app Venema was working on, a unit test discovered a bug that was only in edge cases.
“The unit test caused us to not put something into product and have a failure that we might not have noticed for months on end. If we had not had those unit tests, we would have missed those specific scenarios because they were not common but they were real because customers had data that hit those scenarios. And we would have put something out that would have caused incorrect results for our customers,” she said.
2 unit tests. 0 integration tests. pic.twitter.com/FpForNhhyi
— DEV Community 👩💻👨💻 (@ThePracticalDev) March 25, 2017
Write the Unit Test First
Many unit tests fans argue the test has to be written even before the code.
“If you’re not using a test-driven approach, the drawback of unit testing is that they usually follow the coding instead of being changed and corrected up front, which always leads to an ‘Oh gosh we still need to fix the unit test!’,” Venema said.
Test-driven development or TDD focuses on very short software development cycles where requirements are only turned into very specific use cases and the definition of Done is only passing those new use tests. Venema says this is part of the frustration associated with unit tests:
“When you are doing an intended change and you’re not starting to change the unit test — which is the way of working should be — then the unit test will start failing, and that causes a lot of frustration when people use the unit test and are doing the unit test as the afterthought or have the unit test as the definition of ‘done’ and they haven’t started with it and fixing a unit test is a chore.”
This is why she and many argue that the best way to write unit tests is at the beginning, before you start writing code.
“If you are changing the functionality and there are tests already in place, you need to change the test first and then change the code, so it works correctly and that way the tests help you. If you go the other way around, they feel like a burden,” she said.
She says that thinking of the test first gets you into more detail about what the feature is supposed to be doing and makes sure you are clear on the requirements — maybe even clarifying them with the product owner. And since unit tests are so good at checking error reporting, it makes sure you talk about when things go wrong and how error instances will look.
And even though writing tests takes more time, Venema argues that “You’re speeding up development by getting the requirements clear up front.”
She says that when you are changing a feature because there was a bug, that bug exists because it wasn’t in the unit test.
“You change the existing test to predict the correct outcome and of course when you do that, the current unit test is going to fail because the code base hasn’t changed yet. When you approach your test first, the unit test fails, so you know the test is correct, then when you change the code, and the test gets green, you know that you’re good,” Venema said.
She compares unit testing to an accomplished trapeze artist always performing with a net.
“Unit tests are often seen as a burden because they never catch anything, but if you don’t have one and you slip from the swing, you are just one pile of mess on the floor, but if you have the net, you can jump back on.” Venema continued to call unit testing an early warning system: “With unit testing, you can know why and where the bug is when someone reports a bug.”
But sometimes it can be nearly impossible to cover all tests.
When You Have to Write Around the Unit Test
As software services manager at RedEye, Adam Davis looks after the dev teams, who write lots of unit tests, hundreds even, along with thousands of function tests.
“We tell developers when they are picking up work they need to think about test writing for these features,” he told The New Stack.
Davis says that ideally, they write the test before the code, but that’s often unrealistic.
It’s “always the dream to be able to write the test before the feature. That’s what we strive to do but it’s not really practical.”
He said this is because often there’s a dependency on another team like databases that take time to get access to. In order not to delay development, the team has figured out a workaround. Instead, they have a meeting per feature to plan test automation ahead. This meeting always includes a tester and some of the other developers. This delays the test writing, but in order for a card to reach Done, a unit or feature test must be contained.
They have a heavy UI-centric application so they use unit tests to make sure basic UX processes — Does this button still work? Can you close a window? Can you still do that? Use that? — are working, for which they use more functional tests, while using unit tests to test more of the back-end and database connections.
Davis says they work to make all code “very atomic, something very small we can test.”
But it wasn’t always this way. For the 20-year-old business, they weren’t even originally running tests. Davis says their age “left a very big hole, testing deficit wise.”
At one point, they just had to stop new feature development and spend six months writing tests, so they know the potential cost of unit testing.
When asked if it was worth it, Davis said, “Definitely yes. It’s caught a lot of serious things before we were down the road and had a lot of damage. There’s a difference between knowing and not knowing. Our software releases are much more stable — no critical issues.”
He says unit and functional tests have also improved the role of QA, offering them more creative freedom. The Redeye QA used to have to perform all the tests manually by following scripts. Now they can focus on more exploratory testing.
Not only did this break in developing take a long time, Davis admits what they call their “coverage testing” in general takes a long time and is very complicated, which is why they run the automated tests at deployment time and every evening — about a thousand functional tests and a couple hundred backend unit tests. They run what they refer to as their “coverage” overnight, though developers can still run smaller demands on demand.
Redeye have recently made the transition from Selenium to ChromeDriver for their testing framework, automating all their functional tests. It took another long time to make the change, but they don’t regret moving their Web drivers to be able to tell Chrome what to do.
“Basically when you’re running a test, you don’t want any variance when they run locally or in an environment. ChromeDriver is doing same thing in the server. There shouldn’t be any discrepancy between what the testing environment and what the user does,” Davis said.
Unit Testing Is about Culture too.
Like all changes, you have to develop a culture of unit testing, as it has to be a habit developed with your team. By forming this habit, you not only will assure in your quality of code, as well as “unit testing increases confidence in changing/ maintaining code.”
Johansson offers three rules that you have to set up — and agree on as a team — to develop a unit testing habit.
Rule #1: Unit Testing is Part of the Definition of Done.
Just like at Redeye, Johansson says: “Decide that commits must be unit tested…If a commit does not have unit tests, it does not go into the repository.”
This is a tricky one to sell at the start of a project because the code is so manageable to test. But it will get more complicated. A lot more complicated. Which is when he says you make that sale.
Of course, this will play into Rule #3 because you will have to untangle your code to do this. But, knowing this rule ahead will create cleaner, more isolated-able code, and the habit of unit testing can be formed.
Rule #2: Teach with Mandatory Code Review (Which Will Teach Team to Unit Test)
Unit testing is often performed by the writers of the code, but it can also be performed by your peers.
As LinkedIn Tech Lead Szczepan Faber defines it, a formal code review process “requires every code change to be officially reviewed by another team member before the code goes to production.”
LinkedIn uses code reviews for quality as well as professional growth. This has resulted in a lot of standardization and a more open company culture of feedback.
Johansson says you should include peer code reviews as part of the unit testing process. This improves the quality also of the unit tests and captures more edge cases.
Unit testing is also really interesting way for new developers on the team to get to know the code and the app’s whole user experience. Reading through your unit tests is a great way to see the backend and there’s still no better way than using the app to get the UX.
Rule #3: Unbraid Your Code into Units
Of course, if you want to make your code simpler, you’re going to have to untangle its web. And then when you write unit testing, the result is less complicated code.
One last benefit of unit testing? You can learn a lot from testing your own code.
