A thousand tests: the world of software testing
15/11/2023
Mid-October, an autumn day. It’s raining outside and you’re grateful to be working remotely. Music in your headphones helps you focus. Fingers tapping on the keyboard.
Suddenly, an email notification breaks your concentration from the code that, strangely enough, wasn’t behaving quite right: another pull request had been merged into the main branch.
But this one was special. The release pipeline had successfully run 1,000 tests. Message to the team. A tray of croissants to celebrate the milestone 🥳.
That morning at Aton, when we realised we had reached 1,000 tests in the backend codebase, went more or less like this. It was a small but meaningful achievement, made possible by a journey and a shift in mindset that I’ll try to share with you in this article.
Let’s begin!
Tests and TDD (Test Driven Development) are widely discussed across the tech community, with a substantial body of literature on the subject. Stripped right back, writing tests means writing code that verifies and locks in the behaviour of other code.
Verifying behaviour means checking that, given a certain input, you always get the same output—both in the simplest, most linear cases and in the trickier, more deceptive ones. It also lets you confirm whether what you’ve written actually matches the specifications defined during the analysis phase.
Locking things in, on the other hand, matters even more for keeping a codebase maintainable. A codebase evolves over time and is used by multiple people at once—who may change shared or legacy logic. Having tests as a record of what’s been done gives developers confidence that what they implement won’t break compatibility with what already exists.
None of this comes for free. Writing tests also means stretching development time to include activities like test setup (e.g. mock data, database seeding, configuration parameters), the tests themselves, and fixing business logic when—inevitably—some tests fail.
At the same time, this extra development effort pays for itself when the code doesn’t blow up in production and saves the developer a few sleepless nights of debugging.
OK, that’s the theory. But how do we write tests at Aton?
Since we began developing new product lines at Aton using modern technology stacks (Java/Kotlin and Spring Boot for the backend, and Angular 2 for the frontend), writing tests has become part of our developers’ daily routine.
Wherever possible, we aim to follow TDD: given a feature, tests are written first, and only then the business logic for that feature. Once development is complete, the tests are run to check whether the behaviour defined at the start is respected by the implementation.
If all tests pass, the feature can be considered complete.
If some tests fail, there are two possible reasons:
1. The implementation does not comply with the specifications—for example, it doesn’t handle certain corner cases or it returns a data set inconsistent with expectations;
2. The test definition is wrong—meaning the tests don’t reflect the specifications, or they were created with input data that doesn’t match the expected output.
Regardless of the reason why tests have failed, the developer must go back into the code to ensure that all the tests relating to the feature under development turn green. Once this happens, the work can be considered complete and the developer can open a pull request with the changes.
This is when the second phase of the backend development process comes into play.
Creating a pull request triggers a pipeline in Azure DevOps which diligently runs the tests for the new feature as well as for all existing ones. This step ensures crystallisation: it checks that the new logic doesn’t conflict with what is already in place.
If something breaks, the pipeline stops and prevents the release from being merged into the main branch.
Writing tests has brought great value to Aton, and we intend to continue supporting this practice over time.
That said, the way we write tests may not remain the same forever.
The topic of generative artificial intelligence—and how it can become a tool to support developers (and beyond) by boosting productivity—is widely discussed and highly relevant within the tech community.
Among all the tools currently available, GitHub Copilot is certainly the most useful for professional coders—even when it comes to writing tests. By providing the right context, GitHub Copilot can generate unit tests (and other types) for a given feature.
Using this approach comes with a number of pros and cons:
👍🏻 Shorter development times. Developers spend less time writing tests and more time on business logic.
👍🏻 Unforeseen test cases. AI can generate test cases the developer may not have anticipated, thereby strengthening the codebase.
👍🏻 Unbiased test cases. Developers often feel attached to their own code and may end up writing conservative tests that don’t really challenge it—especially when the same person both implements the code and writes the tests.
👎🏻 False sense of security. AI-generated tests must always be reviewed by the developer to correct any misinterpretations or overly vague contexts.
Whether this will become the definitive path for testing is something we cannot know for now.
However, at Aton we have already launched research and development activities focused on GitHub Copilot to understand the impact it could have on a developer’s daily life.
If, at the beginning of this article, you were wondering why we celebrated the arrival of the thousandth test so much, it should now be a little clearer.
A thousand tests are not just a statistic, but the sign of a journey that has changed the way we work. A journey that is gradually industrialising the development process at Aton and that, thanks to testing, enables us to produce quality software.
If you are still asking yourself whether writing tests is worth it, at Aton we can answer: yes, it is.
It’s worth it because it changes the way you work. It’s worth it because it changes the way you think. It’s worth it because it improves the quality of the code you write. It’s worth it because it gives you the certainty that what you’ve written doesn’t break what was already there. And it’s worth it because, in the end, seeing all those green ticks when the tests pass gives you a pretty good dopamine hit!
