Blog
Automated Tests: The Why
Introduction
Testing is an essential part of software engineering, and their importance cannot be overemphasized. Over the years, the approach to testing systems has evolved a lot. Currently, a lot of emphasis is placed on automated testing.
Despite the fact that automated testing is quite an old concept (for example, the Test-Driven Development By Exmaple book dates back to 2002, i.e. over 20 years ago), I still believe that as an industry we are not using full potential of automated tests.
The main reasons for this situation, in my opinion, are as follows:
- Lack of automated testing culture in teams
- Lack of appropriate skills, knowledge and experience in test automation
- Lack of understanding what automated tests really give
- Emphasis on rapid growth, feature-driven development without thinking in the long term
- Not seeing automated testing as an investment
- Myths related to automated tests and approaches such as TDD and BDD
- Greater faith in manual testing
- Sub-optimal testing strategies causing things to get even worse, ending up abandoning automated testing altogether
- No focus on the testing process at all
In this series of articles, I aim to cover all aspects related to automated testing and testing in general. I will start with one of the most important aspects - the purpose and the problem they solve.
The “Why”
Let’s consider why automated testing can be beneficial in software development. Assuming that we are creating software that will be used for a long time, we can be sure of one thing - changes.
Changes accompany us on a daily basis and can be caused by many factors, such as:
- New business needs requiring new functionality
- Bug fixes required to correct incorrect behavior
- Required adherence to new legal standards
- Performance improvements required
- The need to update a library (e.g. for security reasons)
Since changes are inevitable, it is helpful to determine how easy it is to make changes in a given time. Let’s see what this looks like in a typical project. Assuming we have 2 related functionalities at the start:
Simple, uncomplicated system with 2 existing functionalities
The question is, is it easy to change functionality 1 or functionality 2? Or possibly add new functionality 3? It can be assumed that such changes will be easy to implement. The number of functionalities is small, so it is easy for one person to understand. In addition, there is only one connection, which reduces the complexity of the entire system. The less complex a system is, the easier it is to understand. The ease of understanding the system affects the ease of implementing and testing any changes.
But let’s consider another example:
Complex system, many functionalities dependent on each other
The project is already quite advanced, containing many interrelated functionalities. Is it still easy to make changes in this case? Suppose we want to modify Functionality #8. This functionality depends on previous functionalities, so understanding them is necessary. The result of this functionality is related to other functionalities that also need to be understood, and importantly, we can accidentally break them.
Therefore, it can be concluded that introducing a change to the system with such complexity will be much more difficult.
Let’s see how this looks on the graph, where the horizontal axis is time and the vertical axis is the ease of change:
Ease of change VS Time
As the project progresses and more functionality is added, the number of connections and overall complexity increases, which can lead to decreased productivity.
However, this doesn’t necessarily mean that every system developed over a longer period of time will be slower. This challenge can be addressed through the use of automated tests, which are an essential part of Evolutionary Architecture.
Evolutionary Architecture
As the authors of the Evolutionary Architecture concept themselves claim:
“An evolutionary architecture supports guided, incremental change as a first principle across multiple dimensions.”
The question is how does this architecture support continuous change? The authors mainly mention Continuous Delivery combined with Software Testability and Modularization. In the context of testing, we will focus on the first two, so let’s see how it works:
Evolutionary Architecture, Continuous Delivery and Flywheel
It all starts with automated tests that allow us to refactor our system. Thanks to refactoring, the system becomes more testable, which makes it easier for us to add more automated tests. All these 3 elements form a feedback loop that drives the further process.
Thanks to refactoring, according to the definition, the internal structure of our system is constantly improving. This makes it easier to understand and make changes with it. The ease of change means that we can reduce costs or increase income, which ultimately increases the business value of our system.
In this way, we can achieve a relatively constant ease of change and sustainable growth:
Ease of change VS Time - Evolutionary Architecture
However, we will take a closer look at why automated tests are so important in this process - what exactly they give us.
Test Automation Pros
Fast feedback loop
Manual testing takes too long. If you want to make changes, you can’t rely on manual tests. Even if you have a large QA team, they will not be able to fully test a complex system quickly. Using automated tests it is a matter of seconds or minutes. Manually - hours, days or event weeks. The system cannot be considered easy to change if we have to wait that long. Moreover, what if errors occur during the manual testing process?
Fast Feedback Loop
Faster development
If you have automated tests, you can quickly set up the system that needs testing (System Under Test) without the need to figure out where to click, how to retrieve data, or how to restore the database from a backup.
Automated tests will prepare the environment for you, allowing you to experiment, test different implementations, and iterate quickly. Without automated testing, you may be reluctant to make changes to the system once you’ve implemented its functionality.
Faster Development
No manual, human mistakes
Let’s be honest - people aren’t good at performing the same tasks repeatedly. There’s room for fatigue, monotony, and inevitably, errors. How many times can you run the same test case scenario without missing something or making a mistake? Even the most skilled tester is prone to slip-ups over time.
On the other hand, computers were designed to automate mundane and repetitive tasks. If programmed well, they won’t make unexpected errors or become disengaged. They won’t deviate from their assigned tasks, and their behavior is entirely predictable. Given these benefits, why not leverage automation whenever possible?
Safety net
If you have very quick feedback, you are able to quickly determine if your change is working and if the rest of the system is working properly. This creates a safety net. The more automation you have, the larger this safety net becomes, and the easier it is to introduce further changes (part of flywheel).
Without automated tests, any change can cause errors in the system, and as mentioned earlier, manual tests may detect the errors too late or may not detect them at all.
Safety net
Better design
Tests drive design. If your system is testable, there is a good probability that it is well-designed as well (although we cannot be 100% certain, as there are many factors that contribute to good architecture). Tests can cause:
- Your architecture to be more modular and loosely coupled
- Your modules and objects to have higher cohesion
- Your implementation details to be hidden behind observable behavior
Documentation
The tests illustrate how the system works. They are executable examples, a specification. Thanks to this, they are always up-to-date.
They illustrate what a business process looks like, how algorithms and business logic work, and what the logic of the application and the business rules are. They show how to use given classes or entire use cases.
Are they sufficient as documentation? Probably not, but they certainly play a big role in understanding the system and the business.
Refatoring capability
All the points above mean that we can continuously refactor our system. Without it, we will be afraid that we will break something. If there is fear, we will not do it, which will cause the quality of the system to decrease, which will mean that fear will increase even more. The flywheel will work, but this time to our disadvantage, causing an ever-larger Big Ball of Mud.
Test Automation Cons
Are there any downsides to automated testing?
First of all, as with any tool, you must learn how to use it. An inadequate testing strategy will make the system even harder to change because the code will be cemented by testing. Changing one feature may result in changing more tests than necessary.
Another thing is that test code is code that costs money, it’s an investment. Tests should be maintained, developed, and refactored just like production code - because test code is production code.
Moreover, test writing is a skill that your team needs to acquire. It’s not like we can decide overnight to write tests and everything will be easy. Continuous training, learning, and practice are crucial here, requiring time and trust from decision-makers. We can’t learn it after one webinar or YouTube video.
Excuses
Finally, I would like to deal with the 3 most common excuses why automated tests are not written.
I have no time
It’s not true. As I mentioned above, if your testing strategy is right and you have the testing skills, you will develop faster with automated testing than without. Unless you’re making a greenfield, “to-do list” application with 3 entities and 2 screens - but then, it doesn’t matter what methodology you adopt.
This is a startup, without tests it will be cheaper and faster
It won’t be faster or cheaper. On the contrary - it will be slower and more expensive. If you want to go fast, you need to invest in quality from the start (again, see flywheel). There are no better ways to invest in quality than automated tests - they pay for themselves very quickly.
Automated testing is difficult
It’s just as difficult as writing production code - you just have to learn it. This is another skill. And there’s no better way to learn it than through practice.
In the next posts in this series, I will introduce more theory on how to make system testable, with proper testing strategy and good quality automated tests. Let me start with a crucial attribute of system architecture - testability.
Summary
Below are the key conclusions:
- Still, as an industry we are not using full potential of automated tests
- Change is certain, so ease of change metric is key in most projects
- Evolutionary Architecture support continuous change using Continuous Delivery and automated tests approach
- Fast feedback loop, faster development, predictable process, safety net, executable specification and refactor capability are main pros of automated tests
- The need to learn and practice automated testing, as well as the costs of development and maintenance, are disadvantages that your team needs to deal with.
- There are myths and popular execuses about automated tests - that they are difficult and there is no time to write them. None of them are true, if you know how to do it right
More in series
This post is part of the Automated Tests series: