Test, Develop, Maintain: Short Guide for Sustainable Programming

Photo by James Harrison on Unsplash

In terms of their “useful period”, computer programs can usually be divided to two categories: short-term purpose and long-term purpose. Short-term programs (such as single-use data processing scripts) are usually coded directly by the user who will benefit from the program, serve a niche and short-term purpose, and thrown out after the purpose is complete. In this case, speed is usually the highest priority rather than maintainability. However, the opposite is true for long-term programs which constitutes most of what non-programmer users see as “a computer program”. Here, the program is usually designed to serve a broader audience, fulfill a more general purpose, and likely need to be maintained for extended periods.

For long-term purpose programs, it is imperative that programmers must make their program sustainable by incorporating some proven best practices. This article will show the basics of one such best practices: Test, Develop, Maintain (commonly known as Test Driven Development or TDD).

Addressing the Elephant in the Room: Why?

Some may dismiss the practice of creating tests first before developing a program due to the associated high capital expense. Testing first is seen as an unnecessary step that does not fully ensure the developed program will run without errors so it can be ignored without significant consequences. However, seeing tests as an “insurance” that the program will not have errors misses the whole point of the TDD practice. The main purpose of TDD is to assist in the planning of the program and provides a measure to properly assess the quality of the program. Whether everyone likes it or not, most programs, especially those used for actual business purposes, will go through a testing process anyways. Here are some reasons why TDD is preferred:

1. Most programs already have an acceptance criteria anyways. TDD ensures that the method to evaluate the criteria is in place even before the program is developed. Therefore, programmers can have an easy way to evaluate their program.
2. Creating tests encourage those who are involved in the development to understand the problem and solution better. By doing this from the start, the development time can usually be reduced.
3. Tests created from before the development process impose a common standard across all programmers working on the program.
4. Creating tests assists in the “divide-and-conquer” thinking and encourages modularity. It is easier to test small solutions rather than large solutions.
5. Tests offloads the “intended” solution from the developer’s mind to a tangible artifact. This eases collaboration since a new collaborator can be easily onboarded by introducing them through the tests first.
6. Well-developed tests can consistently reproduce pitfalls which ease the debugging process. Some of the tests can even be automated to reduce testing time.

In a professional environment, the question of whether TDD is important is even considered irrelevant. TDD has become so important for sustainable programming that the relevant question is now about how to implement them.

The Init-Red-Green-Refactor Workflow

TDD is often also known by its standard workflow sequence: “red-green-refactor”. The name comes from the default color of a testing cycle in which red is associated with failure and green is associated with success. Each part of the sequence (which also includes Init in this article) refers to the major steps in a test-driven development setting as shown below:

1. Init: This part mainly involves the development of a program skeleton. The skeleton should not contain any logic except for the program’s structure which will be used in developing the test.
2. Red: This part is where the tests are created. The tests should not be influenced from any logic implementation and focuses on what the program under test needs to do. The tests should induce a program failure since the implementation is yet to be written.
3. Green: This part involves creating the implementation according to the guidance set by the requirements and the tests.
4. Refactor: Most programmers cannot write an optimal code in one shot, especially for complex programs. While the tests may pass with suboptimal code in Green, this part covers the steps to reach that optimal code. It is important to note that this code should still pass the test.

If a version control system is used in the program’s development, it is strongly recommended to commit codes at every parts. In addition to assisting in damage recovery, this will also make collaboration more seamless since the artifacts can be published, reviewed, and used by others in parallel, which can introduce constructive feedbacks.

The Various Types of Tests

While TDD is usually seen as closely related to unit testing, tests in a TDD practice can actually include an entire suite of tests that is designed to comprehensively test the program according to its acceptance criteria.

In the development of a large program, there are usually multiple abstraction layers of acceptance criteria where each layers have their own test suite. Starting from the innermost layer (the code implementation layer), some tests that are available are as follows:

1. Unit Testing handles testing in the code implementation level. These tests are developed for the smallest program unit (functions, classes, etc.), and are closely related to the actual code that is being developed. Example of such tests include Python’s unittest and Java’s JUnit.
2. Integration Testing handles testing between multiple parts of the program to ensure that they can be “integrated” into a whole program. The test usually involves executing through a whole use case flow instead of testing a specific part of the flow. Libraries for unit testing can also usually enable integration testing.
3. Validation Testing measures whether the program as a whole meets the program’s initial requirements and expectations. This is usually referred to as “end-to-end testing” since the whole program is under test, usually under its intended usage environment, such as using web browsers or API tools for web-based program or using emulators or actual devices for mobile-based program.

Developing an Effective Test Case

A case study from the development of Crowd+ software by Nice PeoPLe team
Note: This software is developed under Python which does not have strict type enforcements. Some suggestion might not be relevant for statically-typed languages.

While a dedicated professional is usually available for test case development (usually known as Quality Assurance Engineer), programmers must also familiarize themselves with creating test cases, especially in the code implementation level. This constitutes the “first ring” of software testing and is the first line of defense for the program’s development. Therefore, this part will focus on TDD in the code implementation level (unit testing), although some tips are applicable for other levels as well.

In this level, it is widely known that “coverage” is one of the important metrics of a test quality. Put simply, coverage shows how much implemented code is covered by the available test suite. In an ideal TDD scenario, the coverage should be pinned at 100% which means that all parts of the code are tested. Do note that tested does not necessarily mean properly tested. Therefore, it is important to plan your implementation thoroughly, including possible alternative scenarios.

For example, I were to develop some repository methods for User models according to the following criteria:

(some requirements are omitted for the sake of brevity)
Users are uniquely identified by the email that they registered to the site. Therefore, a method to fetch the user’s data from the database is required to be developed which uses the email as the “key”.

Init

The first will be the Init phase in which the code skeleton for this method needs to be created. Let’s say that this method is called get_user_by_email which receives one argument: user_email. Therefore, the skeleton will be as follows:

Observe that the above code barely contains any logic whatsoever. It simply defines the method signature (including its intended inputs and outputs). In the Init phase, the result should be a code skeleton that is runnable but does not have any logic. Code skeleton does not mean that only the signature should be provided. At minimum, the method should at least be complete by adding a stub method body (such as returning null/zero values). This will be important during the testing phase.

Test

During this phase, I need to think deeper about this method to be able to create an effective test case. Here, we will need to draw a line between testing a program and properly testing a program by setting the criteria for an effective test suite as follows:

1. The test suite should be able to completely test the program (100% coverage).
2. The test suite should be able to test alternative scenarios and uncover potential errors.
3. The test suite should be able to be developed in a timely manner.

One of the unit testing strategy that I use for this software is positive-negative testing. In a positive-negative testing scenario, tests are divided into those which uses the correct inputs and those which uses the wrong inputs. In a data-driven program, this testing strategy is very effective at uncovering potential errors.

In the case of the method that I am currently developing, which will need to fetch user’s data from the database according to their email, there are two possible scenarios:

1. The email refers to an existing user in the database so the method should yield a user data.
2. The email refers to a nonexistent user so the method should not yield any data.

Note that I am not testing the possibility of a database problem. Since I am testing the repository’s behavior, database problems are irrelevant and must be tested in the database interface itself. Therefore, there will be two test cases as shown below:

It is important to consider that a test preferably should only induce failures, not errors. Testing libraries usually provides distinction between errors (the test fails to run properly) and failures (the test does not receive the expected values). This is why the code skeleton should be a complete method. Errors due to incomplete code skeleton or other reasons may cause the test suite to fail even though the tests and its associated implementation is already completed later on. These initial two phases must be coordinated in such a way that the test without the implementation should only induce failures due to unexpected values. This usually means that the test is written properly.

Note that the tests put not-None assertions first in order to induce failure when None is returned instead of errors due to NullPointerException. This is important when using null values as a stub (or a possible return value).

Implementation (Green-Refactor)

Once the test suite is complete, the implementation can now commence. While developing the implementation, it is important to keep consulting the tests, especially for expected values. In this case, since I initially planned that if no user is found with the associated email, the method should yield no data (None), I need to add a defensive mechanism to catch the ObjectDoesNotExist exception to return None instead of throwing an exception. This shows the importance of advance planning using tests.

Therefore the final implementation code is as follows:

What if My Tests Have Problems?

Some people may worry when implementing their program using TDD and discovering that they have bugs or other problems in their tests while they were already in the Green phase. That’s okay since getting the test right on first try can be challenging, especially for large programs. Requirements can also change which can lead to test modification. Modifying the test in the middle of the Green phase is fine, as long as you commit to not let your implementation influence the test modification. Be sure to only modify the test to address the problem or the changing requirements.

Key Takeaways and Tips

1. The Test, Develop, Maintain principle is critical for programmers hoping to achieve sustainable programming.
2. Creating tests first will improve long-term programming performance by putting comprehensive quality assurance methods in place as early as possible.
3. Testing does not strictly mean unit testing, but can cover a broad range of tests catering to various layers of acceptance criteria.
4. Code skeleton that is created before testing and the testing kit should be designed such that it only induces failures, not errors.
5. Tests should be able to cover all written codes if TDD is properly implemented.