Bugs! Every application has it and no matter how careful you are you will create one sooner or later. Especially if you are working on large enterprise software. Now, I know a bunch of people have a “My code is clean and bugless” attitude, but creating bug-free code is extremely hard if not impossible. However, that doesn’t mean that you should give up and just write spaghetti code that just doesn’t work. We – developers, should give our best to write high-quality code. High-quality means a low number of bugs, among other things.

Apart from that, having a bug in production is extremely expensive. You probably know that comparison, where a bug found during development is 100 times cheaper than finding the same bug during production. So, we should focus on finding our bugs as soon as possible. Our first line of defense is testing. That is a scary metric.

We could test our applications manually, just by running them and clicking around. However, this approach has many pitfalls. The first one is that it is time-consuming, which basically means expensive. This way of testing makes regression testing extremely hard too.

Imagine that you’ve just added a new feature to your application. You will have to make sure that this new feature didn’t break any of the old functionalities, which means testing your whole application from the beginning. Again, time-consuming and costly. What is the solution? Automated testing of course.

In this article we cover:

1. Automated Tests
2. Unit Tests
3. What is Test-Driven Development?
4. Solving a problem with TDD and Python
5. Mock Objects
6. Patching

The video version of the article: 

1. Automated Tests

As we could see, manual testing is not really working for us especially if we want to detect issues early during the development phase. So, we decided to automate our tests. Based on the level of abstraction that tests are done they can be:

• Unit Tests – It is a piece of a code that invokes another piece of code (unit) and checks if an output of that action is the same as the desired output.
• Integration Tests – It is testing a unit without full control over all parties in the test. They are using one or more of its outside dependencies, such as a database, threads, network, time, etc.
• Functional Tests – It is testing a slice of a functionality of the system, observing it as a black box and verifying the output of the system to the functional specification.
• Acceptance Tests – It is testing does the system fulfill expected business and contract requirements. It is considering the system as a black box.

These definitions are a bit loose since everyone is using different names for different types of tests. For example, some people use the name “developer’s tests” instead of “unit tests”, because those are tests that developers can read and understand. However, we will not go down that rabbit hole, important thing is that you get the point.

In the image above, you can see the so-called Pyramid of Tests. It displays the number of tests that we should have in our application per type of test. We can see that we have the largest number of unit tests. For the purpose of this article, we will consider only this type of test, since they are crucial for the TDD process, as we will see in a little bit.

2. Unit Tests

As previously mentioned unit tests are testing the functionality of a unit. This brings us to a philosophical question about what exactly is “unit”? A unit is the set of actions between the invocation of a method in the system and a single noticeable output of that system. The equally philosophical answer, right? The important thing to understand here is that the unit test is a piece of code that tests another piece of code. Over the years, this type of test turned out to be one of the best tools for increasing software quality. They were introduced by Kent Back in Smalltalk back in the 1970s and since then they are used in pretty much any programming language.

So, how can we write unit tests in Python? Unit tests are always written using some sort of unit test framework. For Python that is module unittest. Here is how we can use this module to write our first tests:

First, we create the class FirstTestClass, which is inheriting TestCase from the unittest module. Using this inheritance we are defining a test class that contains our test methods or test cases. These test cases will be registered within unittest module and we will be able to run them later. In this class, we are having only one test case test_upper.

This method uses function asserEqual to verify that the call of the upper method on the string really returns the same string with all caps. The module unittest has a lot of these functions that start with the word assert.
Essentially, every test method should call one of these methods to verify the results and so the test runner can accumulate all test results and produce a report. Finally, at the end of this file, we are calling unitest.main. This will run all registered tests. Here is what we get when we run this:

If we want to know which tests cases are called you can just add -v as an argument:

As we can see we run our one test case, and got the result that it passes, ie. the condition that we check with assertEqual is true. Congratulations, you’ve just run your first test with Python! Now, let’s see how we can test some functionality that we made. Take a look at this code:

It is a very simple function get_greetings which is just returning the ‘Hello World!’ string. This is how we test it:

Pretty easy, right? We just import function from the file, write a class that inherits unittest.TestCase and verify the result using assertEqual within test method test_get_helloworld. The output looks like this:

4. What is Test Driven Development?

Test Driven Development (TDD) is an evolutionary approach to building and designing software solutions. It is consisting of small cycles in which we are writing a unit test, that will initially fail, and then implementing the minimum amount of code to pass that test. After that code can be refactored to follow some good principles. Refactoring has a safety net, because we wrote the tests already, so we can reshape our solution stress-free.

These three important steps of TDD are easy to remember by using something that I like to call the TDD mantra. It goes like this: Red – Green – Refactor. Red is corresponding with the phase in which we write a test that will fail. Then we implement the code to make the previously written test pass meaning it is – Green. And finally, we refactor our code – and we don’t really have a color for that one. So there you go, TDD Mantra – Red, Green, Refactor.

You might wonder what is the difference between just writing unit tests for your code and TDD? In general, we are using unit tests in both cases. The crucial difference between TDD and traditional testing is the moment in which we are writing the tests. When we are writing code using TDD we first write the tests and then the code itself, and not another way around. The benefit of this approach is that we are minimizing the possibility of forgetting to write tests for some part of the code. Ideally, we end up with the code that is fully tested upfront and solutions that are implemented using TDD usually have 90%-100% of code covered with tests.

Of course, when our code is tested it is less likely that we have a bug in our system. Another important difference is that we are writing small chunks of code to satisfy our test. This way the process itself drives our design and forces us to keep things simple. By using TDD we avoid creating over complicated designs and overengineered systems. Arguably this is the biggest benefit of this approach. When we use it we end up with clearer design and API. This approach also forces you to design classes properly and to follow good code principles like SOLID and DRY. Personally, I find this way of development as a great procrastination killer and a great motivator. It kinda keeps you in the zone.

4. Solving a problem using TDD

Before we proceed let’s examine what kind of problem we are trying to solve. Do you guys like TV show Rick and Morty? I love it. The show follows the adventures of cynical mad scientist Rick Sanchez and his grandson Morty Smith. Rick owns a portal gun and takes Morty to different dimensions/universes. Different versions of these characters inhabit those other dimensions. The Citadel is the place where Ricks and Mortys have formed a society built by their counterparts from an infinite amount of realities. We are in luck because we have a request from The Citadel for one Python module. Here are the user stories:

• A user is able to assign Ricks and Mortys a universe number
• A user is able to add residents to the Citadel
• A user is able to turn all Ricks with assigned Mortys to pickles (watch s03e03)

4.1 First User Story

Ok, let’s start from the first user story and work our TDD magic to the last user story. The first user story tells us that we should have two classes, one for Rick and one for Morty. However, since we are using TDD, we write the unit tests first. We implement Rick test class like this:

 Of course, if we run this test we will get an error saying that Rick class is not existing:

We need to define the class and initialize it through the constructor with the value for the universe:

Now, when we re-run the tests, we get this:

We are following the same pattern for Morty. The first the failing test:

Followed by the implementation:

 And passed test:

4.2 Second User Story

You might notice that this “dance” seems unnatural at first. It takes some time to get used to it, but once you do it is enchanting. You will wonder how you were able to do it another way for years. Ok, so we implemented our first user story. Let’s move onto the second one. This one is pretty easy as well – “A user is able to add residents to the Citadel”. However, if we want to add residents, this means that the Citadel class should have some sort of list or array of residents. Let’s first make a function that will return all residents. We write a test for Citadel class:

This test fails because Citadel implementation doesn’t exist yet. Since this seems a little bit more complicated than previous implementations, we write something like this and try to make our test pass:

As you can see, we defined private field __residents__ and added method get_all_residents which is not doing anything at the moment. This way we can run our test, but it fails again:

So, in order to fix this we have to return that private field through the method:

Re-run the test:

Hooray! Test passes and we are making progress. Still, functionality that satisfies the second user story is not implemented. That is why we write another test so that the complete Citadel test class now looks like this:

Running this will fail because we don’t have add_residents method. So, let’s implement it:

Now when we run the test we get this:

4.3 Third User Story

Awesome! We finished two out of three user stories. However, the last one is the trickiest. Let’s examine it – A user is able to turn all Ricks with assigned Mortys to pickles. We can detect several tasks within this one sentence. We should be able to assign Morty to a Rick, meaning we need to extend both of those classes. We should be able to turn Rick into a pickle, as well. Finally, we should be able to do that for all Ricks in the Citadel with assigned Morties. So, let’s proceed in that order. First, we extend Morty with the is_assigned field. Extended Morty test class looks like this:

The new test of the class will fail. We have to extend the Morty class implementation as well:

Run the test again:

Ok, we are getting closer. Now Rick class should be extended so Morty can be assigned to Rick. We extend the Rick test class:

Test is failing because we are missing morty field in Rick class. Let’s extend Rick:

Ignite tests again:

Nice, now let’s extend tests for assign method, through which we will assign Morty to Rick:

As you can see we are checking two things after Morty is assigned. When we extend Rick class to support these changes it looks like this:

And when we re-run the tests for Rick class:

Don’t give up on me now, we are halfway through the third user story! We need to make Rick “pickable” and turn all Ricks with assigned Mortys in the Citadel into pickles (I never thought I would write down a sentence like this :)). To the Rick test class!

This seems familiar. Test test_has_is_pickle fails because, well, the Rick class still has no field is_pickle. Rick class needs to be extended for that:

Awesome! Now, to the Citadel test class. We add one large test:

Here is the explanation. We create all necessary objects, assign Morty to Rick, add both objects into Citadel and call a method that should turn all Ricks with Mortys into pickles. Cool, let’s reflect that in Citadel class implementation:

Alright, let’s re-run the tests:

Woooohoooo! The tests are passing and we completed our third and final user story!

5. Mock Objects

We need to know how to mock objects in order to fully master TDD. So what would this mean? Well, in Object-Oriented programming mock objects are defined as simulated objects. They mimic the behavior of real objects in controlled ways. Why would anyone do that? Well, the whole point of unit testing is isolating certain functionality (unit) and test just that. We are trying to remove all other dependencies.

Sometimes this can be a database or file system, so using mock objects we don’t step into the world of integration tests. Apart from that, this would mean that we need to take care of the data in the database before and after every test, and that is something we want to avoid. Other times dependency can be just some other class or function. This certainly is not helping us with our goal – the isolation of the unit. That is why we define mock objects.

A mock object should be used in the situations when:

• The real object is slow
• The real object rarely and is difficult to produce artificially
• The real object produces non-deterministic results
• The real object does not yet exist (often the case in TDD )

5.1 Mock Objects in Python

For the purpose of this article, we use the unittest module. To be more specific, we use the Mock class. Essentially, this class is the core class using which we can create stubs in our test methods. After performing an action on these objects, you can check various details which can help you determine if your functionality is behaving properly. The cool trick is that with the objects of Mock class you can mock any function, variable, or object. Check out this test:

Note that you can define and access any ‘field’ of a Mock object which returns another object of a Mock class. Apart from that, you can observe this object as function as well. This means that you can prepare this mock object in any way you want. For example, you can assign some value to the fields of this object. It can be achieved either by assigning this value directly or using configure_mock:

5.2 Mocking Functions in Python

Of course, you can configure mock methods as well. For example, if you want your mock object to have a method that returns certain value, you can do it like this:

You see that we used return_value option. This is pretty simple, however, sometimes we want to test non-happy paths. For example, we want our method to throw an exception. That can be configured like this:

This side_effect option is a really important feature. We can do other things with it too. If you want to return a different value on each call of the function this option got you covered:

Using Mock class we can verify that some method has been called as well:

For that matter, we can check if some function has been called once or multiple times:

This is very useful in situations in which we inject one object into another and then we need to verify that some functions of the first object have been called. Finally, whenever we want to reset some of these internal counters we can use reset_mock. Note that this way we are only resting interactions, but not the configuration itself:

Ok, these are some simple behaviors that we can do with Mock class. However, we still haven’t answered a question of how we can mock classes that are we wrote or classes that come from some third-party library. These things are done using patching.

6. Patching

Patching is quite simple. Basically, we use it to mock parts of your system that shouldn’t be called in our unit tests. In a nutshell, we isolate unit that we want to test. We can mock pretty much anything this way. Let’s we observe this piece of code inside of path.py file:

In our unit tests for current_path function, we don’t really want to call os method because it is not essential for our functionality, and also the return
value might differ between environments. This is some third-party/import we talked about which we want to simulate and remove that particular dependency. This is where the patch method gets into the picture:

Using the patch method, we create a mock object that will handle this call – os_mocked. Note the way we called patch on ‘path.os’. This way we can mock any object from any import. The other way we can write this is:

Pay attention to the additional parameter that we pass into the test method. This additional parameter holds our mock object. In this article, we will use this convention. The same approach we can use if we want to mock context managers. Let’s extend our path.py with the function for reading from a file:

Now, we want to mock open context manager, because we don’t want to really open the file in our unit test. This can take a while and we can load a lot of data in memory. So, we can do something like this:

We use StringIO as a return value of open context manager mock. There’s nothing we haven’t already seen here except the __enter__method.

Patching can be applied to the classes from our system as well. If you remember, in the previous chapter, we developed some funny Rick and Morty classes. One of the functionalities was that we can assign Morty to a Rick. Here is how those classes look like, once again:

As a reminder here is how the test in Rick tests for assignment process looks like:

This might not be the best approach. Namely, we create objects of two different classes in the same test. Some may argue that this is more an integration test than a unit test. Others may say that this is a unit test if we define the whole assignment process as a unit. Both might be right. However, let’s say that we want to isolate functionality in the Rick class more. That would mean that we need to mock the Morty object. This is how we can modify this test to better fit these principles:

In a nutshell, we mock Morty object using the patch concept, and checked all the same things using this mock object.

Another cool thing that we can do with Mock class is to mock just certain aspects of some class, ie. mock partially. Let’s mock just the mentioned assign method in Rick class:

This way you can isolate units even further.

Conclusion

In this article, we went through several concepts. We explored what kind of automated tests exists. We focused on the unit tests since they are the backbone of Test Driven Development, which we also explained. Finally, we implemented one solution using this technique. You might notice that we haven’t done a lot of refactoring because the examples were pretty straightforward. However, we could notice how this way of development is driving our implementation, and how it forces us to write clean and testable code.

Mock objects are an essential step if we want to properly isolate units that we want to test. With that, it is crucial to Test Driven Development. In this article, we got a chance to see how the Mock class from the unittest module works and how to utilize it. We also had a chance to see various scenarios of mocking and how the patch method is used in these situations. We explored the ways to mock third-party classes, context managers, classes that we wrote, and even how to mock some classes partially.

Thank you for reading!