Admittedly, nothing in this post is new. Nothing here is hard to find in documentation elsewhere or is pushing any sort of cleverness boundaries whatsoever.
That said, correctly mocking external entities in your unit tests is a bit tricky syntactically, and I have to look up the exact incantations to type every single time they're needed. Thus, I'll make a quick sales pitch about mocking, quickly explain it to the best of my current understanding, and end with a sequence of tricks to be updated as I learn more. In essence, it's my own personal cheat sheet, stored in an accessible location I'm unlikely to misplace.
Also, I would be remiss to not give partial author credit to my homeboys. The importance of mock was first explained to me by Jace Browning, one of the best Python programmers I know. Partial author credit for this post goes to Scott Anderson, who helped read documentation and figure out a bunch of the syntactical specifics.
Python has a library called mock that is basically a malleable override patch you can place on top of any other code. You can point it at a certain function in a certain class and assign arbitrary, instant return values. You can tell it to do certain things based on the parameters it (the mocked function) receives. At first glance this sounds totally insane, and in fact I remember the first time I learned about mock, I was pretty sure it was the most useless library under the sun. As it turns out, I was wrong.
Your unit tests should test your code. Let's say you're making 3rd party calls to Facebook for user information -- this isn't something you want your test suite to actually, physically, do. Better is to construct the outgoing URL and payload in their own functions, then in another separate function get your bits onto the wire. This means your unit tests can rigorously examine the URL construction, payload assembly, and Facebook response parsing all separately.
Take, for example, the following function. At first glance it seems pretty reasonable, but unit testing it is a nightmare.
class FacebookUpdater(object):
def update_user_info_from_facebook(self, user):
"""
Asks Facebook for updated information about a user,
then stores the response.
"""
url = 'https://graph.facebook.com/%s/?access_token=%s' % (user.facebook_uid, user.access_token,)
# A call like this to Facebook would be a GET without any payload,
# but we're in hypothetical land so do me a solid and suspend your disbelief
payload = {
'id': user.id,
'method': 'post_friends_social_security_numbers',
# etc
}
response = requests.post(url, data=payload).json()
user.last_updated = timezone.now()
user.friend_count = response['friend_count']
# etc
That function seems pretty innocent. It makes a quick request and does stuff with the response. But as stated above, it's a real nightmare to test. Instead, consider this chunkier alternative:
class FacebookUpdater(object):
def update_user_info_from_facebook(self, user):
"""
Asks Facebook for updated information about a user,
then stores the response.
"""
# Assemble the goods
url = self.build_update_url(user)
payload = self.build_payload(user)
# Get bits onto the wire
response = self.query_facebook(url, payload)
# Save stuff
user.apply_updates(user, response)
def build_update_url(self, user):
"""
Assembles the Facebook-specific URL for this update
"""
return 'https://graph.facebook.com/%s/?access_token=%s' % (user.facebook_uid, user.access_token,)
def build_payload(self, user):
"""
Compiles all information required for a successful update
"""
return {
'id': user.id,
'method': 'post_friends_social_security_numbers',
# etc
}
def query_facebook(self, url, payload):
"""
Zip it and ship it.
"""
return requests.post(url, data=payload).json()
def apply_updates(self, user, response):
"""
Saves the response from whatever update we're doing in this class
"""
user.last_updated = timezone.now()
user.friend_count = response['friend_count']
# etc
The LOC count has sky rocketed, but that's a good thing because now you're set up unit test each piece of logic. The compartmentalized logic means you can test URL construction and payload assembly without having to actually query Facebook in your test suite. Requiring actual network requests to Facebook in your unit tests could lead to API limit consumption or false negatives in your results.
None of the above is groundbreaking, though its something every new programmer has to come to understand at some point in their career. If the soundness of this design pattern isn't obvious, do a little reading about the Single Responsibility Principle. That Wikipedia article about it mostly at the class level, but it applies to functions all the same.
And now we can finally dive into some actual mock syntax. We broke apart the above function for the purposes of mocking the call to Facebook, so that's where we should begin.
Consider this unit test code:
def test_facebook_update(self):
"""
Verifies that a successful Facebook update is correctly saved
"""
user = User.objects.first()
fb_updater = FacebookUpdater()
with mock.patch.object(FacebookUpdater, 'query_facebook') as mocked_method:
mocked_method.return_value = {'friend_count': 100, 'other_data': 'some_value'}
# Note that specific verification of our URL and payload
# construction should still happen, just not in this
# particular unit test.
fb_updater.update_user_info_from_facebook(user)
self.assertEquals(user.friend_count, 100)
# More assertions go here
Now we've got something pretty cool. Our one function to rule them all got refactored to lean on several smaller functions that each handle just one chunk of the logic. This allowed the particular chunk that involved a 3rd party to be mocked/replaced with an instantaneous result. We A) will have faster tests, B) will be able to test our code offline, or when Facebook is down, C) won't abuse any API limits various 3rd parties may impose, and D) can also force failure states to unit test the portion of our code that is supposed to gracefully handle a failed outbound request.
Maybe we're testing caching logic and after a certain value is cached we expect to be able to make additional requests without certain workhorse functions ever getting called.
def test_caching_layer(self):
self.cache.set(key='marco', value='polo')
with mock.patch.object(SomeClass, 'workhorse_function') as mocked_method:
sc = SomeClass()
sc.get_by_username('marco')
if mocked_method.called:
self.fail("This was supposed to be a cache hit")
requestsAbove, we mocked query_facebook's dictionary response. But what if that function was designed to return the raw requests.Response object instead? (Which, by the way, is a better design. The calling code is likely to care about the response's status code). That mock is a touch more involved:
def test_something(self):
with mock.patch.object(FacebookUpdater, `query_facebook`) as mocked_method:
mocked_response = requests.Response()
mocked_response.status_code = 400
mocked_response._content = {"error": "Your access token is expired"}
mocked_method.return_value = mocked_response
You're now ready to unit test graceful handling of a failed update with Facebook.
It's easy to imagine needing the same mock across 10 different unit tests. Luckily, this can also be done. The following is Django-specific syntax.
from django.test import TestCase
class FacebookUpdateTester(TestCase):
def setUp(self):
self.patch = mock.patch.object(FacebookUpdate, 'function_i_want_to_mock')
self.patch.return_value = True
self.patch.start()
def tearDown(self):
self.patch.stop()
def test_stuff(self):
# Any calls in here to `function_i_want_to_mock` from within
# an instance of the `FacebookUpdate` class will automatically
# use the mock instead of actually executing.
def test_other_stuff(self):
# The same is true here
@propertyThere's a tiny syntactical difference when mocking a descriptor on your classes.
with mock.patch.object(SomeClass, 'a_property_callable', new_callable=mock.PropertyMock) as mocked_prop:
mocked_prop.return_value = True
sc = SomeClass()
sc.a_property_callable # True
Scott Anderson (correctly) wanted me to point that one sneaky underlying truth about mocks is that you're statically freezing reality. You're lying to your code. That Facebook response you hard-coded into the test is good as long as Facebook doesn't change anything, but when they do your test will quickly turn into a false-positive. That's why, as you can probably guess, final integration testing is the truest icing on your test suite cake.