Modularization of Python Code
(Image taken from here)
I visioned that this article will be your first article in how to properly start coding in Python. And lets be brief in this: you are going to understand what does it means by the following three ways of organizing Python code:
All, through the eye of a Python programmer.
Those three terms are your fundamental key in understanding how to properly organize your Python application.
Great. Lets get started!
Why do I have to organize my application code?
Well, actually you don't have to.
In the context of whether it's a mandatory or not, well, it is not. Your application code can live peacefully in a single file, like our single file version of Flask Biography application or it can be organized into a much more modular application, like our Blueprint version of Flask Biography application. Certainly the reasons that drive a better code organization are:
This term highly related to the fact that your unit of application code are easily identified into distinguishable part.
As the unit of application code already easily identified, you will then will be very easy in extending the current application code
- Self documenting
Now that it's highly modular, using a proper code convention such as PEP 8, you will then will be able to create a code that clear to understand.
- Easier to communicate
With self documenting code, there come a whole lot of easier way in communicating your code to others. This is particularly important in a team work and still also applied if you are working alone. And why is that so? Because if you do encounter difficult task in your code development, you can easily explained your problem to others by exposing your high quality code. People on the internet will have a better chance to solved your problem if your code is easy to understand.
Be advise though, that the above four reasons are coming from the many reasons that one can come up by thinking the advantages of properly organizing your code. You will come up with more reasons once you develop your own application.
Study Case: A Glimpse of Our Next Productivity Application
This article in some way is a continuation of my previous article that specifically discuss how to use PyWin32 package to revive MS Agent. In preparing its overall application design, I realize that I haven't add another article in Python for Beginner series. Therefore, I think it's a perfect time for a beginner article that revisit how to properly organize a Python application, by using our yet-to-be productivity application, code named : "Northed"
For sake of simplicity, the thing that will be our concern right now for Northed application functionality is its ability to manage and display alarm in certain interval: whether a one time alarm or repeated alarm. That's all. Nothing more. Although, got to say, I really tempted to begin with a full closure of the overall application features. But I believe it will only clutter up the discussion with unnecessary complexity. Another point to note is, for a working draft of what this Northed application be like, you can try to run the current MS Agent application that will stay silent in the system tray and regularly announce the time in hourly bases. I have use it in more than a month now, and got several ideas how to make the application more useful for my daily need. And that's why I initiate a new article series that specifically talk about how to design and implement this productivity application.
Also, we will not going to implement the real code for this alarm feature. Only the mockup suit us just fine for the purpose of this article.
Step 1: Alarm Feature As A Python Module
In its simplest, this alarm features must reside in a Python
module, which is just a regular file having .py extension. Inside this
module you will have to implement alarm features using all sort of
function and other related data structures. In this first step, I would
like to present the module in its functions signature only. I hope the
function name intuitive enough for you to guess what their roles are (I
also accompany it with some comment). Therefore, the function body will
only have a
pass statement, which does nothing: it only act as a placeholder for future code.
Great. Below is the content of
alarm.py using intuitive functions name:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41
def get_alarms(): """ return list of alarms """ pass def add_alarm(id, title, time_of_occurrence, repeat_days = ''): """ add new alarm to the list """ pass def update_alarm(alarm_id, title=None, time_of_occurrence=None, repeat_days=None): """ update alarm by its id, with optionally new data """ pass def remove_alarm(alarm_id): """ remove alarm by its id """ pass def clear_alarms(): """ remove all alarms """ pass def get_count(): """ return count of alarms """ pass def get_alarm(id): """ get an alarm by its id """ pass
The next question is, how do you use all of the above functions from another Python module (*.py file) ? Here is simplest way to do that:
1 2 3 4 5 6 7 8 9
import alarm alarm.clear_alarms() alarm.add_alarm(id=1, title='Take daughter to science club', time_of_occurrence='2PM', repeat_days='wed') alarm.add_alarm(id=2, title='Capoeira regular training', time_of_occurrence='4.30PM', repeat_days='mon,fri') alarm.update_alarm(alarm_id=1,title='Take daughter+son to science club') alarm1 = alarm.get_alarm(1) alarm.remove_alarm(2) print(alarm.get_count())
The above code was saved in the file
main.py located in the same directory as the file
alarm.py. Have a look at how I simply call the statement
import alarm, and then you can call all functions within that module using the dot operator: e.g.,
alarm.remove_alarm(2). You can also shortened how you call all functions within
alarm module this way:
1 2 3 4 5 6 7 8 9
from alarm import * clear_alarms() add_alarm(id=1, title='Take daughter to science club', time_of_occurrence='2PM', repeat_days='wed') add_alarm(id=2, title='Capoeira regular training', time_of_occurrence='4.30PM', repeat_days='mon,fri') update_alarm(alarm_id=1,title='Take daughter+son to science club') alarm1 = get_alarm(1) remove_alarm(2) print(get_count())
Overall, the above code is not a good habit. Things can become confusing when you already import more than one module: you simply can't relate which functions belong to which modules. Although, there are cases when this simplification is needed. Decide it by your own, whether you will not get confused when using this import style.
Finally, you can import just the method that you need. Have a look at this code:
1 2 3 4 5 6 7 8 9
from alarm import clear_alarms, add_alarm, update_alarm, get_alarm, remove_alarm, get_count clear_alarms() add_alarm(id=1, title='Take daughter to science club', time_of_occurrence='2PM', repeat_days='wed') add_alarm(id=2, title='Capoeira regular training', time_of_occurrence='4.30PM', repeat_days='mon,fri') update_alarm(alarm_id=1,title='Take daughter+son to science club') alarm1 = get_alarm(1) remove_alarm(2) print(get_count())
With the above import style, there will be a good reference for you (or other that read your code) from which modules are certain functions coming from.
Step 2: Make Alarm Module As Part of A Package
Although in this article I present to you only a single alarm module, you don't suppose that the final product will only composed with one file module, right? If, say, you have 15 modules (meaning 15 *.py files), while all modules can be grouped by its different purpose (e.g. GUI layout, database access, models, scheduling, etc), you will need another way to organize your python application: package.
Package is implemented in Python as directory containing a single
file. Within this directory you can put any related python modules
(*.py files) and then import it in another module, just the way you
alarm feature previously. Have a look below at our
application without special consideration of its packages organization.
Notice how I add several hypothetical modules to complete our example:
Without clear package organization, it's not easy to understand your application structure
I bet you will not be able to quickly distinguish the groups where each modules belong to. The structure above only gives you little to no information regarding where each modules belongs to. Compare the above structure with below structure, that already organize its modules using packages:
Using a good package organization, even new programmer who read your code will be able to quickly understand your application structure
Now you can clearly see what are groups of packages that composed your application. Somehow, you even can tell that main.py will responsible in starting up your application. When you want to dig more to the application structure, you can expand the packages to be as follows:
In expanded state, you can have a full closure of your application structure
You maybe thinking, "So? What so special about it? Creating directory is a regular way of organizing files". Yes, it is, if it's not for that special
Python will treat
__init__.py file differently. Each time the package was imported, Python will execute
__init__.py, making it as initializer function for that package. For example, when importing
alarm module from
package, we can make sure --hypothetically-- that any alarm that
haven't been started and fall within near future occurrence time, to be
started and ready to run. This behaviour as far as I know is specific
for Python programming only. You won't find this kind of behaviour of
treating a file as initializer code for a package in other programming
languages (at least that I know of). To be honest, this is one simple
thing that make me so intrigued in trying my best in mastering Python...
Step 3: Turn Alarm Into Class
Actually, this topic can be spin off into its own article: discussing
mainly about Object Oriented Programming (OOP) in Python. I've even
began to think that we need to revisit this OOP topic in subsequent
article in this site. Also, although the philosophical concept of OOP is so daunting, its implementation is rather easy. It gets even easier in Python. Have a look at our
alarm.py code above. To convert the above module into class, you will have to do three steps:
classstatement in the beginning of alarm code
- Increase indentation level for all existing alarm code
- For all functions (now we call them methods, as now we are talking in OOP term), add
selfas first argument
- Move all variables (now we call them properties) needed by alarm code into a new method --intuitively-- named
Below is the result of turning our Alarm code (still saved in the previous
alarm.py) into valid Python class:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45
class Alarm(object): def __init__(self): pass def get_alarms(self): """ return list of alarms """ pass def add_alarm(self, id, title, time_of_occurrence, repeat_days = ''): """ add new alarm to the list """ pass def update_alarm(self, alarm_id, title=None, time_of_occurrence=None, repeat_days=None): """ update alarm by its id, with optionally new data """ pass def remove_alarm(self, alarm_id): """ remove alarm by its id """ pass def clear_alarms(self): """ remove all alarms """ pass def get_count(self): """ return count of alarms """ pass def get_alarm(self, id): """ get an alarm by its id """ pass
Pretty easy, right? Now, to use this new Alarm class, we do it this way:
1 2 3 4 5 6 7 8 9 10
import alarm main_alarm = alarm.Alarm() main_alarm.clear_alarms() main_alarm.add_alarm(id=1, title='Take daughter to science club', time_of_occurrence='2PM', repeat_days='wed') main_alarm.add_alarm(id=2, title='Capoeira regular training', time_of_occurrence='4.30PM', repeat_days='mon,fri') main_alarm.update_alarm(alarm_id=1,title='Take daughter+son to science club') alarm1 = main_alarm.get_alarm(1) main_alarm.remove_alarm(2) print(main_alarm.get_count())
Notice how we still have to import the
alarm module first, and then instantiate a new object using the new
Alarm class, into variable named
main_alarm. After that, you can access all the methods and properties available.
You maybe wondering, "It's not that different. What is the advantage of using class instead of just module?". Great, lets be brief on this, below I give some advantages when using class instead of plain module:
You do realize that we store the instantiation of
main_alarm, right? This is the feature that only existed in OOP: where you can encapsulate methods and properties into a class and then you can have multiple objects which is instantiation of that single class. Hence, you will be able to have another
Alarmclass instantiation into unlimited number of objects such as
- Inheritance: you can define a new class using existing class, adding new important features to the new class.
- In class, you will have a better way to relate properties/variables to methods/functions, than it is in module.
Bonus Step 4: Implement Alarm Feature Using Dict
Actually, I would love to stop at Step 3 when we discuss how to turn Alarm into a class. But, for those that curious enough, I add another step here: implementing Alarm using Python dict. Have a look at this code (in plain module version):
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44
__author__ = 'Eko Wibowo' alarms = dict() def get_alarms(): return alarms def add_alarm(id, title, time_of_occurrence, repeat_days = ''): alarm = dict() alarm['id'] = id alarm['title'] = title alarm['time_of_occurence'] = time_of_occurrence alarm['repeat_days'] = repeat_days alarms[id] = alarm def update_alarm(alarm_id, title=None, time_of_occurrence=None, repeat_days=None): new_alarm = dict() existing_alarm = alarms[alarm_id] existing_alarm['title'] = title if not title != None else title existing_alarm['time_of_occurrence'] = title if not title != None else title existing_alarm['title'] = title if not title != None else title existing_alarm['title'] = title if not title != None else title new_alarm['title'] = title new_alarm['time_of_occurence'] = time_of_occurrence new_alarm['repeat_days'] = repeat_days alarms[alarm_id] = new_alarm def remove_alarm(alarm_id): alarms.pop(alarm_id) def clear_alarms(): alarms.clear() def get_count(): return len(alarms) def get_alarm(id): result = None if id in alarms: result = alarms[id] return result
Dictionary is a data structure where you an associate a key (in this case an integer) to value (another dictionary object to hold alarm data). How do we use this new module?
At least there are two approaches to test a new module: you can directly
use it inside the module itself or create a separate unit test for it
(for a rather complete unit testing article, you can refer to this article
in this site). I choose the later approach as it keep the module clean.
Therefore, below is the unit test module that clearly demonstrate how
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
import unittest import alarm class TestAlarm(unittest.TestCase): def test_alarm_crud(self): alarm.clear_alarms() self.assertFalse(alarm.get_alarms()) alarm.add_alarm(id=1, title='Take daughter to science club', time_of_occurrence='2PM', repeat_days='wed') self.assertTrue(alarm.get_alarms()) #or self.assertEqual(alarm.get_count(), 1) alarm.add_alarm(id=2, title='Capoeira regular training', time_of_occurrence='4.30PM', repeat_days='mon,fri') self.assertEqual(alarm.get_count(), 2) alarm.update_alarm(alarm_id=1,title='Take daughter+son to science club') alarm1 = alarm.get_alarm(1) self.assertEqual(alarm1['title'], 'Take daughter+son to science club') alarm.remove_alarm(2) self.assertEqual(alarm.get_count(), 1)
If you have successfully run the unit test code either in PyCharm or any other Python IDE, ... congratulations! That is a wonderful progress you have there.
I am serious.
I intended the discussion in this article to serve as the first "official" Python introductory material in this site. Although when publishing article in this site I always take beginner level into account, I never really dedicated introductory article for it. Therefore, I do hope that I aim at the right spot when discussing ways you can organize your Python code.
You can follow the github repository of this article in the following github repo: pythonthusiast/python-for-beginner
Stay tuned for my next article!