Part 3: [Addendum] How Python Organize Your Application Code

 

Modularization of Python Code (Image taken from here)

Modularization of Python Code
(Image taken from here)

I visioned that this article will be your first article in how to properly start coding in Python. And lets be brief in this: you are going to understand what does it means by the following three ways of organizing Python code:

  1. Module
  2. Package
  3. Class

All, through the eye of a Python programmer.

Those three terms are your fundamental key in understanding how to properly organize your Python application.

Great. Lets get started!

 

Why do I have to organize my application code?

Well, actually you don't have to. Smile 

In the context of whether it's a mandatory or not, well, it is not.  Your application code can live peacefully in a single file, like our single file version of Flask Biography application or it can be organized into a much more modular application, like our Blueprint version of Flask Biography application. Certainly the reasons that drive a better code organization are:

  1. Modularity
    This term highly related to the fact that your unit of application code are easily identified into distinguishable part. 
  2. Extensibility
    As the unit of application code already easily identified, you will then will be very easy in extending the current application code
  3. Self documenting
    Now that it's highly modular, using a proper code convention such as PEP 8, you will then will be able to create a code that clear to understand.
  4. Easier to communicate
    With self documenting code, there come a whole lot of easier way in communicating your code to others. This is particularly important in a team work and still also applied if you are working alone. And why is that so? Because if you do encounter difficult task in your code development, you can easily explained your problem to others by exposing your high quality code. People on the internet will have a better chance to solved your problem if your code is easy to understand.

Be advise though, that the above four reasons are coming from the many reasons that one can come up by thinking the advantages of properly organizing your code. You will come up with more reasons once you develop your own application.

Study Case: A Glimpse of Our Next Productivity Application

This article in some way is a continuation of my previous article that specifically discuss how to use PyWin32 package to revive MS Agent. In preparing its overall application design, I realize that I haven't add another article in Python for Beginner series. Therefore, I think it's a perfect time for a beginner article that revisit how to properly organize a Python application, by using our yet-to-be productivity application, code named : "Northed"

For sake of simplicity, the thing that will be our concern right now for Northed application functionality is its ability to manage and display alarm in certain interval: whether a one time alarm or repeated alarm. That's all. Nothing more. Although, got to say, I really tempted to begin with a full closure of the overall application features. But I believe it will only clutter up the discussion with unnecessary complexity. Another point to note is, for a working draft of what this Northed application be like, you can try to run the current MS Agent application that will stay silent in the system tray and regularly announce the time in hourly bases. I have use it in more than a month now, and got several ideas how to make the application more useful for my daily need. And that's why I initiate a new article series that specifically talk about how to design and implement this productivity application.

Also, we will not going to implement the real code for this alarm feature. Only the mockup suit us just fine for the purpose of this article.

Step 1: Alarm Feature As A Python Module

In its simplest, this alarm features must reside in a Python module, which is just a regular file having .py extension. Inside this module you will have to implement alarm features using all sort of function and other related data structures. In this first step, I would like to present the module in its functions signature only. I hope the function name intuitive enough for you to guess what their roles are (I also accompany it with some comment). Therefore, the function body will only have a pass statement, which does nothing: it only act as a placeholder for future code.

Great. Below is the content of alarm.py using intuitive functions name:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
def get_alarms():
    """
    return list of alarms
    """
    pass
 
def add_alarm(id, title, time_of_occurrence, repeat_days = ''):
    """
    add new alarm to the list
    """
    pass
 
def update_alarm(alarm_id, title=None, time_of_occurrence=None, repeat_days=None):
    """
    update alarm by its id, with optionally new data
    """
    pass
 
def remove_alarm(alarm_id):
    """
    remove alarm by its id
    """
    pass
 
def clear_alarms():
    """
    remove all alarms
    """
    pass
 
def get_count():
    """
    return count of alarms
    """
    pass
 
def get_alarm(id):
    """
    get an alarm by its id
    """
    pass

The next question is, how do you use all of the above functions from another Python module (*.py file) ? Here is simplest way to do that:

1
2
3
4
5
6
7
8
9
import alarm
 
alarm.clear_alarms()
alarm.add_alarm(id=1, title='Take daughter to science club', time_of_occurrence='2PM', repeat_days='wed')
alarm.add_alarm(id=2, title='Capoeira regular training', time_of_occurrence='4.30PM', repeat_days='mon,fri')
alarm.update_alarm(alarm_id=1,title='Take daughter+son to science club')
alarm1 = alarm.get_alarm(1)
alarm.remove_alarm(2)
print(alarm.get_count())

The above code was saved in the file main.py located in the same directory as the file alarm.py. Have a look at how I simply call the statement import alarm, and then you can call all functions within that module using the dot operator: e.g., alarm.remove_alarm(2). You can also shortened how you call all functions within alarm module this way:

1
2
3
4
5
6
7
8
9
from alarm import *
 
clear_alarms()
add_alarm(id=1, title='Take daughter to science club', time_of_occurrence='2PM', repeat_days='wed')
add_alarm(id=2, title='Capoeira regular training', time_of_occurrence='4.30PM', repeat_days='mon,fri')
update_alarm(alarm_id=1,title='Take daughter+son to science club')
alarm1 = get_alarm(1)
remove_alarm(2)
print(get_count())

Overall, the above code is not a good habit. Things can become confusing when you already import more than one module: you simply can't relate which functions belong to which modules. Although, there are cases when this simplification is needed. Decide it by your own, whether you will not get confused when using this import style. 

Finally, you can import just the method that you need. Have a look at this code:

1
2
3
4
5
6
7
8
9
from alarm import clear_alarms, add_alarm, update_alarm, get_alarm, remove_alarm, get_count
 
clear_alarms()
add_alarm(id=1, title='Take daughter to science club', time_of_occurrence='2PM', repeat_days='wed')
add_alarm(id=2, title='Capoeira regular training', time_of_occurrence='4.30PM', repeat_days='mon,fri')
update_alarm(alarm_id=1,title='Take daughter+son to science club')
alarm1 = get_alarm(1)
remove_alarm(2)
print(get_count())

With the above import style, there will be a good reference for you (or other that read your code) from which modules are certain functions coming from.

 

Step 2: Make Alarm Module As Part of A Package

Although in this article I present to you only a single alarm module, you don't suppose that the final product will only composed with one file module, right? If, say, you have 15 modules (meaning 15 *.py files), while all modules can be grouped by its different purpose (e.g. GUI layout, database access, models, scheduling, etc), you will need another way to organize your python application: package.

Package is implemented in Python as directory containing a single __init__.py file. Within this directory you can put any related python modules (*.py files) and then import it in another module, just the way you imported alarm feature previously. Have a look below at our application without special consideration of its packages organization. Notice how I add several hypothetical modules to complete our example:

 
Without clear package organization, it's not easy to understand your application structure

Without clear package organization, it's not easy to understand your application structure

I bet you will not be able to quickly distinguish the groups where each modules belong to. The structure above only gives you little to no information regarding where each modules belongs to. Compare the above structure with below structure, that already organize its modules using packages:

Using a good package organization, even new programmer who read your code will be able to quickly understand your application structure

Using a good package organization, even new programmer who read your code will be able to quickly understand your application structure

Now you can clearly see what are groups of packages that composed your application. Somehow, you even can tell that main.py will responsible in starting up your application. When you want to dig more to the application structure, you can expand the packages to be as follows:

In expanded state, you can have a full closure of your application structure

In expanded state, you can have a full closure of your application structure

 

You maybe thinking, "So? What so special about it? Creating directory is a regular way of organizing files". Yes, it is, if it's not for that special __init__.py file.

Python will treat __init__.py file differently. Each time the package was imported, Python will execute __init__.py, making it as initializer function for that package. For example, when importing alarm module from scheduling package, we can make sure --hypothetically-- that any alarm that haven't been started and fall within near future occurrence time, to be started and ready to run. This behaviour as far as I know is specific for Python programming only. You won't find this kind of behaviour of treating a file as initializer code for a package in other programming languages (at least that I know of). To be honest, this is one simple thing that make me so intrigued in trying my best in mastering Python...

Step 3: Turn Alarm Into Class

Actually, this topic can be spin off into its own article: discussing mainly about Object Oriented Programming (OOP) in Python. I've even began to think that we need to revisit this OOP topic in subsequent article in this site. Also, although the philosophical concept of OOP is so daunting, its implementation is rather easy. It gets even easier in Python. Have a look at our alarm.py code above. To convert the above module into class, you will have to do three steps:

  1. Add class statement in the beginning of alarm code
  2. Increase indentation level for all existing alarm code
  3. For all functions (now we call them methods, as now we are talking in OOP term), add self as first argument
  4. Move all variables (now we call them properties) needed by alarm code into a new method --intuitively-- named __init__.

Below is the result of turning our Alarm code (still saved in the previous alarm.py) into valid Python class:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
class Alarm(object):
    def __init__(self):
        pass
    
    def get_alarms(self):
        """
        return list of alarms
        """
        pass
 
    def add_alarm(self, id, title, time_of_occurrence, repeat_days = ''):
        """
        add new alarm to the list
        """
        pass
 
    def update_alarm(self, alarm_id, title=None, time_of_occurrence=None, repeat_days=None):
        """
        update alarm by its id, with optionally new data
        """
        pass
 
    def remove_alarm(self, alarm_id):
        """
        remove alarm by its id
        """
        pass
 
    def clear_alarms(self):
        """
        remove all alarms
        """
        pass
 
    def get_count(self):
        """
        return count of alarms
        """
        pass
 
    def get_alarm(self, id):
        """
        get an alarm by its id
        """
        pass

Pretty easy, right? Now, to use this new Alarm class, we do it this way:

1
2
3
4
5
6
7
8
9
10
import alarm
 
main_alarm = alarm.Alarm()
main_alarm.clear_alarms()
main_alarm.add_alarm(id=1, title='Take daughter to science club', time_of_occurrence='2PM', repeat_days='wed')
main_alarm.add_alarm(id=2, title='Capoeira regular training', time_of_occurrence='4.30PM', repeat_days='mon,fri')
main_alarm.update_alarm(alarm_id=1,title='Take daughter+son to science club')
alarm1 = main_alarm.get_alarm(1)
main_alarm.remove_alarm(2)
print(main_alarm.get_count())

Notice how we still have to import the alarm module first, and then instantiate a new object using the new Alarm class, into variable named main_alarm. After that, you can access all the methods and properties available.

You maybe wondering, "It's not that different. What is the advantage of using class instead of just module?". Great, lets be brief on this, below I give some advantages when using class instead of plain module:

  1. You do realize that we store the instantiation of Alarm class into main_alarm, right? This is the feature that only existed in OOP: where you can encapsulate methods and properties into a class and then you can have multiple objects which is instantiation of that single class. Hence, you will  be able to have another Alarm class instantiation into unlimited number of objects such as secondary_alarm, support_alarm, etc.
  2. Inheritance: you can define a new class using existing class, adding new important features to the new class. 
  3. In class, you will have a better way to relate properties/variables to methods/functions, than it is in module.

Bonus Step 4: Implement Alarm Feature Using Dict

Actually, I would love to stop at Step 3 when we discuss how to turn Alarm into a class. But, for those that curious enough, I add another step here: implementing Alarm using Python dict.  Have a look at this code (in plain module version):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
__author__ = 'Eko Wibowo'
 
alarms = dict()
def get_alarms():
    return alarms
 
def add_alarm(id, title, time_of_occurrence, repeat_days = ''):
    alarm = dict()
    alarm['id'] = id
    alarm['title'] = title
    alarm['time_of_occurence'] = time_of_occurrence
    alarm['repeat_days'] = repeat_days
 
    alarms[id] = alarm
 
def update_alarm(alarm_id, title=None, time_of_occurrence=None, repeat_days=None):
    new_alarm = dict()
    existing_alarm = alarms[alarm_id]
    existing_alarm['title'] = title if not title != None else title
    existing_alarm['time_of_occurrence'] = title if not title != None else title
    existing_alarm['title'] = title if not title != None else title
    existing_alarm['title'] = title if not title != None else title
 
    new_alarm['title'] = title
    new_alarm['time_of_occurence'] = time_of_occurrence
    new_alarm['repeat_days'] = repeat_days
    alarms[alarm_id] = new_alarm
 
def remove_alarm(alarm_id):
    alarms.pop(alarm_id)
 
def clear_alarms():
    alarms.clear()
 
 
def get_count():
    return len(alarms)
 
 
def get_alarm(id):
    result = None
    if id in alarms:
        result = alarms[id]
    return result

 

Dictionary is a data structure where you an associate a key (in this case an integer) to value (another dictionary object to hold alarm data). How do we use this new module? 

At least there are two approaches to test a new module: you can directly use it inside the module itself or create a separate unit test for it (for a rather complete unit testing article, you can refer to this article in this site). I choose the later approach as it keep the module clean. Therefore, below is the unit test module that clearly demonstrate how to use  alarm module:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
import unittest
import alarm
 
class TestAlarm(unittest.TestCase):
    def test_alarm_crud(self):
        alarm.clear_alarms()
        self.assertFalse(alarm.get_alarms())
    
        alarm.add_alarm(id=1, title='Take daughter to science club', time_of_occurrence='2PM', repeat_days='wed')
        self.assertTrue(alarm.get_alarms()) #or self.assertEqual(alarm.get_count(), 1)
    
        alarm.add_alarm(id=2, title='Capoeira regular training', time_of_occurrence='4.30PM', repeat_days='mon,fri')
        self.assertEqual(alarm.get_count(), 2)
    
        alarm.update_alarm(alarm_id=1,title='Take daughter+son to science club')
        alarm1 = alarm.get_alarm(1)
        self.assertEqual(alarm1['title'], 'Take daughter+son to science club')
    
        alarm.remove_alarm(2)
        self.assertEqual(alarm.get_count(), 1)

If you have successfully run the unit test code either in PyCharm or any other Python IDE, ... congratulations! That is a wonderful progress you have there.

I am serious.

What's Next?

I intended the discussion in this article to serve as the first "official" Python introductory material in this site. Although when publishing article in this site I always take beginner level into account, I never really dedicated introductory article for it. Therefore, I do hope that I aim at the right spot when discussing ways you can organize your Python code.

You can follow the github repository of this article in the following github repo: pythonthusiast/python-for-beginner

Stay tuned for my next article!




Leave comments

authimage
  • Hi @all,

    @james : Thank you for your honest assessment on this article. I'll try better in my next article. It's kinda hard to satisfy all different level of users. I got to focus on totally beginner in Python first. Please read this article from beginner point of view, not expert one as yourself ;)

    @alex : if you look closer, you can see that the naming on methods were because it was previously live inside a module. But that's kinda important too! I'll revise it and mention you on it! Thanks for the If title. I got to elaborate the code here, but surely I will add its abbreviation.

    @jeff: Thank for your feedback on this! I do thinking to create a much more complete Unittest, but then again.. it will obscured the simple fact of how to properly (meaning, sequentially) use this class. Actually, you can begin by elaborate more on the things that wrongs. It won't be infinite, right? ;)

    • Eko S. Wibowo
  • stop writing trash like this you dumb dumb

    • James
  • I think your naming conventions are not very user-friendly and can be improved. A typical scenario is for you to type something like `alarm.add_alarm()`.

    If the class is called `Alarm`, then the function names could be `add`, `remove` and `update` - it is obvious from the context that you're doing that with an alarm, not with something else.

    What is happening here? `existing_alarm['title'] = title if not title != None else title`, why not just write `if title` instead of `if title != None`?

    • Alex
  • Please don't teach beginners how to write bad unittests. Your test_alarm_crud is wrong on so many levels I don't even know where to start.

    • Jeff

Copyright(c) 2014 - PythonBlogs.com
By using this website, you signify your acceptance of Terms and Conditions and Privacy Policy
All rights reserved