Django KFIMember Tutorial Part IV - Using South to Manage Schema Migration

The future of Django Schema Migrations, but at the current release of 1.6.1, we still have to use South as Third Party Application

The future of Django Schema Migrations, but at the current release of 1.6.1, we still have to use South as a Third Party Application

To be honest, I believe this article will have a short life. Why? It's simply because migrations is a moving target in Django. As of Django 1.6.x, we still don't have migrations feature built-in. But the good news is, Andrew Godwin, the author of South, is currently working on Kickstarter funded project, to bring migrations feature into the core of Django itself. You can read about it usage here, but you must be aware that it's targeted for Django 1.7. As of now, we still have to use South as a third party Django application. And this is what I try to introduce in this articl.

 

What is Schema Migration?

The first time I have a hands-on on schema migration is when I tried Ruby on Rails. Just a glimpse of it, but it's --at that time-- a mind blowing experience for me. The idea of synching database schema with application is truly encouraging : you can be sure that at any given version of our application, it (should) always work with the database. It's because there are migrations code that will gradually change database schema into appropriate state, synched with the application code. Django has this feature also, but as I said previously, through third party Django package called South. But before trying South, let see what will be the case if we don't use a proper migration utility. We will start with creating our first Django application.

Creating our First Django Application

Previously, I already elaborate the differences between Django project and Django application. Here, let's create our first Django application. You can use command line python manage.py startapp <yourappname> to begin creating an app, from within terminal/console at the directory wsgi/kfimember. Because our project name is kfimember, you can't use it again. Try to use the name kfimemberapp. Or, if you use Python Tools for Visual Studio like me, you can right click on kfimember project node, choose Add, and select Django App. You will be presented with Add Django App dialog, and type your application name there.

Create app

This command will create a folder name kfimemberapp consisting of four files:

  1. __init__.py  : an empty Python file indicating that this folder is a Python package
  2. models.py  : contains all your models for this application
  3. tests.py  : a unit test that will be run automatically as part of this Django project unit test 
  4. views.py  : contains all your views for this application

Our next step is to really use this application in our Django project by editing settings.py and add kfimemberapp into INSTALLED_APPS section, as follows:

1
2
3
4
5
6
7
8
9
INSTALLED_APPS = (
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'kfimemberapp',
)

As we don't have yet add any code for our application, testing the application right now would not bring any effect. So, let's add our first models.

Adding Application Models

To understand application models developed for this tutorial series, you may want to review our application requirements in the first article. I am not going to be fully thorough in this step. Depicted below, I will only show enough detail of the application model for us to get started with using South to manage its schema migration. 

Simplistic view of our application model

Simplistic view of our application model

 

We will use the existing Users model and hence, using the above simple view of our model, we only need to add a new SertifikatModal (a.k.a Stock Certificate) model in kfimemberapp/models.py , seen below.

1
2
3
4
5
6
from django.db import models
from django.contrib.auth.models import User
 
class SertifikatModal(models.Model):
    user = models.ForeignKey(User)
    nomor = models.CharField(max_length=30) #a.k.a number

Because of the above simplicity, you may want to refer to a complete list of all Django Field class that you can use in your application model.

"Only two attributes?", you may asked. Yes, as I said previously, we just need the very most basic fields/properties to get started. Later on we can begin detailing its fields. This new model need its own tables in our database. But to no worry, as Django programmer we can left the task of creating this table to our beloved pals, manage.py syncdb. There are great limitations though : it only able to create new tables. It doesn't have the ability to modify existing tables. So, if you plan to add (of course you will) another properties to this models, well, you are out of luck with just a plain manage.py syncdb. For that, we need South.

But let see first how it is done using just plain manage.py syncdb. Open up your terminal/console, and at the path wsgi/kfimember issue this command: manage.py syncdb, as seen below.

Add new table based on new model

Add new table based on new model

Have a look on your PostgreSQL database. You will see that a new table has been created for you. But not only that! You can also instruct Django to automatically create back-end administration page for this new model. Simply create a file kfimemberapp/admin.py with the following Python code:

1
2
3
4
from django.contrib import admin
from kfimemberapp.models import SertifikatModal 
 
admin.site.register(SertifikatModal)

Run your Django server with the command manage.py runserver or simply hit F5 from within Visual Studio, and you will see that SertifikatModal has been added to the list of models in Django admin application. In there, you can do all complete CRUD actions. Cool isn't it?

This will serve as a great starting point for back-end admin feature for all  models in your application

This will serve as a great starting point for back-end administration application for all models in your application

Using the currently Django manage.py syncdb to introduce new table to the database may suffice for the moment. But things goes awkward when you start to modify your existing model (hence, the corresponding table). For example, suppose you want to add a new description field to the existing SertifikatModal model. Running manage.py syncdb again will not modify your related tables. You got to add that field manually to the database table. Yes, manually.

The solution? Let use South for this so called schema migrations problem!

Starting to Use South

There are two ways to use South :
  1. Early in the beginning of application development step, prior any application models being introduced to the system.
  2. After you satisfy with the current application models, decide that it's quite good, and afterward would like to manage its migrations using South.

You can opted to use any ways as you wish, but in my opinion it's better to use South only after a rather stable application models were established. It's rather overkill to use South in the early step of application models development. You can simply drop and recreate your application database before each application testing. Once you satisfied with the application, then you can use any migrations utilities that you wish. I  use this technique in my previous Flask Bio application development tutorial. In there I use Alembic to manage its migrations after the application reach some kind of stable state.

Here, we are going to explore both ways so that you can decide it on your own.

Adding South to Django Project

In your local environment, you can simply issue the command : pip install south to install South into your Python site-packages directory, so that it will available to all your Python application (if it was installed into global Python site-packages) or into specific Python application in their own virtual environment (if you use virtualenv utility).

For remote environment (your hosting provider), as this tutorial assume that you use Openshift, you can simply add this requirements into file setup.py, as shown below.

1
2
3
4
5
6
7
8
9
10
from setuptools import setup
 
setup(name='KFI Member App',
      version='1.0',
      description='Manage membership features in Cooperative organization',
      author='Eko S. Wibowo',
      author_email='swdev.bali@gmail.com',
      url='http://www.python.org/sigs/distutils-sig/',
      install_requires=['Django==1.6.1','psycopg2', 'south'],
     )

Lastly, of course you have to include South into Django INSTALLED_APPS in the file settings.py, as shown below.

1
2
3
4
5
6
7
8
9
10
INSTALLED_APPS = (
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'kfimemberapp',
    'south'
)

 

Preparing South south_migrationhistory Table

Whether you are using South prior any models introduced or afterward, you still have to create South own table to manage its migrations information : south_migrationhistory. This step was done after South requirement has been added to the Django project. The action needed was as simple as issuing the command python manage.py syncdb, which will create south_migrationhistory table. Depicted below, are our current list of tables in the database up until this step.

Check it using \dt

After adding South and running manage.py sycndb

After having this South table created, you can begin the next step of using South into your Django project. Pick the method that suit your current situation. 

Using South Prior Applications Model's Tables Created

If you follow this Django tutorial series as it is, well, what you really need is actually not this step. You can skip this section and go to the next section of "Using South After Creating Applications Model's Tables". It is because you have use manage.py sycndb to create SertifikatModal table. 

If, however, you want to use South prior any application model's tables were created, the steps (including its current application state) are the following:

  1. You have already run python manage.py syncdb command to create Django built-in application's tables.
  2. You have already included South in INSTALLED_APPS in file setings.py and have already run syncdb afterward to create south_migrationhistory table.
  3. You have already created your application models code in the file models.py
  4. Afterward, run python manage.py schemamigration kfimemberapp --initial to prepare South first initial migration code. It will create migrations directory in your application directory,  read the current state of your applications model and create initial Python migration code in migrations directory, to let you create its required database tables later on. Remember, if you use application name other than kfimemberapp, then use it.
  5. The previous step is only creating its initial Python migration code. It didn't actually create those tables for you. To really create it, you can simply use the command : python manage.py migrate kfimemberapp.

The final result is your application model's tables were created, and you can simply modify your application models easily, knowing that South will be able to modify the related tables for you. Or in other word, after running python manage.py migrate kfimemberapp command,  your database already in synched with your model. Below are its screenshot of this command output:

Adding South prior application model's table creation

Adding South prior application model's table creation

Using South After Creating Applications Model's Table

This step is the correct step if you follow this tutorial series as is, that is, you have code your application models in file kfimemberapp/models.py  and have already created the related table(s) into application database using python manage.py syncdb command. To introduce South in this point of application development, the steps are the following:

  1. You have to make sure that your database schema already in synched with your application models and there is no migrations directory below wsgi/kfimember/kfimemberapp. In order to do this, you may want to delete (if any) migrations directory, drop your database (or just delete the tables related to your application, namely all tables with the name prefixes with kfimemberapp_), recreate the database and running python manage.py syncdb command to synched the database with your application model.
  2. Just issue this simple South convert command : python manage.py convert_to_south kfimemberapp.

It will create  migrations directory, creating initial database migrations code inside it, and running fake migrations command. It's fake, because South only pretend that it runs the migration script, although it's not, because the database already synched with the application model . The final result will be the same like our previous method of introducing South prior application model's tables were created. Below is the screenshot of this command output:

Adding South after application model's table were created with syncdb

Adding South after application model's table were created with syncdb

Using South to Manage Automatic Migrations

If the previous steps seem somewhat of a burden, now is the time to make South shine : automatically manage schema migration. That is, you can concentrate modifying your application model and let South take care of managing its database schema synchronization for you.  For example, suppose that you want to add another field to SertifikatModal model, let say jumlah which is Indonesian word for amount that denote how much is the stock certificate hold for this particular transaction.

1
2
3
4
5
6
7
from django.db import models
from django.contrib.auth.models import User
 
class SertifikatModal(models.Model):
    user = models.ForeignKey(User)
    nomor = models.CharField(max_length=30) #a.k.a number
    jumlah = models.FloatField(default = 0) #a.k.a amount

 

Now, how do we use South to properly synched our database? Simple. Just issue the command python manage.py schemamigration kfimemberapp --auto to let South create the migration code for your and afterward, simply issue the command python manage.py migrate kfimemberapp to apply the migration. The result? You don't have to manually touch your database to synch all of its related tables and field. Depicted below is the output of this command.

Using auto migration feature of South

Using auto migration feature of South

Have a check on your database table! You will see that it already have a new amount field inside kfimemberapp_sertifikatmodal table.

Making It Synched with Openshift Database

After having local database schema synched with your application models, if you happened to use Openshift (or any other Cloud computing solution such as Heroku, or a private VPS server) you will have to apply the same series of command into your remote database terminal/console using its SSH connection. Therefore, this step will be the same as in our previous article.

Conclusion

If you have followed this article series until this point, chances are you already know how to properly manage your Django project and its schema migrations using South, although I left out some of South features in this article (such as manual migration). For that, you can read it at South own documentation web here.

As always you can download the code for this article here : kfimember-part-4.zip.

Or, follow its development in Github repo here : kfimember

Stay tuned!





Leave comments

 authimage

Copyright(c) 2014 - PythonBlogs.com
By using this website, you signify your acceptance of Terms and Conditions and Privacy Policy
All rights reserved