The future of Django Schema Migrations, but at the current release of 1.6.1, we still have to use South as a Third Party Application
To be honest, I believe this article will have a short life. Why? It's simply because migrations is a moving target in Django. As of Django 1.6.x, we still don't have migrations feature built-in. But the good news is, Andrew Godwin, the author of South, is currently working on Kickstarter funded project, to bring migrations feature into the core of Django itself. You can read about it usage here, but you must be aware that it's targeted for Django 1.7. As of now, we still have to use South as a third party Django application. And this is what I try to introduce in this articl.
What is Schema Migration?
The first time I have a hands-on on schema migration is when I tried Ruby on Rails. Just a glimpse of it, but it's --at that time-- a mind blowing experience for me. The idea of synching database schema with application is truly encouraging : you can be sure that at any given version of our application, it (should) always work with the database. It's because there are migrations code that will gradually change database schema into appropriate state, synched with the application code. Django has this feature also, but as I said previously, through third party Django package called South. But before trying South, let see what will be the case if we don't use a proper migration utility. We will start with creating our first Django application.
Creating our First Django Application
Previously, I already elaborate the differences between Django project and Django application. Here, let's create our first Django application. You can use command line
python manage.py startapp <yourappname> to begin creating an app, from within terminal/console at the directory
wsgi/kfimember. Because our project name is
kfimember, you can't use it again. Try to use the name
kfimemberapp. Or, if you use Python Tools for Visual Studio like me, you can right click on
kfimember project node, choose Add, and select Django App. You will be presented with Add Django App dialog, and type your application name there.
This command will create a folder name
kfimemberapp consisting of four files:
__init__.py: an empty Python file indicating that this folder is a Python package
models.py: contains all your models for this application
tests.py: a unit test that will be run automatically as part of this Django project unit test
views.py: contains all your views for this application
Our next step is to really use this application in our Django project by editing
settings.py and add
INSTALLED_APPS section, as follows:
1 2 3 4 5 6 7 8 9
INSTALLED_APPS = ( 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'kfimemberapp', )
As we don't have yet add any code for our application, testing the application right now would not bring any effect. So, let's add our first models.
Adding Application Models
To understand application models developed for this tutorial series, you may want to review our application requirements in the first article. I am not going to be fully thorough in this step. Depicted below, I will only show enough detail of the application model for us to get started with using South to manage its schema migration.
Simplistic view of our application model
We will use the existing Users model and hence, using the above simple view of our model, we only need to add a new
SertifikatModal (a.k.a Stock Certificate) model in
kfimemberapp/models.py , seen below.
1 2 3 4 5 6
from django.db import models from django.contrib.auth.models import User class SertifikatModal(models.Model): user = models.ForeignKey(User) nomor = models.CharField(max_length=30) #a.k.a number
Because of the above simplicity, you may want to refer to a complete list of all Django Field class that you can use in your application model.
"Only two attributes?", you may asked. Yes, as I said previously, we just need the very most basic fields/properties to get started. Later on we can begin detailing its fields. This new model need its own tables in our database. But to no worry, as Django programmer we can left the task of creating this table to our beloved pals,
manage.py syncdb. There are great limitations though : it only able to create new tables. It doesn't have the ability to modify existing tables. So, if you plan to add (of course you will) another properties to this models, well, you are out of luck with just a plain
manage.py syncdb. For that, we need South.
But let see first how it is done using just plain
manage.py syncdb. Open up your terminal/console, and at the path
wsgi/kfimember issue this command:
manage.py syncdb, as seen below.
Add new table based on new model
Have a look on your PostgreSQL database. You will see that a new table has been created for you. But not only that! You can also instruct Django to automatically create back-end administration page for this new model. Simply create a file
kfimemberapp/admin.py with the following Python code:
1 2 3 4
from django.contrib import admin from kfimemberapp.models import SertifikatModal admin.site.register(SertifikatModal)
Run your Django server with the command
or simply hit F5 from within Visual Studio, and you will see that
SertifikatModal has been added to the list of models in Django admin application. In there, you can do all complete CRUD actions. Cool isn't it?
This will serve as a great starting point for back-end administration application for all models in your application
Using the currently Django
manage.py syncdb to introduce new table to the database may suffice for the moment. But things goes awkward when you start to modify your existing model (hence, the corresponding table). For example, suppose you want to add a new
description field to the existing
SertifikatModal model. Running
manage.py syncdb again will not modify your related tables. You got to add that field manually to the database table. Yes, manually.
The solution? Let use South for this so called schema migrations problem!
Starting to Use South
- Early in the beginning of application development step, prior any application models being introduced to the system.
- After you satisfy with the current application models, decide that it's quite good, and afterward would like to manage its migrations using South.
You can opted to use any ways as you wish, but in my opinion it's better to use South only after a rather stable application models were established. It's rather overkill to use South in the early step of application models development. You can simply drop and recreate your application database before each application testing. Once you satisfied with the application, then you can use any migrations utilities that you wish. I use this technique in my previous Flask Bio application development tutorial. In there I use Alembic to manage its migrations after the application reach some kind of stable state.
Here, we are going to explore both ways so that you can decide it on your own.
Adding South to Django Project
In your local environment, you can simply issue the command :
pip install south to install South into your Python
site-packages directory, so that it will available to all your Python application (if it was installed into global Python
site-packages) or into specific Python application in their own virtual environment (if you use virtualenv utility).
For remote environment (your hosting provider), as this tutorial assume that you use Openshift, you can simply add this requirements into file
setup.py, as shown below.
1 2 3 4 5 6 7 8 9 10
from setuptools import setup setup(name='KFI Member App', version='1.0', description='Manage membership features in Cooperative organization', author='Eko S. Wibowo', email@example.com', url='http://www.python.org/sigs/distutils-sig/', install_requires=['Django==1.6.1','psycopg2', 'south'], )
Lastly, of course you have to include South into Django
INSTALLED_APPS in the file
settings.py, as shown below.
1 2 3 4 5 6 7 8 9 10
INSTALLED_APPS = ( 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'kfimemberapp', 'south' )
Preparing South south_migrationhistory Table
Whether you are using South prior any models introduced or afterward, you still have to create South own table to manage its migrations information :
south_migrationhistory. This step was done after South requirement has been added to the Django project. The action needed was as simple as issuing the command
python manage.py syncdb, which will create
south_migrationhistory table. Depicted below, are our current list of tables in the database up until this step.
After adding South and running manage.py sycndb
After having this South table created, you can begin the next step of using South into your Django project. Pick the method that suit your current situation.
Using South Prior Applications Model's Tables Created
If you follow this Django tutorial series as it is, well, what you really need is actually not this step. You can skip this section and go to the next section of "Using South After Creating Applications Model's Tables". It is because you have use
manage.py sycndb to create
If, however, you want to use South prior any application model's tables were created, the steps (including its current application state) are the following:
You have already run
python manage.py syncdbcommand to create Django built-in application's tables.
You have already included South in
setings.pyand have already run
syncdbafterward to create
You have already created your application models code in the file
python manage.py schemamigration kfimemberapp --initialto prepare South first initial migration code. It will create
migrationsdirectory in your application directory, read the current state of your applications model and create initial Python migration code in
migrationsdirectory, to let you create its required database tables later on. Remember, if you use application name other than
kfimemberapp, then use it.
- The previous step is only creating its initial Python migration code. It didn't actually create those tables for you. To really create it, you can simply use the command :
python manage.py migrate kfimemberapp.
The final result is your application model's tables were created, and you can simply modify your application models easily, knowing that South will be able to modify the related tables for you. Or in other word, after running
python manage.py migrate kfimemberapp command, your database already in synched with your model. Below are its screenshot of this command output:
Adding South prior application model's table creation
Using South After Creating Applications Model's Table
This step is the correct step if you follow this tutorial series as is, that is, you have code your application models in file
kfimemberapp/models.py and have already created the related table(s) into application database using
python manage.py syncdb command. To introduce South in this point of application development, the steps are the following:
You have to make sure that your database schema already in synched with your application models and there is no
wsgi/kfimember/kfimemberapp. In order to do this, you may want to delete (if any)
migrationsdirectory, drop your database (or just delete the tables related to your application, namely all tables with the name prefixes with
kfimemberapp_), recreate the database and running
python manage.py syncdbcommand to synched the database with your application model.
Just issue this simple South convert command :
python manage.py convert_to_south kfimemberapp.
It will create
migrations directory, creating initial database migrations code inside it, and running fake migrations command. It's fake, because South only pretend that it runs the migration script, although it's not, because the database already synched with the application model . The final result will be the same like our previous method of introducing South prior application model's tables were created. Below is the screenshot of this command output:
Adding South after application model's table were created with syncdb
Using South to Manage Automatic Migrations
If the previous steps seem somewhat of a burden, now is the time to make South shine : automatically manage schema migration. That is, you can concentrate modifying your application model and let South take care of managing its database schema synchronization for you. For example, suppose that you want to add another field to
SertifikatModal model, let say
jumlah which is Indonesian word for
amount that denote how much is the stock certificate hold for this particular transaction.
1 2 3 4 5 6 7
from django.db import models from django.contrib.auth.models import User class SertifikatModal(models.Model): user = models.ForeignKey(User) nomor = models.CharField(max_length=30) #a.k.a number jumlah = models.FloatField(default = 0) #a.k.a amount
Now, how do we use South to properly synched our database? Simple. Just issue the command
python manage.py schemamigration kfimemberapp --auto to let South create the migration code for your and afterward, simply issue the command
python manage.py migrate kfimemberapp to apply the migration. The result? You don't have to manually touch your database to synch all of its related tables and field. Depicted below is the output of this command.
Using auto migration feature of South
Have a check on your database table! You will see that it already have a new
amount field inside
Making It Synched with Openshift Database
If you have followed this article series until this point, chances are you already know how to properly manage your Django project and its schema migrations using South, although I left out some of South features in this article (such as manual migration). For that, you can read it at South own documentation web here.
As always you can download the code for this article here : kfimember-part-4.zip.
Or, follow its development in Github repo here : kfimember.