Please wait...

Custom Django data migrations with RunPython Class

by in Django

In the Django framework, Migration is a very important part for operation on the database.
so today I’m showing you How Django migration can be useful for automatic data upload on creating a table in the database.

Let me create a simple Model Country in models.py 

from django.db import models


class Country(models.Model):
    name = models.CharField(max_length=100)
    code = models.CharField(max_length=10)

 

As you can see Country model have name and code field and we need to add data in the country model for all countries of the world,
So we have a solution that, Admin will go to Django admin panel and add entries one by one  in the Country model

But for making this process easy and add all countries in a model, Django migrations provides a class RunPython that can call our defined function.

For reading docs about Django RunPython class here 

https://docs.djangoproject.com/en/3.1/ref/migration-operations/#django.db.migrations.operations.RunPython

Now let me create a migrations file for Country models.py

python manage.py makemigrations
from django.db import migrations, models



class Migration(migrations.Migration):
    initial = True

    dependencies = [
    ]

    operations = [
        migrations.CreateModel(
            name='Country',
            fields=[
                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
                ('name', models.CharField(max_length=100)),
                ('code', models.CharField(max_length=10)),
            ],
        ),
    ]

 

So this migration file created by Django automatic

Now we need to modify this file for upload data in the Country table,

As you can see this migration file creating a Country table in the database so we need to upload data after creating table,

I found Country CSV file https://datahub.io/core/country-list here, so I’ll use this file for Countries data.

Before work on migration, install requests library: https://pypi.org/project/requests/

pip install requests

Let's modify the country migration file according to below

import csv
import io

import requests
from django.db import migrations, models


def upload_countries(apps, schema_editor):
    Country = apps.get_model('country', 'Country')
    url = "https://pkgstore.datahub.io/core/country-list/data_csv/data/d7c9d7cfb42cb69f4422dec222dbbaa8/data_csv.csv"
    response = requests.get(url)
    str_file = io.StringIO(response.content.decode('utf-8'), newline='\n')
    reader = csv.reader(str_file)
    for index, record in enumerate(reader):
        if not index:
            # For skip column names
            continue        
        Country.objects.create(name=record[0], code=record[1])
        print("{}. Added Country: {}".format(index + 1, record[0]))


class Migration(migrations.Migration):
    initial = True

    dependencies = [
    ]

    operations = [
        migrations.CreateModel(
            name='Country',
            fields=[
                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
                ('name', models.CharField(max_length=100)),
                ('code', models.CharField(max_length=10)),
            ],
        ),
        migrations.RunPython(upload_countries)
    ]

 

Country = apps.get_model('country', 'Country')

Here "country" is my app name so use this name according to your app name, second is a Model name.


As we can see I added

migrations.RunPython(upload_countries)

On the last index of operations Array because we want to execute code after table creation, so migrate command iterate operations array and execute, so we need to put our RunPython on the last index.

We have upload_countries function that calls with some default parameter.
Now we use requests library to download CSV from online and using CSV reader, we iterate each country one by one and saved in the Country model.

So this is an easy process for us to upload data in a table without further operation, If you move this project to the production server so this process automatic upload data on table with migrate command
Now run migrate command

 

python manage.py migrate

Output

Operations to perform:
  Apply all migrations: country
Running migrations:
  Applying country.0001_initial...
1. Added Country: Afghanistan
2. Added Country: Åland Islands
3. Added Country: Albania
4. Added Country: Algeria
5. Added Country: American Samoa
6. Added Country: Andorra
7. Added Country: Angola
8. Added Country: Anguilla
9. Added Country: Antarctica
10. Added Country: Antigua and Barbuda
...


So RunPython sometimes very useful for default data and do some different operations depending on requirements with migrations files