peopledoc/django-MacFly

Name: django-MacFly

Owner: PeopleDoc

Description: Output sql migrations for ahead of time schema changes.

Created: 2015-10-06 13:03:46.0

Updated: 2017-11-28 21:42:38.0

Pushed: 2015-12-22 13:18:05.0

Homepage: null

Size: 21

Language: Python

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

django-MacFly ? Forward To The Past

Output sql migrations for ahead of time schema changes.

TL;DR

install django-MacFly

ython
ALLED_APPS += ['django_macfly']

On a develop environment, deploy the old (version N) code and run migrations. Then, deploy the new code (version N+1) without running the migrations.

Then run:

on manage.py sqlmigratefwd

Read the result, think, hack, do.

Rationale

Here, at Peopledoc we found that:

• ZDD is great. We need to achieve at least “Almost Zero Downtime Deployments”
• ZDD is (quite) easy except when there are SQL Database schema migrations.
• Django migration system is useless in ZDD context, especially for large databases.

Our strategy for schema migration is to run migrations ahead of time, the database is migrated to schema N+1 while the code is at N.

This strategy requires coding discipline (do not rename a DB Column, do not change a column type…) and we needed a tool that checks the code changes, warn us if the changes are not ZDD-friendly, and gather all (safe) SQL commands to make the migration.

WARNING

This is NOT AT ALL an automatic migration tool. It's just a (dumb) HELPER. It outputs hints that you MUST read and understand. Your ahead-of-time migration will probably need DB triggers, query re-writing, default values and all sort of DBA voodoo stuff to work properly. MacFly DO NOT handle this. Candide use may eat your data and voids even the no-warranty-of-any-kind clause.

What it does

Basically it walks through pending migration (using django migrate internals).

For each migration:

• Check migration safety
• if it's unsafe, MacFly writes commented sql and issues a warning It's your responsabilty to interpret the issue reading SQL and the migration script
• it it's safe, MacFly prints a modified version of sqlmigrate:
  • sqlmigrate remove default values for NOT NULL columns. Here we need a SQL default because the code doesn't know the column, and inserts would fail. Hence, sqlmigratefwd doesn't drop defaults
  • sqlmigratefwd also issues a faked migration (an insert in django_migration table). So it's safe at deploy time to run migrate