Packaging a Django Application as a Stand-alone Application

Here is a little diversion from my normal Software Craftsmanship ramblings. Recently, I needed to package a Django web app as a stand-alone Windows Application with an installer (see below for the rationale)

In order to make this work I needed to get around a few hurdles:

• Since I'm using Django, I didn't want to customer to have to install Python, so I needed to turn the Python code into an .exe. Py2exe is great for this.
• I also use a bunch of other python libraries such as rpyc, Python Win32 extensions and have a bunch of custom C++ components that interface with Python via Swig. These all needed to get wrapped up and included as well.

I didn't want the customer to have to deal with installing each of these components and sewing them together. Assume no IT person is available. I also didn't want to show the installer spawning off all these other sub-installers. It needs to install and appear in the menus like every other Windows application.

"Well", I thought, "best get started. Let's get py2exe running on manage.py (the django project startup utility) and make a .exe out of it." And that's when the fight started.

Django does some funky magic with imports under the hood that breaks the component discovery mechanism of py2exe. The first thing I needed to do was eliminate manage.py. There were a couple of good blog posts that talk about this very problem. The trick is mapping your inside and outside directories so you can keep your database and template files outside of the package that py2exe fabricates.

Following their lead, I had a bootstrap program I could use to launch my application as an executable. All I had to do then was throw an install script around it with InnoSetup we were off to the races.

Then there was Part Two: I had my daemon process that did the network monitoring. This was also Python and used the rpyc library to communicate with the Django server. It also talked to a bunch of C++ programs that were python-wrappered using Swig, so it was .dll hell. This was going to be my other .exe.

Py2exe normally does a great job of dealing with .pyd files (essentially Python extension dll's), but it has no way of knowing when a .pyd references a .dll which may reference many other dll's since they are loaded dynamically at runtime ... as their name suggests. The result was a crashing application in my test VM.

In order to track down all the dll dependencies I used Dependency Walker to see which dll's were being accessed and which were failing. By using the "Profiling" option, you are able to see when a LoadLibrary call is made and where it fails.

Happy to say ... it works like a charm.

Why do this?

This is an experiment. There's a part of me that likes the idea of packaging a web-application as a standalone application. While this is only applicable to certain class of applications, using a lightweight HTTP server and a tiny database like SqlLite to deploy is pretty sweet. Why? A browser is a familiar experience and it's way faster to write UI for web than GUI frameworks. People are familiar with web UI semantics. They know what happens when you close a browser window, when you hit the refresh or back buttons, when you click on a link. GUI products require new learning. Most of the users work day is in a browser. Why make them switch context to another tool?

I can hear you now "Why not just make it Software as a Service (SaaS) like everyone else?"

Normally I would. 99.9% of the time SaaS is the way to go. But for this application I need to have a continuously running process (which listens on the internal network) while the user interface is running. I could still do this with Saas, but it would require people trusting to install the daemon process inside their network and let it call home. And, for our customers, often times their production network does not have access to the WAN.

Down the road, they can buy a SaaS version if we deem the product worthy of that effort, or upgrade to more robust components. But for now, it's an easy way to get started. And their entire team can try the application without every having to install it and futz with networking. This also falls into the "release early, release often" camp of product development. I need to get feedback from our early adopter customers and this is a simple way to get started.

Moreover, look at all the other tools I can integrate with a simple link: I'm using UserVoice to capture customer feedback and jquery for some nice UI. I can use Google Analytics to see usage patterns. There is so much more I can do with a local website than is remotely available to a conventional fat-app. I can easily put a REST API on the product if need be and online help is a simple addition.

The obvious issue, in the short term, is that someone needs to remember to launch the server before hitting the web page. This could be an issue. There's also no name resolution for the person that installed the application. It's essentially a localhost website. And, there are known security issues with this particular http server. I may need to use something a little more secure soon ... but still lightweight. We don't always want to notify the IT department, get new databases created, add new modules to the application server. This, to me, is a middle ground ... well, at least I hope so. I may regret those words.

I'll let you know later how this experiment goes.