Creating a Lightweight Application Server Using Werkzeug

When it came time to put up the second phase of the Python tutorial I realized that Werkzeug does not support any kind of hostname binding. I've also mentioned that running these applications on port 80 is problematic due to the fact that root privileges are required to access those ports. Werkzeug is WSGI compatible so installing Apache would be the smartest choice in a production environment but for the sake of a learning exercise I wanted to explore other possibilities. I had already done some research on how to use port 80 so that's where I started.

Running an Application on Port 80

After running a few searches there were three most recommended solutions.

  1. More recent versions of Linux have a method for giving an executable the CAP_NET_BIND_SERVICE capability which allows an application to open a lower port using setcap.
  2. Use Python's os.setuid which allows the application to switch to a user with lower privileges after opening the port.
  3. Reroute port 80 traffic to a higher port using the iptables command.

Unfortunately the first suggestion doesn't work with scripts. From my limited research there appears to be a way to convert Python scripts to executables but I didn't feel like going that route for now. I did give some consideration to the second choice but I was concerned that I wouldn't be able to implement it properly without a deeper understanding of the Linux OS. The third solution however seemed to fit my needs perfectly. There are some concerns with this approach since it allows any application to intercept this traffic but since I won't be running any other applications on this server it isn't a huge concern. The command to reroute the port traffic was relatively simple.

sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8000

This also turned out to be an answer for another problem I had when I thought I would open up multiple ports using Werkzeug but the run_simple method blocks and I wasn't too keen on getting into multithreading just yet. I decided to just have the server run on a single port and assume that any traffic it was intended to receive had already been configured to reroute to it's single port.

Hostname Binding

Before I dove into how I wanted the server to function I started off with designing a configuration file. As you may have guessed by now I'm pretty fond of JSON and unsurprisingly I decided to use it for the syntax.

/bindings.cfg


{
    "provider": {
        "port"8000 },
    "applications": [ {
        "name""MVC Passport - Phase I",
        "moduleName""netortech.passport.v1.main",
        "className""Passport",
        "bindings": [ {
            "hostname""*",
            "port"80 }, {
            "hostname""*",
            "port"8000 } ],
        "settings": {
            "connstr""mysql://passport:somepass@localhost:3306/passport" }
        }, {
        "name""MVC Passport - Phase II",
        "moduleName""netortech.passport.v2.main",
        "className""Passport",
        "bindings": [ {
            "hostname""phase2.linuxtutorial.netortech.com",
            "port"80 }, {
            "hostname""*",
            "port"8001 } ],
        "settings": {
            "connstr""mysql://passport:somepass@localhost:3306/passport" }
        } ]
}

As I mentioned earlier, the server will be running on a single port that's specified at provider.port. The 8000 port seemed like a good choice so I stuck with that. The configuration file supports multiple applications and rather than using a file system location simply specifies the Python module and class to be used. There is also an optional settings value which allows us to provide application specific information. I opted to set the database connection string here in order to isolate my configuration to a single file. Each binding has a port and hostname property. I only allowed for full wildcard bindings because at this point I don't need partial wildcards. However a few tweaks of the server code would easily resolve this. Speaking of which, let's take a look at the server.

/server.py


import json
import importlib
from werkzeug.wrappers import Request, Response

These are all the imports we'll need. The json class will allow us to easily parse our configuration file and the Werkzeug Request object will allow us to parse the hostname and port number of the request. We'll also need the Response object in case the request doesn't resolve to anything and we need to toss the client an error message. In order to load our application using the module and class names we'll be using the importlib class.

def loadApplication(application):
    if not 'name' in application:
        print 'Name must be specified for all applications'
        return None

    appName = application['name']

    if not 'moduleName' in application:
        print 'No module name specified for application %r.' % (appName)
        return None

    moduleName = application['moduleName']

    if not 'className' in application:
        print 'No class name specified for application %r' % (appName)
        return None

    className = application['className']

    settings = {}
    if 'settings' in application:
        settings = application['settings']

The application parameter will simply be the resulting dict from the parsed configuration JSON. We have a few lines of code that ensure we have the required properties name, moduleName and className defined and finally we default our settings to an empty dictionary.

    try:
        module = importlib.import_module(moduleName)
    except:
        print 'Failed to load module %r.' % (moduleName)
        return None

    try:
        cls = getattr(module, className)
    except:
        print 'No class %r found in %r.' % (className, moduleName)
        return None

    try:
        instance = cls(settings=settings)
    except:
        print 'Failed to instantiate class %r in %r.' % (className, moduleName)
        return None

    return instance

Once we have all the necessary information we can attempt to import the module and then grab the application class. Passing the appropriate error messages back to the console should any steps in this process fail. The final steps are to pass the settings value to the class constructor and return the instance.

def loadConfig(filename):
    result = { 'applications': {}, 'bindings': {}, 'provider': { 'port'8000 } }   

    try:
        configStr = open(filename).read()
    except:
        print 'Failed to open file %r.' % (filename)
        return result

    try:
        config = json.loads(configStr)
    except:
        print 'Failed to parse file %r.' % (filename)
        return result

    if not 'applications' in config:
        print 'No applications configured.'
        return result

Our loadConfig method will be responsible for opening the configuration file and returning a dictionary that's easier to work with. Should any of this fail the method will return our empty configuration result.

    bindings = result['bindings']

    for application in config['applications']:
        appInstance = loadApplication(application)

        if appInstance is None:
            continue

        appName = application['name']

Once we've parsed the file it's time to iterate through all the applications entries. We first make a call to our previously defined loadApplication method and determine if the application loaded successfully. If not our continue statement will skipp the rest of the loop and move on to the next application.

        if not 'bindings' in application:
            print 'No bindings defined for %r' % (appName)
            continue

        for binding in application['bindings']:
            if not 'hostname' in binding:
                print 'Hostname must be specified for all %r bindings.' % (appName)
                continue

            if not 'port' in binding:
                print 'Port must be specified for all %r bindings.' % (appName)
                continue

            port = binding['port']
            hostname = binding['hostname']

            if port in bindings:
                if hostname in bindings[port]:
                    print 'Conflicting bindings between applications %r and %r.' % \
                        (appName, bindings[port][hostname]['name'])
                else:
                    bindings[port][hostname] = application
                    if not appName in result['applications']:
                        result['applications'][appName] = {
                            'application': application,
                            'instance': appInstance }
            else:
                bindings[port] = { hostname: application }
                if not appName in result['applications']:
                    result['applications'][appName] = {
                        'application': application,
                        'instance': appInstance }

We now can start iterating through our application's bindings and creating a list of them in our result. Each port will have a single binding with multiple host entries underneath making it easy to detect duplicate bindings. If all goes well we can add the hostname entry populated with our application dictionary and use it's name as a key for our applications property in the result.

    if 'provider' in config:
        result['provider'] = config['provider']

    return result

Finally all we have to do is set the provider and return our result. We're almost done!

class Provider(object):
    def __init__(self, config):
        self.config = config

    def __call__(self, environ, start_response):
        request = Request(environ)
        hostname, port, _ = (request.host + ':80:').split(':'2)
        port = int(port)

        appInfo = None

        if port in self.config['bindings']:
            if hostname in self.config['bindings'][port]:
                appInfo = self.config['bindings'][port][hostname]
            elif '*' in self.config['bindings'][port]:
                appInfo = self.config['bindings'][port]['*']

        if appInfo is None:
            response = Response("Service unavailable.")
            return response(environ, start_response)

        print 'Request %r resolved to application %r.' % (request.host, appInfo['name'])

        app = config['applications'][appInfo['name']]['instance']
        return app(environ, start_response)

This class serves as a proxy for our applications by accepting all requests and then mapping them to the appropriate binding. If no binding is found we respond to the client with an error message otherwise we make a call to the app the same way Werkzeug would.

if __name__ == '__main__':
    from werkzeug.serving import run_simple
    config = loadConfig('bindings.cfg'
    run_simple('0.0.0.0', config['provider']['port'], Provider(config), use_debugger=True, use_reloader=True)

Since loadConfig does most of the heavy lifting our entry point is fairly simple. We now have a light weight application server.

Links: Download

Conclusion

While this certainly isn't intended to replace a web server it is a simpler approach for development and light production environments that don't require SSL, cache control or a number of other features offered by a typical web server. Using the changes to the Passport project I explained in the previous article I was able to install the two phases of the Passport application to a new virtual environment and serve them both using the configuration file mentioned at the beginning of this article.