Web Services in Python - Part II - Authentication and Login Sessions

In the previous phase of the project we demonstrated how to create a simple web service using Werkzeug and implementing a basic MVC pattern. For this phase of the project we'll be concentrating on handling an authentication request and showing how we can restrict certain methods only to authenticated users. For fun we're going to handle authentication using a challenge response scheme.

Challenge Response

A challenge response authentication scheme is useful for determining if two parties know the same information over an insecure connection using some clever hashing techniques. In short, when the client requests to authenticate the server responds with the user's password salt along with a newly generated challenge salt and a challenge id. The client then computes the resulting hash using the following method.

hash(hash(password + salt) + challenge)

The client then sends this result to the server along with the challenge's id. Now the server can look up the challenge and compute the hash using the same information to determine if it's a match. A challenge response scheme also allows the client to send it's own challenge to allow the client to determine if the server knows the password as well but we'll be skipping this step. Bear in mind that a challenge response scheme is not a replacement for SSL since without it you will be forced to send passwords as plain text during user registration.

You can read more about the challenge response scheme on Wikipedia.

For this implementation I've chosen to use the SHA-256 hashing function which is available in the PyCrypto library. We'll need to install this to our virtual environment so after activating use the command pip install pycrypto. I did get a warning that the GMP or MPIR libraries weren't found but it wasn't related to the hashing functions.

Getting Started

Before we start making any changes to the web service code we'll need to make some changes to the database. We'll be adding five new tables, a couple functions and we'll need some additional initialization information.

/databases/passport_schema.sql


DROP DATABASE IF EXISTS passport;
CREATE DATABASE passport;

USE passport;

CREATE TABLE application_info (
    applicationid INT NOT NULL AUTO_INCREMENT,
    name VARCHAR(50) NOT NULL,
    PRIMARY KEY (applicationid),
    UNIQUE(name)
) ENGINE=InnoDB;

CREATE TABLE user_info (
    userid INT NOT NULL AUTO_INCREMENT,
    uniqueid CHAR(36) NOT NULL,
    username VARCHAR(50) NOT NULL,
    password VARCHAR(100) NOT NULL DEFAULT '',
    salt CHAR(20) NOT NULL DEFAULT '',
    email VARCHAR(50) NOT NULL DEFAULT '',
    emailverified INT NOT NULL DEFAULT 1,
    active INT NOT NULL DEFAULT 1,
    PRIMARY KEY (userid),
    UNIQUE(username)
) ENGINE=InnoDB;

DELIMITER //
CREATE TRIGGER t_user_info_insert BEFORE INSERT ON user_info
    FOR EACH ROW 
    BEGIN
        DECLARE uuid CHAR(36);
        SET uuid = randomUUID();
        WHILE EXISTS(SELECT userid FROM user_info WHERE uniqueid = uuid) DO
            SET uuid = randomUUID();
        END WHILE;
        SET NEW.uniqueid = uuid;
        SET NEW.salt = randomString(20, NULL);
    END//
DELIMITER ;

CREATE TABLE user_applications (
    bindid INT NOT NULL AUTO_INCREMENT,
    applicationid INT NOT NULL,
    userid INT NOT NULL,
    PRIMARY KEY (bindid)
) ENGINE=InnoDB;

CREATE TABLE user_logins (
    loginid INT NOT NULL AUTO_INCREMENT,
    uniqueid CHAR(36) NOT NULL,
    userid INT NOT NULL,
    applicationid INT NOT NULL,
    ttl INT NOT NULL,
    loggedin DATETIME NOT NULL,
    PRIMARY KEY (loginid)
) ENGINE=InnoDB;

DELIMITER //
CREATE TRIGGER t_user_logins_insert BEFORE INSERT ON user_logins
    FOR EACH ROW 
    BEGIN
        DECLARE uuid CHAR(36);
        SET uuid = randomUUID();
        WHILE EXISTS(SELECT loginid FROM user_logins WHERE uniqueid = uuid) DO
            SET uuid = randomUUID();
        END WHILE;
        SET NEW.uniqueid = uuid;
        SET NEW.loggedin = NOW();
        SET NEW.ttl = 90;
    END//
DELIMITER ;

CREATE TABLE challenge_info (
    challengeid INT NOT NULL AUTO_INCREMENT,
    uniqueid CHAR(36) NOT NULL,
    userid INT NOT NULL,
    value CHAR(20) NOT NULL,
    expires DATETIME NOT NULL,
    PRIMARY KEY (challengeid)
) ENGINE=InnoDB;

DELIMITER //
CREATE TRIGGER t_challenge_info_insert BEFORE INSERT ON challenge_info
    FOR EACH ROW
    BEGIN
        DECLARE uuid CHAR(36);
        SET uuid = randomUUID();
        WHILE EXISTS(SELECT challengeid FROM challenge_info WHERE uniqueid = uuid) DO
            SET uuid = randomUUID();
        END WHILE;
        SET NEW.uniqueid = uuid;
        SET NEW.value = randomString(20, NULL);
        SET NEW.expires = DATE_ADD(NOW(), INTERVAL 30 MINUTE);
        DO randomString(20, NULL);
    END//
DELIMITER ;

CREATE TABLE challenge_spoof (
    challengeid INT NOT NULL AUTO_INCREMENT,
    uniqueid CHAR(36) NOT NULL,
    username VARCHAR(50) NOT NULL,
    salt CHAR(36) NOT NULL,
    value CHAR(36) NOT NULL,
    expires DATETIME NOT NULL,
    PRIMARY KEY (challengeid)
) ENGINE=InnoDB;

DELIMITER //
CREATE TRIGGER t_challenge_spoof_insert BEFORE INSERT ON challenge_spoof
    FOR EACH ROW
    BEGIN
        DECLARE uuid CHAR(36);
        SET uuid = randomUUID();
        WHILE EXISTS(SELECT challengeid FROM challenge_spoof WHERE uniqueid = uuid) DO
            SET uuid = randomUUID();
        END WHILE;
        SET NEW.uniqueid = uuid;
        SET NEW.value = randomString(20, NULL);
        SET NEW.expires = DATE_ADD(NOW(), INTERVAL 30 MINUTE);
        SET NEW.salt = randomString(20, NULL);
    END//
DELIMITER ;

As you can see we're creating a user_info table which will contain our users authentication information. In addition, user_logins will be used to keep track of which users are currently logged in and for how long and user_applications will define what applications each user has access to. The challenge_info and challenge_spoof tables serve nearly the same purpose except that the challenge_spoof table will handle challenges for usernames that do not exist so we do not alert the client to which usernames are present inside the system. As you can see we're making calls to a randomString and randomUUID function so let's take a look at those.

/databases/passport_function.sql


USE passport;

DROP FUNCTION IF EXISTS randomString;

DELIMITER //
CREATE FUNCTION randomString($length int, $charlist varchar(128)) RETURNS VARCHAR(128)
BEGIN
    SET @chars = '0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ';
    IF $charlist IS NOT NULL THEN
        SET @chars = $charlist;
    END IF;
    SET @charLen = length(@chars);

    SET @randomString = '';

    WHILE length(@randomString) < $length DO
        SET @randomString = concat(@randomString, substr(@chars,CEILING(RAND() * @charLen),1));
    END WHILE;

    RETURN @randomString;
END//
DELIMITER ;

DROP FUNCTION IF EXISTS randomGUID;

DELIMITER //
CREATE FUNCTION randomUUID()
RETURNS CHAR(36)
BEGIN
    SET @chars = '0123456789abcdef';
    RETURN CONCAT(randomString(8, @chars), 
        CONCAT('-',
        CONCAT(randomString(4, @chars),
        CONCAT('-',
        CONCAT(randomString(4, @chars),
        CONCAT('-',
        CONCAT(randomString(4, @chars),
        CONCAT('-', randomString(12, @chars)))))))));
END//
DELIMITER ;

MySQL does have a UUID() function but the UUIDs it generates are a bit too predictable and are based on information about the system where they were created. Neither of these behaviors is ideal for generating UUIDs fit for public consumption. All we have to do now is populate the tables with some initialization information.

/databases/passport_init.sql


USE passport;

DELETE FROM application_info;
INSERT INTO application_info (name) VALUES ('Passport');

DELETE FROM user_info;
INSERT INTO user_info (username) VALUES ('root');
UPDATE user_info SET 
    password = 'ec0cc4659dab4197e246f28e8eb333f3e01afedff7e016939cce553a40b8c1d7', 
    salt = 'somesalt'
    WHERE username = 'root';

DELETE FROM user_applications;
INSERT INTO user_applications (applicationid, userid) VALUES (
    (SELECT applicationid FROM application_info WHERE name = 'Passport'),
    (SELECT userid FROM user_info WHERE username = 'root')
);

You can alter the password to whatever you like of course but for easy demonstration this is the SHA256 hash of password1 combined with the salt. That's all we need to do to our database. Just run these scripts in order and don't forget about passport_user.sql and we'll be good to go. Now let's take a look at our updated models.

/models/data.py


import uuid
from utilities.database import staticquerymethod, querymethod
from Crypto.Hash import SHA256
from storm.locals import *
from storm.properties import UUID

class Application(Storm):
    __storm_table__ = 'application_info'
    applicationid = Int(primary=True)
    name = Unicode()
    users = ReferenceSet(   applicationid,
                'ApplicationUser.applicationid',
                'ApplicationUser.userid',
                'User.userid')

    @staticquerymethod
    def findByName(store, name):
        return store.find(Application, Application.name == name).one()

class User(Storm):
    __storm_table__ = 'user_info'
    userid = Int(primary=True)
    uniqueid = UUID()
    username = Unicode()
    password = Unicode()
    salt = Unicode()
    email = Unicode()
    emailverified = Int()
    active = Int()
    challenges = ReferenceSet(userid, 'Challenge.userid')

    @staticquerymethod
    def get(store, id):
        return store.find(User, User.userid == id).one()

    @staticquerymethod
    def findByUsername(store, username):
        return store.find(User, User.username == username).one()

    def authenticate(self, challenge, response):
        value = challenge.read()
        result = SHA256.new(self.password + value).hexdigest()
        return result == response

class ApplicationUser(Storm):
    __storm_table__ = 'user_applications'
    __storm_primary__ = 'userid''applicationid'
    bindid = Int(primary=True)
    userid = Int()
    applicationid = Int()

As you can see we've added another member to our Application entity which uses Storms ReferenceSet class to define a many-to-many relationship. Unlike nHibernate Storm requires that the bridge entity be defined so we've created our ApplicationUser class to serve this purpose. Notice that we're setting the __storm_primary__ member to a list of the fields we'll be using to establish the relationship. In the User class you can see that we're again using the ReferenceSet class but this time it's being used to define a one-to-many relationship with our Challenge entity. Finally take a look at the authentication method and you'll see this is where the server computes the SHA-256 hash of the password hash and the challenge to authenticate the user. Since we don't want the challenged to be used multiple times we're using the read() method to return the value and delete the challenge so we don't forget.

class UserLogin(Storm):
    __storm_table__ = 'user_logins'
    loginid = Int(primary=True)
    uniqueid = UUID()
    ttl = Int()
    loggedin = DateTime()
    userid = Int()
    applicationid = Int()
    user = Reference(userid, 'User.userid')
    application = Reference(applicationid, 'Application.applicationid')

    def __init__(self, user, application):
        self.user = user
        self.application = application

    @staticquerymethod
    def get(store, id):
        return store.find(UserLogin, UserLogin.loginid == id).one()

    @staticquerymethod
    def findByUniqueId(store, uniqueid):
        try:
            UniqueId = uuid.UUID(uniqueid)
        except:
            return None
        return store.find(UserLogin, UserLogin.uniqueid == UniqueId).one()

    @staticquerymethod
    def findExisting(store, userid, applicationid):
        return store.find(UserLogin, UserLogin.userid == userid and \
                        UserLogin.applicationid == applicationid).one()

    @staticquerymethod
    def create(store, user, application = None):
        if application is None:
            application = Application.findByName(unicode('passport'))
        result = UserLogin.findExisting(user.userid, application.applicationid)
        if result is None:
            result = store.add(UserLogin(user, application))
            store.commit()
        return result

class Challenge(Storm):
    __storm_table__ = 'challenge_info'
    challengeid = Int(primary=True)
    uniqueid = UUID()
    userid = Int()
    value = Unicode()
    expires = DateTime()
    user = Reference(userid, 'User.userid')

    def __init__(self, user):
        self.user = user

    @staticquerymethod
    def get(store, id):
        return store.find(Challenge, Challenge.challengeid == id).one()

    @staticquerymethod
    def findByUniqueId(store, uniqueid):
        try:
            UniqueId = uuid.UUID(uniqueid)
        except:
            return None
        return store.find(Challenge, Challenge.uniqueid == UniqueId).one()

    @staticquerymethod
    def findByUser(store, user):
        return store.find(Challenge, Challenge.userid == user.userid).one()

    @staticquerymethod
    def create(store, user):
        result = Challenge.findByUser(user)
        if result is None:
            result = store.add(Challenge(user))
            store.commit()
        return result

    @staticmethod
    def spoof(username):
        return ChallengeSpoof.create(username)

    @querymethod
    def read(store, self):
        result = self.value
        store.remove(self)
        store.commit()
        return result

class ChallengeSpoof(Storm):
    __storm_table__ = 'challenge_spoof'
    challengeid = Int(primary=True)
    uniqueid = UUID()
    username = Unicode()
    salt = Unicode()
    value = Unicode()
    expires = DateTime()

    def __init__(self, username):
        self.username = username

    @staticquerymethod
    def findBySpoof(store, username):
        return store.find(ChallengeSpoof, ChallengeSpoof.username == username).one()

    @staticquerymethod
    def create(store, username):
        result = ChallengeSpoof.findBySpoof(username)
        if result is None:
            result = store.add(ChallengeSpoof(username))
            store.commit()
        return result

Once the user has been authenticated we need to create a login session for them using the UserLogin.create() method. This method also accepts an application parameter which we default to the Passport application if none is supplied. In addition to the Challenge class we've also defined ChallengeSpoof which we'll use to fake a challenge for usernames that don't exist to avoid announcing this to the client. It's not a perfect solution especially since during registration we'll have to give some indication of this but there are steps we can take there to mitigate this security concern as well.

Before we move on to the ServiceController let's take a look at some changes that were made to the BaseController.

/controllers/basecontroller.py


from models.data import UserLogin
from utilities.messages import Message
from utilities.exceptions import ClientException
from werkzeug.urls import url_decode

class BaseController(object):
    loginCookieName = 'loginid'
    _applicationName = 'passport'

    def __init__(self, request):
        self._request = request
        self._cookies = request.cookies.copy()
        self._querystring = url_decode(request.query_string)
        self._userLogin = self._get_user_login()

    @staticmethod
    def _get_args(data, func):
        argcount = func.func_code.co_argcount
        if argcount == 0:
            return dict()
        args = [arg.lower() for arg in func.func_code.co_varnames[0:argcount] if arg[0:1] != '_']
        result = dict((k.lower(), v) for k, v in data.iteritems() if k.lower() in args)
        for arg in filter(lambda x: x not in result, args):
            result[arg] = None
        return result

    def _get_user_login(self):
        loginid = None
        if BaseController.loginCookieName in self._cookies:
            loginid = self._cookies[BaseController.loginCookieName]
        elif loginCookieName in self._querystring:
            loginid = self._querystring[BaseController.loginCookieName]

        if loginid is None:
            return None

        return UserLogin.findByUniqueId(loginid)

    def setCookie(self, name, value):
        self._cookies[name] = value

    def getCookies(self):
        return self._cookies.copy()

    def isLoggedIn(self):
        return self._userLogin is not None      

    @classmethod
    def restrictedmethod(cls, func):
        def inner(self, *args, **kwargs):
            if self._userLogin is None or \
                self._userLogin.application.name.lower() != BaseController._applicationName:
                raise ClientException(  source = BaseController.loginCookieName,
                            message = 'Not authorized.',
                            number = 403 )
            return func(self, *args, **kwargs)
        return inner

    @classmethod
    def httpget(cls, func):
        def inner(self, *args, **kwargs):
            return func(self, *args, **BaseController._get_args(self._querystring, func))
        return inner

Here we've created a new wrapper called restrictedmethod which will allow us to lock down specific controller methods. We've also added support for handling cookie values using the setCookie and getCookies methods which will be necessary for maintaining our login session. As we saw in the last phase the ControllerProvider class is responsible for passing cookies to our output method in the Message class but until now all it did was copy the request cookies. In order to remedy this we need to only alter the return statement of runMethod.

/controllers/provider/controllerprovider.py


return success, loggedIn, clientExceptions, serverExceptions, data, instance.getCookies()

Simple enough. We now have the ability to set cookies from inside a controller. Let's take a look at the new service methods we'll be using.

/controllers/servicecontroller.py

@BaseController.httpget
def getChallengeMethod(_self, username):
    if username is None:
        raise ClientException( source = 'username',
                    message = 'User name must be specified.',
                    number = 1 )
    user = User.findByUsername(username)
    if user is None:
        challenge = Challenge.spoof(username)
        salt = challenge.salt
    else:
        challenge = Challenge.create(user)
        salt = user.salt
    return { 
        'Challenge' : {
            'ID' : str(challenge.uniqueid),
            'Value' : str(challenge.value),
            'Salt'str(salt) } } 

This is the first request a client makes when trying to authenticate with our service. You can see that it does all the usual checks to make sure that a username was supplied and if not throws a helpful ClientException to the user. Depending on if the username exists a real or a spoofed challenge and salt information is returned.

@BaseController.httpget
def authenticateChallengeMethod(_self, username, response, id):
    user = User.findByUsername(username)
    challenge = Challenge.findByUniqueId(id)
    if challenge is None or user is None or not user.authenticate(challenge, response):
        raise ClientException( source = 'password',
                    message = 'Incorrect username or password.',
                    number = 1 )
    login = UserLogin.create(user = user)
    _self.setCookie(BaseController.loginCookieName, str(login.uniqueid))
    return None     

Once the client has computed the hash it returns the response to the server along with the challenge identifier. Note that we don't allow the client to return the challenge itself because we can't trust them not to make up an easy challenge for themselves. We don't need any extra logic for spoofed challenges because those will always result in a login failure. Once the response has been verified we create a UserLogin instance and set it's uniqueid value to the loginCookieName we defined in BaseController.

@BaseController.httpget
def logoutMethod(_self):
    CookieManager.setCookie(BaseController.loginCookieName, None)
    return None 

If we want to remove a cookie all we need to do is set it's value to None and the Message.output method will handle the rest. There's only one final edit we need to make.

@BaseController.restrictedmethod
@BaseController.httpget
def getApplicationMethod(_self, name):

Adding our BaseController.restrictedmethod is the only necessary change in order to disallow access to any method. In the future this will also allow us to add permission based security as we can pass in parameters to specify which permissions are necessary to access the method. Of course we still need a way test this. All we need is a simple HTML file and a SHA-256 javascript hash implementation.

passport.htm


<!DOCTYPE html><html><head>
    <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.5.1/jquery.min.js" language="javascript" type="text/javascript"></script>
    <script src="http://www.webtoolkit.info/djs/webtoolkit.sha256.js" language="javascript" type="text/javascript"></script>
</head><body>
<script language="javascript" type="text/javascript">
    $(document).ready(function () {
        $("#txtPassword").keypress(function (e) { if (e.which == 13) DoLogin(); });
        $("#btnLogin").click(function () { DoLogin(); });
        $("#btnLogout").click(function () {
            PassportLoadData("/Service/LogOut/"nullfunction (msg) {
                alert("Successfully logged out.");
            });
        });
        $("#btnGetApplication").click(function () {
            PassportLoadData("/Service/GetApplication/", { Name: "passport" }, function (msg) {
                alert("Passport application has the ID " + msg.Data.Application.ID + " and the users " + msg.Data.Application.Users)
            });
        });
        $("#txtUsername").focus();
    });

    function PassportLoadData(url, data, successCallBack, clientExceptionCallBack) {
        url = "http://phase2.linuxtutorial.netortech.com:8001" + url;
        $.ajax({
            type: "GET",
            data: data,
            url: url,
            contentType: "application/javascript; charset=utf-8",
            dataType: "jsonp",
            success: function (msg) {
            if(msg.success)
                successCallBack(msg);
            else if(msg.ClientExceptions.length > 0)
                if(!clientExceptionCallBack)
                    $.each(msg.ClientExceptions, function (index, item) {
                                alert(item.Number + ": " + item.Source + " caused error " + item.Message + ", value: " + item.Value);
                    });
                else
                    clientExceptionCallBack(msg);
            },
            error: function (msg) {
                AppState().pushMessage("AJAXLoadData failed for url: " + url);
            }
        });
    }

    function DoLogin() {
        $("#rowLoginError").hide();
        PassportLoadData("/Service/GetChallenge/", { Username: $("#txtUsername").val() }, function (msg) {
            var response = SHA256(SHA256($("#txtPassword").val() + msg.Data.Challenge.Salt) + msg.Data.Challenge.Value);
            PassportLoadData("/Service/AuthenticateChallenge/", { Username: $("#txtUsername").val(), Response: response, ID: msg.Data.Challenge.ID }, function (msg) {
                alert("Login success!");
            }, function (msg) {
                $("#rowLoginError").show();
            });
        });
    }
</script>
<table>
    <tr id="rowLoginError" style="display: none;"><td colspan="2">Incorrect Username or Password.</td></tr>
    <tr><td>Username:</td><td><input type="text" id="txtUsername" /></td></tr>
    <tr><td>Password:</td><td><input type="password" id="txtPassword" /></td></tr>
    <tr><td colspan="2">
        <input type="button" id="btnLogin" value="Login" />
        <input type="button" id="btnLogout" value="Logout" />
    </td></tr>
    <tr><td colspan="2"><input type="button" id="btnGetApplication" value="Get Passport Info" /></td></tr>
</table></body></html>

Since the service is JSONP compatible you can open this HTML file directly from your desktop and try it out. I borrowed a lot of concepts that I went over in the ASP.Net MVC web service tutorial so take a look at that article for a more in depth explanation of what's going on. After this phase I'll be putting all the HTML views into our ASP.Net project but really these can be hosted on any web server.

Phase II Links: Download | Demo

Conclusion

That's it! We now have an authentication system in place and have a demonstration of a restricted method. Next we're going to add some service methods which will allow us to create new applications, register new users and allow external applications to confirm a user's credentials.

Quick Links: << Previous: Setting up an MVC Pattern | Next: String Resources and Some More Web Methods >>