A web server in twisted - Part 2

Yesterday I wrote this post about how to write a very simple web server for a predefined content. Now I want to change it and discuss the solution a bit more in detail. First of all let's make it a real webserver, serving files from a known directory instead of serving some dummy content and avoid calling the browser which is not really needed.

from twisted.internet import reactor
from twisted.web2 import server, static, channel, http
PORT = 8080
DIRECTORY = '/Users/dialtone/'
s = server.Site(static.File(DIRECTORY))
reactor.listenTCP(PORT, channel.HTTPFactory(s))
reactor.run()

This is a better solution now. But how is it any good?

As this post states, the HTTPListener solution of the .Net world is not really useful since it only supports Windows Server 2003 or Windows XP with SP2, not even SP1 or Windows 2000. The author then goes on explaining what will be done to rewrite a new webserver that will solve this problem and how hard it is to handle all the request parsing for HTTP.

Mr. Andrea Boschin is very right in his statements, HTTP is not an easy protocol to implement given all the details that it comes with, especially considering that its RFC is sometimes contraddictory with itself. Fortunately the real advantage of Twisted Matrix is that all these details are already solved for you by the library author. In this case we are using the second version (which is currently under development but stable and used in many project, I use it myself in some of my projects) of the HTTP 1.1 protocol implementation.

The Original Poster starts dealing with the first problems with the integration between the existing application and the new to-be-written web server, the solution proposed is threading, since he is running this webserver as a Windows service he is in the need of running a parallel thread to deal with HTTP requests. Being completely async allows you to already forget about this. You can easily run many different services in the same process (like an SMTP server, an HTTP server and an IMAP server) without the need of threading while still keeping a good request rate in the order of 900 requests fullfilled each second on an Opteron 2.4Ghz.

The solution I came up with is already capable of adding a complete SMTP server to its features, in fact:

from twisted.internet import reactor
from twisted.web2 import server, static, channel, http
from twisted.mail import mail, maildir

PORT = 8080
DIRECTORY = '/Users/dialtone/'
MAIL_PATH= '/Users/dialtone/mail/'
SMTP_PORT = 25
POP3_PORT = 110

s = server.Site(static.File(DIRECTORY))
reactor.listenTCP(PORT, channel.HTTPFactory(s))

mailservice = mail.MailService()
domain = maildir.MaildirDirdbmDomain(mailservice, os.path.abspath(MAIL_PATH))
domain.addUser('foo', 'bar')
mailservice.addDomain('localhost', domain)

smtp = mailservice.getSMTPFactory()
pop3 = mailservice.getPOP3Factory()

reactor.listenTCP(SMTP_PORT, smtp)
reactor.listenTCP(POP3_PORT, pop3)

reactor.run()

This is that service with an HTTP server plus an SMTP server and a POP3 server. (Note: due to lack of time I haven't been able to completely test this piece of code, but it should serve as an idea about how to achieve the goal).

What is all this post about then, besides writing something about twisted and how easily you can integrate different protocols without worrying too much about concurrency (since Twisted itself handles it for you)? This post was just made to show how the .Net platform is actually limiting its users not only in the features' 'space' (it's hard to write even simple services and it's even harder to integrate them) but also from the legacy point of view. This should serve as a good lesson about the strengths of Open Source against Proprietary Software too.