A web server in twisted - Part 2

Yesterday I wrote this post about how to write a very simple web server for a predefined content. Now I want to change it and discuss the solution a bit more in detail. First of all let's make it a real webserver, serving files from a known directory instead of serving some dummy content and avoid calling the browser which is not really needed.

from twisted.internet import reactor
from twisted.web2 import server, static, channel, http
PORT = 8080
DIRECTORY = '/Users/dialtone/'
s = server.Site(static.File(DIRECTORY))
reactor.listenTCP(PORT, channel.HTTPFactory(s))
reactor.run()

This is a better solution now. But how is it any good?

As this post states, the HTTPListener solution of the .Net world is not really useful since it only supports Windows Server 2003 or Windows XP with SP2, not even SP1 or Windows 2000. The author then goes on explaining what will be done to rewrite a new webserver that will solve this problem and how hard it is to handle all the request parsing for HTTP.

Mr. Andrea Boschin is very right in his statements, HTTP is not an easy protocol to implement given all the details that it comes with, especially considering that its RFC is sometimes contraddictory with itself. Fortunately the real advantage of Twisted Matrix is that all these details are already solved for you by the library author. In this case we are using the second version (which is currently under development but stable and used in many project, I use it myself in some of my projects) of the HTTP 1.1 protocol implementation.

The Original Poster starts dealing with the first problems with the integration between the existing application and the new to-be-written web server, the solution proposed is threading, since he is running this webserver as a Windows service he is in the need of running a parallel thread to deal with HTTP requests. Being completely async allows you to already forget about this. You can easily run many different services in the same process (like an SMTP server, an HTTP server and an IMAP server) without the need of threading while still keeping a good request rate in the order of 900 requests fullfilled each second on an Opteron 2.4Ghz.

The solution I came up with is already capable of adding a complete SMTP server to its features, in fact:

from twisted.internet import reactor
from twisted.web2 import server, static, channel, http
from twisted.mail import mail, maildir

PORT = 8080
DIRECTORY = '/Users/dialtone/'
MAIL_PATH= '/Users/dialtone/mail/'
SMTP_PORT = 25
POP3_PORT = 110

s = server.Site(static.File(DIRECTORY))
reactor.listenTCP(PORT, channel.HTTPFactory(s))

mailservice = mail.MailService()
domain = maildir.MaildirDirdbmDomain(mailservice, os.path.abspath(MAIL_PATH))
domain.addUser('foo', 'bar')
mailservice.addDomain('localhost', domain)

smtp = mailservice.getSMTPFactory()
pop3 = mailservice.getPOP3Factory()

reactor.listenTCP(SMTP_PORT, smtp)
reactor.listenTCP(POP3_PORT, pop3)

reactor.run()

This is that service with an HTTP server plus an SMTP server and a POP3 server. (Note: due to lack of time I haven't been able to completely test this piece of code, but it should serve as an idea about how to achieve the goal).

What is all this post about then, besides writing something about twisted and how easily you can integrate different protocols without worrying too much about concurrency (since Twisted itself handles it for you)? This post was just made to show how the .Net platform is actually limiting its users not only in the features' 'space' (it's hard to write even simple services and it's even harder to integrate them) but also from the legacy point of view. This should serve as a good lesson about the strengths of Open Source against Proprietary Software too.

A Web server in Twisted

Starting from this post that was triggered by this one I decided to do the same thing using Twisted Matrix.

Again the basic task is to write a webserver (the original poster was surprised in finding out how 'easy' it is to write a web server using .Net 2).

This is my solution:

from twisted.internet import reactor
from twisted.web2 import server, resource, channel, http
import webbrowser
PORT = 8080

class Page(resource.Resource):
    addSlash = True
    def render(self, ctx):
        return http.Response(stream="""
            <html><body><h1>This is the HTML body</h1></body></html>
        """)

s = server.Site(Page())
reactor.listenTCP(PORT, channel.HTTPFactory(s))
webbrowser.open("http://%s:%d" % ("localhost", PORT))
reactor.run()

Easy isn't it?

Random image rotators everywhere

After I came across this I decided that I should write something about the same thing but done in Nevow.
First of all the task is to write a random image rotation implementation for jpegs, gifs and pngs. This is easily accomplished by doing the following 3 things:

1. Add this to the root Page of your web site. 'random/images/directory' is a directory in your HDD containing lots of images and other random data.

child_random_images = static.File('random/images/directory')

2. Add this to each of your Page objects that use the random image (or use a common base class to have this behaviour everywhere for free):

 
def render_random_image(self, ctx, data):
    files = [f for f in os.listdir('random/images/directory') 
               if os.path.splitext(f)[1] in ['.jpg', '.gif', '.png']]
    return ctx.tag(src='/random_images/'+random.choice(files))
    

3. Use the following tag wherever you want to have a randomly rotated image in your web app:

 <img nevow:render="random_image" alt="Random Image" /> 

Now you have a wonderfully randomly rotated image in your web browser and with 2 lines less than the Ruby On Rails counterpart without sacrifying readability too much.