Calling reactor methods (like .listenTCP and
.run) directly, as in the examples in Writing Servers, is a good way to immediately
demonstrate the use of Factories and Protocols. But you would ask for more
from a fully-fledged, easy-to-run, easy-to-configure Internet Server (with
capital I and S). To be precise, your users (defined as someone who wants to
install your server without knowing all the details of how it works) will
ask for more from it. Twisted provides this for you.
What more could we want from our little test program? Well:
suppose your QOTD server behaves a bit more like the normal port 17
server and pulls a random line from /usr/share/fortunes. Your
QOTDFactory() might take a filename to indicate where the QOTD protocols
should pull these lines. It would be nice if the person installing your
quote server didn't have to modify any Python code to change where this
file should be found.
Likewise, what if they want it to listen on some other port? That shouldn't require editing the code.
If your protocol demands that you keep some state from one invocation of the server to the next, you'll need to save some information before the server shuts down, and to restore it again when you start back up.
Suppose your protocol's purpose in life is to generate one-time keys, and that people can connect to it to retrieve a single-use key. (Don't ask me why they might want to do this. Security is such a weird big thing that chances are somebody out there will want to do something that's probably pretty dumb when you think about it carefully). The important thing is that you never give out the same key twice. So you have to remember a sequence number, and each time you give out a key, you bump up the number. Before you shut down, you save the number to a file somewhere; at start up, if the file exists you read the number from it, if it doesn't exist, you start at 0. (an example is included below)
This kind of persistent data is a common need, and many kinds of servers require it. Hence Twisted provides an easy way to record and reload this data.
This functionality is provided by the Application class (defined in twisted/internet/app.py). You create an Application with a constructor like any other object. Then you tell the app to listen to ports (just like you told the reactor to in the previous example), providing a Factory on each one. The difference is that the App won't starting listening on those ports right away, but will wait until it starts to run.
When you're done setting up the ports, you have two options: you can
start running the app immediately, by calling the .run() method, or
you can save the Application out to a file by calling the .save()
method. The saved application can then be started later by using the
twistd utility.
Here is a short example of the first option, running the server immediately. This example uses the pre-defined Daytime protocol, which simply sends the current time to each client.:
app1.pyThis program will start listening to port 8813 in the app.run()
call, and won't return from that call until the server is terminated
(probably when you send it SIGINT).
To use the second option and launch the server later, just use
.save() instead of .run(). The .save() method
takes a base name for the generated .tap file:
...
app.listenTCP(8813, f)
app.save("start")
When you run this program, it will create a file called
daytime-start.tap, and then exit. (The name is obtained by
combining the application name with the argument to .save()). To
start the server from the freeze-dried
.tap file, use
twistd (text wrapped to be more readable):
% ./app2.py Saving daytimer application to daytimer-start.tap... Saved. % twistd -f daytimer-start.tap % tail twistd.log 30/09/2002 01:38 [-] Log opened. 30/09/2002 01:38 [-] twistd 0.99.2 (/usr/bin/python2.2 2.2.1) starting up 30/09/2002 01:38 [-] license user: Nobody <> 30/09/2002 01:38 [-] organization: No Organization 30/09/2002 01:38 [-] reactor class: twisted.internet.default.SelectReactor 30/09/2002 01:38 [-] Loading daytimer-start.tap... 30/09/2002 01:38 [-] Loaded. 30/09/2002 01:38 [*daytimer*] twisted.internet.protocol.Factory starting on 8813 30/09/2002 01:38 [*daytimer*] Starting factory <twisted.internet.protocol.Factory instance at 0x81ac9fc> %
That will thaw out
the .tap file, create the
Application, and
then run it just as if you'd invoked app.run() yourself. It forks the new
server off into the background (so twistd itself completes instead of
waiting for the server to die), writes the server's process ID to a file
called twistd.pid, and directs all the server's stdout messages
to a file
called twistd.log (these file names can be changed by appropriate
arguments to twistd: see twistd -h for a list).
When you try this example, be aware that twistd returns right
away, but it takes a second or two for the server to actually start. The
twistd.pid file won't be created until it does. Wait a moment
before doing ls or netstat, or you'll think that the
server failed to start. If it persists in failing, look in
twistd.log for details. Remember that trying to bind to a reserved
port will fail unless you're root, and the exception will be listed at the
end of the log file.
To kill the server, just do:
% kill `cat twistd.pid`
When the server is shut down, you'll notice that it creates a file called
daytimer-shutdown.tap in the directory it was run from
(again, the name is
derived from the application name and the word shutdown
). This
.tap file is just like the daytimer-start.tap
created by your
original setup program, except that it represents the state of the
Application object as it existed just before shutdown, rather than when it
was freshly created by your code.
Also note that the twistd.pid file is automatically deleted when
the application shuts down.
You can add persistent data (like that
sequence number described above) to the protocol Factory object, and it will
get saved in the -shutdown.tap file. Then, if you restart the
server with twistd -f daytimer-shutdown.tap, the new server will
get the data saved by the old server, and it can pick up where the old one
left off, as if the server had been running continuously the whole time.
To take advantage of this, simply add the attributes you want to the Factory, or to your subclass of Service (see the docs on Perspective Broker for details about Services). When the application terminates, it simply pickles up the whole Application (and everything it references, including Factories and Services). Any attributes or objects you have added will be saved and later restored.
Here is an example:
app3.pyTo demonstrate this, do the following:
% ./app3.py Saving otk application to otk-start.tap... Saved. % twistd -f otk-start.tap % % nc localhost 8123 0 % nc localhost 8123 1 % nc localhost 8123 2 %
Note that the stdout of the process is being directed into the log file,
contained in twistd.log. Now stop the server, verify that it is no
longer running, then restart it from the saved-at-shutdown .tap
file:
% kill `cat twistd.pid ` % nc localhost 8123 localhost [127.0.0.1] 8123 (?) : Connection refused % twistd -f otk-shutdown.tap % nc localhost 8123 3 %
Notice how the saved .nextkey attribute was restored, and the
application picks up where it left off.
To do this right, you'll want to follow the sequence described by the
writing plugins document. Instead of writing a short
program that creates
a .tap file (by creating an Application, doing various .listenTCPs
on it, then calling .save), you will write a subroutine called
updateApplication(). This subroutine should take a bunch of config arguments
(using the usage.Options class described in the plugins document) and use
them to create Factories and feed them to .listenTCP on an existing
Application instance.
With that in place, and a few files to register this new server you've
created, a utility program called 'mktap' can relieve you of the
business of gathering user arguments and creating the app instance.
mktap can use the Options subclass you define in your build-a-tap
class to figure out what arguments are legal (--port taking a
number, --quotes taking a filename, etc), provide --help
with a list of valid arguments, and parse everything the user passes in
argv[]. It creates the Application, then passes the app and the
parsed options to your updateApplication() method, where you do the
server-specific creation of a Factory and the various listenTCP
calls. Then mktap saves out the .tap file, ready for
starting by twistd.
The end result is that installing your new server is simplified to the following steps:
/usr/local/lib/python.mktap program, giving it the name of your
module and whatever configuration arguments it requires. Watch it create a
.tap file.twistd to start the server contained in the .tap
file. Pretty easy. At least your users will think so.
And, once your application is defined by the .tap file,
there are other tools that can be used to configure it.
tap2deb is a tool that creates installable Debian .deb
packages from your .tap file, making installation even easier.
The Application object has some other features designed to solve common server needs: