Creating and working with a telnet server

Run mktap telnet -p 4040 -u admin -w woohoo at your shell prompt. If you list the contents of your current directory, you'll notice a new file -- telnet.tap. After you do this, run twistd -f telnet.tap. Since the Application has a telnet server that you specified to be on port 4040, it will start listening for connections on this port. Try connecting with your favorite telnet utility to 127.0.0.1 port 4040.

$ telnet localhost 4040
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

twisted.protocols.telnet.ShellFactory
Twisted 0.15.5
username: admin
password: ******
>>>

Now, you should see a Python prompt -- >>>. You can type any valid Python code here. Let's try looking around.

>>> dir()
['__builtins__']

Ok, not much. let's play a little more:

>>> import __main__
>>> dir(__main__)
['EverythingEphemeral', 'ServerOptions', '__builtins__', '__doc__', '__name__',
 'application', 'config', 'copyright', 'imp', 'initRun', 'load', 'log', 
'logFile', 'logPath', 'logfile', 'main', 'mainMod', 'oldstderr', 'oldstdin', 
'oldstdout', 'os', 'platformType', 'rotateLog', 'runtime', 'signal', 'string',
 'styles', 'sys', 'traceback', 'usage', 'util']

>>> __main__.application
<telnet app>
>>> dir(__main__.application)
['authorizer', 'connectors', 'delayeds', 'gid', 'name', 'persistenceVersion',
 'ports', 'resolver', 'running', 'services', 'uid', 'written']

From this session we learned that there is an application object stored in __main__ that's a telnet app, and it has some scary attributes that we're not going to worry about for now.

Alright, so now you've decided that you hate Twisted and want to shut it down. Or you just want to go to bed. Either way, I'll tell you what to do. First, disconnect from your telnet server. Then, back at your system's shell prompt, type kill `cat twistd.pid` (the quotes around cat twistd.pid are backticks, not single-quotes). If you list the contents of your current directory again, you'll notice that there will be a file named telnet-shutdown.tap. If you wanted to restart the server with exactly the same state as you left it, you could just run twistd -f telnet-shutdown.tap. This is why Twisted doesn't need any sort of configuration files -- all the configuration data is stored right in the objects!

Now that you've learned how to create a telnet server with 'mktap telnet', we'll delve a little deeper and learn how one is created behind the scenes. Start up a python interpreter and make sure that the 'twisted' directory is in your module search path.

Python 1.5.2 (#0, Dec 27 2000, 13:59:38)  [GCC 2.95.2 20000220 (Debian
GNU/Linux)] on linux2
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> import sys
>>> sys.path.append('/twisted/Twisted')

I installed Twisted in /twisted, so the place where my 'twisted' package directory is at is /twisted/Twisted/twisted (confusing, I know). For Python to find the 'twisted' package, it must have the directory containing the package in sys.path -- which is why I added /twisted/Twisted.

>>> from twisted.internet import app, tcp
>>> from twisted.protocols import telnet
>>> application = app.Application('telnet')
>>> ts = telnet.ShellFactory()
>>> application.listenTCP(4040, ts)

The above is basically what mktap telnet does. First we create a new Twisted Application, we create a new telnet Shell Factory, and we tell the application to listen on TCP port 4040 with the ShellFactory we've created.

Now let's start the application. This causes all ports on the application to start listening for incoming connections. This step is basically what the 'twistd' utility does.

>>> application.run()
twisted.protocols.telnet.ShellFactory starting on 4040

You now have a functioning telnet server! You can connect with your telnet program and work with it just the same as you did before. When you're done using the telnet server, you can switch back to your python console and hit ctrl-C. The following should appear:

Starting Shutdown Sequence.
Stopping main loop.
Main loop terminated.
Saving telnet application to telnet-shutdown.tap...
Saved.
>>>

Your server was pickled up again and saved to the telnet-shutdown.tap file, just like when you did kill `cat twistd.pid`.

More Complicated Configuration

Let's suppose that we have the following application:

manhole1.py

Once this is running, it would be nice to poke around inside it. We can add the manhole-shell by adding a few lines to create a new server (a Factory) listening on a different point:

manhole2.py

With this in place, you can telnet to port 8007, give the username boss and password sekrit, and you'll end up with a shell that behaves very much like the Python interpreter that you get by running python all by itself, with lines you type prefixed with >>>.

% telnet localhost 8007
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

twisted.manhole.telnet.ShellFactory
Twisted 0.99.2
username: boss
password: *****
>>>

Note that the original Quote-Of-The-Day server is still running on port 8123 by using nc localhost 8123 (or telnet localhost 8123 if you don't have netcat installed).

% nc localhost 8123
An apple a day keeps the doctor away.

The initial namespace of the manhole interpreter is defined by a dictionary stored in the 'namespace' attribute of the ShellFactory. For convenience, you can put references to any objects you like in that dict (f.namespace['foo'] = 12), and then retrieve them by name from the telnet session.

>>> foo
12

Of course we can change that namespace by evaluating expressions in the interpreter. To be a useful debugging tool, however, we want to get access to our servers (the protocol Factory instances and everything hanging off of them). We start by gaining access to the main Application instance through a global variable stored in the app module:

>>> import twisted.internet.app         
>>> a = twisted.internet.app.theApplication
>>> a
<'demo' app>

This object holds three things of interest: the list of Delayeds (functions scheduled to run some number of seconds in the future), the list of Services (subclasses of ApplicationService that have been added to the application, most notably Perspective Broker services), and the list of ports on which protocol Factories are listening. The ports are kept in a list, and the Factory object itself is available inside that list (word wrapped for clarity):

>>> a.tcpPorts
[(8123, <twisted.internet.protocol.Factory instance at 0x8249b8c>, 5, ''),
(8007, <twisted.manhole.telnet.ShellFactory instance at 0x824aefc>, 5, '')
]
>>> f = a.tcpPorts[0][1]
>>> f
<twisted.internet.protocol.Factory instance at 0x8249b8c>

Now that we have access to that Factory, what can we do? We can modify any attribute of the object, or call functions on it. Remember that the Factory stores a reference to a subclass of Protocol, and it uses that reference to create new Protocol instances for each new connection. We can change that reference to make the Factory create something else:

>>> f.protocol
<class twisted.protocols.wire.QOTD at 0x824a66c>
>>> from twisted.protocols.wire import Daytime
>>> f.protocol = Daytime

Congratulations, you've just changed the Factory to use the Daytime protocol instead of the QOTD protocol. You have just transformed the QOTD server into a Daytime server. Connect to port 8123 now and see the difference: you get a timestamp instead of a quote:

% nc localhost 8123
Sat Sep 28 09:11:37 2002

From here, you can do anything you want to your application. It is a good idea to check the source for the Application and Service classes to see what else you can extract from them.

Note: to terminate your session, you'll need to exit the telnet or netcat program (the usual control-D that works in the Python interpreter won't work here). Try control-] for telnet. Also note that any exceptions caused by your manhole session will be displayed both in the telnet session and in the stderr on the application side.