revjim.net

Python: better than a hammer, if I could just quit hitting my thumb

May 6th, 2003
by Jim Reverend.

I’ve been learning Python. What initially was simply a quest to make some small alterations to Pyrite BlogAuthor, has become an all out quest to learn Python. I’ve already seen lots of things that I like. I’ve also seen lots of things that I don’t like, and lots of things that could really get me in trouble.

I’ve also found several… bugs… or limitations. First of all, xmlrpclib doesn’t support an HTTP Proxy. That’s annoying. It should. It only makes sense that it should. However, it does allow you to redefine its transport mechanism, which is almost just as good.

So, after scrounging around here and there, I find the following code that uses urllib which supports an HTTP Proxy:

class UrllibTransport(xmlrpclib.Transport):

        def request(self, host, handler, request_body):
                import urllib
                urlopener = urllib.FancyURLopener()
                urlopener.addheaders = [('User-agent', self.user_agent)]
                # probably should use appropriate 'join' methods instead of 'http://'+host+handler
                f = urlopener.open('http://'+host+handler, request_body)
                return(self.parse_response(f))

Now I don’t fully understand the code above, but that’s never stopped me from going forward in the past, so why should it stop me now? I alter my xmlrpclib.ServerProxy line to use it. I go from this:

self._proxy = xmlrpclib.ServerProxy(self.account.target, verbose=0)

To this:

self.transport = UrllibTransport()
self._proxy = xmlrpclib.ServerProxy(self.account.target, self.transport)

I initially included the verbose=0 portion as well. But that led to odd errors. Removing it led to different errors. Eventually, the only way I could seem to get it to work was to add one additional parameter to UrllibTransport.request: verbose=0. Additionally, I had to also add a self.verbose = verbose, though, I don’t really know why.

Unfortunately, urllib is broken. It sees that I am supplying data (in this case the XML-RPC data) and therefore decides, on its own, that I should make a POST request (which is actually fine in this case). Additionally, regardless of how many times you assign anything to urllib.addheaders, it also adds a Content-Type: application/x-www-form-urlencoded header. Now urllib uses httplib to do its HTTP bidding. And, in what I believe is an attempt at flexibility, httplib will join all the values that have the same header with a comma (“,”). Which means that, if I set my own Content-Type to text/xml (as an XML-RPC call should be, I believe), the actual Content-Type that gets sent is applicaton/x-www-form-urlencoded, text/xml. Yuck!

This wouldn’t be much of an issue, except that I am trying to speak with a Perl XML-RPC implementation. And, in the PERL world, if the Content-Type isn’t either NOTHING at all or text/xml, it screams and yells, and then takes its marbles and goes home.

So, in the end, I’ve learned a lot and yet still haven’t been able to do what I want to do.

Any Python gurus have any pointers?

Posted in: Uncategorized.

← →