yt.utilities.poster.streaminghttp module

Streaming HTTP uploads module.

This module extends the standard httplib and urllib2 objects so that iterable objects can be used in the body of HTTP requests.

In most cases all one should have to do is call register_openers() to register the new streaming http handlers which will take priority over the default handlers, and then you can use iterable objects in the body of HTTP requests.

N.B. You must specify a Content-Length header if using an iterable object since there is no way to determine in advance the total size that will be yielded, and there is no way to reset an iterator.

Example usage:

>>> from StringIO import StringIO
>>> import urllib2, poster.streaminghttp
>>> opener = poster.streaminghttp.register_openers()
>>> s = "Test file data"
>>> f = StringIO(s)
>>> req = urllib2.Request("http://localhost:5000", f,
...                       {'Content-Length': str(len(s))})
class yt.utilities.poster.streaminghttp.StreamingHTTPConnection(host, port=None, timeout=<object object>, source_address=None)[source]

Bases: yt.utilities.poster.streaminghttp._StreamingHTTPMixin, http.client.HTTPConnection

Subclass of httplib.HTTPConnection that overrides the send() method to support iterable body objects

auto_open = 1

Close the connection to the HTTP server.


Connect to the host and port specified in __init__.

debuglevel = 0
default_port = 80

Indicate that the last header line has been sent to the server.

This method sends the request to the server. The optional message_body argument can be used to pass a message body associated with the request. The message body will be sent in the same packet as the message headers if it is a string, otherwise it is sent as a separate packet.


Get the response from the server.

If the HTTPConnection is in the correct state, returns an instance of HTTPResponse or of whatever object is returned by class the response_class variable.

If a request has not been sent or if a previous response has not be handled, ResponseNotReady is raised. If the HTTP response indicates that the connection should be closed, then it will be closed before the response is returned. When the connection is closed, the underlying socket is closed.

mss = 16384
putheader(header, *values)

Send a request header line to the server.

For example: h.putheader(‘Accept’, ‘text/html’)

putrequest(method, url, skip_host=0, skip_accept_encoding=0)

Send a request to the server.

`method’ specifies an HTTP request method, e.g. ‘GET’. `url’ specifies the object being requested, e.g. ‘/index.html’. `skip_host’ if True does not add automatically a ‘Host:’ header `skip_accept_encoding’ if True does not add automatically an

‘Accept-Encoding:’ header
request(method, url, body=None, headers={})

Send a complete request to the server.


alias of HTTPResponse


Send value to the server.

value can be a string object, a file-like object that supports a .read() method, or an iterable object that supports a .next() method.

set_tunnel(host, port=None, headers=None)

Set up host and port for HTTP CONNECT tunnelling.

In a connection that uses HTTP CONNECT tunneling, the host passed to the constructor is used as a proxy server that relays all communication to the endpoint passed to set_tunnel. This done by sending an HTTP CONNECT request to the proxy server when the connection is established.

This method must be called before the HTML connection has been established.

The headers argument should be a mapping of extra HTTP headers to send with the CONNECT request.

class yt.utilities.poster.streaminghttp.StreamingHTTPRedirectHandler[source]

Bases: urllib.request.HTTPRedirectHandler

Subclass of urllib2.HTTPRedirectHandler that overrides the redirect_request method to properly handle redirected POST requests

This class is required because python 2.5’s HTTPRedirectHandler does not remove the Content-Type or Content-Length headers when requesting the new resource, but the body of the original request is not preserved.

handler_order = 499
http_error_301(req, fp, code, msg, headers)
http_error_302(req, fp, code, msg, headers)
http_error_303(req, fp, code, msg, headers)
http_error_307(req, fp, code, msg, headers)
inf_msg = 'The HTTP server returned a redirect error that would lead to an infinite loop.\nThe last 30x error message was:\n'
max_redirections = 10
max_repeats = 4
redirect_request(req, fp, code, msg, headers, newurl)[source]

Return a Request or None in response to a redirect.

This is called by the http_error_30x methods when a redirection response is received. If a redirection should take place, return a new Request to allow http_error_30x to perform the redirect. Otherwise, raise HTTPError if no-one else should try to handle this url. Return None if you can’t but another Handler might.

class yt.utilities.poster.streaminghttp.StreamingHTTPHandler(debuglevel=0)[source]

Bases: urllib.request.HTTPHandler

Subclass of urllib2.HTTPHandler that uses StreamingHTTPConnection as its http connection class.

do_open(http_class, req, **http_conn_args)

Return an HTTPResponse object for the request, using http_class.

http_class must implement the HTTPConnection API from http.client.

handler_order = 499

Open a StreamingHTTPConnection for the given request


Handle a HTTP request. Make sure that Content-Length is specified if we’re using an iterable value


Register the streaming http handlers in the global urllib2 default opener object.

Returns the created OpenerDirector object.