python... urllib... offsets...

[ScRapZ_1]

And now for a better description ;)

I'm writing a sort of download manager in python. I've got nearly everything worked out except for this. When I download a file using urllib.urlretrieve() it happily downloads the file, but overwrites whatever had already been previously downloaded. Which means re-downloading everything all over again. Is it possible to start downloading from an offset? What I'm thinking is using os.stat() to find the filesize, then pass that through as the offset to start downloading. There doesnt seem to be anything in the docs about urlretrieve doing this, and I cant find anything suitable. Is there something I can hack up, or another module that can do what I want it to?


from urllib import urlretrieve as getfile

def sofar(numblock, blocksize, totalsize):
print "%i of %i so far" % ((numblock*blocksize)/1024, totalsize/1024)

getfile(remote,local,sofar)


TTFN,
Scrapz :p

[kmj]

HTTP1.1 RFC:
ftp://ftp.isi.edu/in-notes/rfc2616.txt

Skip to section 14.35

I don't think urllib can do what you need, because it doesn't seem to allow you to specify anything in the http header. I think you need to use the "Range" header.

The httplib module lets you specify the header, and it's not hard to use. The sample code given in the docs for that module should give you a good head start.