BigString (reducing peak memory)
Gordon Tyler
gordon at doxxx.net
Wed Nov 16 16:26:44 UTC 2011
This is just a wild idea, but would using Python generator expressions
help with keeping only a small part of a large data file in memory? If you
only ever need to deal with a non-random-access stream of bytes, then a
custom generator could read data from the file in chunks as bytes are
requested. It would have the advantage of not having to make code aware of
the fact that the data is being chunked.
Ciao,
Gordon
More information about the bazaar
mailing list