Mysterious Python pyc file corruption problems

[Brian Murray]

On Wed, May 15, 2013 at 03:15:05PM -0700, Brian Murray wrote:
> On Wed, May 15, 2013 at 04:36:56PM -0400, Barry Warsaw wrote:
> > Hello Developers,
> > 
> > I am trying to debug and fix a particularly vexing problem in Python that
> > manifests on Ubuntu in several different ways.  I have a hypothesis about the
> > problem, but there are still some mysteries and I don't know how to reproduce
> > it.  I think I can fix it, but I'm sending this message (and soon, another one
> > to python-dev with more technical details) in the hopes that you might have
> > other ideas about how it can happen, or have a reliable way to reproduce the
> > bug.
> > 
> > The problem can show up in any package, but Brian has started to collect a
> > number of bugs that all seem to be related (and I think Steve is going to open
> > a megabug to dupe them all to).  The common way this manifests is a traceback
> > on an import statement.  The actual error can be a "ValueError: bad marshal
> > data (unknown type code)" such as in LP: #1010077, or an "EOFError: EOF read
> > where not expected" as in LP: #1060842.  We have many more instances of both
> > of these.
> > 
> > Both of these exceptions come from Python's marshal code (marshal.c).  marshal
> > is the low-level serialization protocol used to cache Python byte code into
> > .pyc files, so both of these exception imply corrupt .pyc files, and in fact,
> > the workaround is always to essentially blow away the .pyc file and re-create
> > it.  (Various different techniques can be used, but they all boil down to the
> > same thing.)
> > 
> > Another commonality is that this bug -- so far -- has not been observed in any
> > Python 3.3 code, only 3.2 and earlier, including 2.7 and 2.6.
> 
> I've found one bug where the interpreter is python3.3:
> 
> http://launchpad.net/bugs/1178599
> 
> This looks to be a fresh install of Raring so the apport data seems
> trustworthy.  (There are some apport issues with delayed reporting and
> distribution upgrades.)  I'll be doing a full search of Launchpad bugs
> shortly.

And http://launchpad.net/bugs/1107175

That's all the bugs tagged apport-crash and raring with python3.3 in
InterpreterPath and EOFError or bad marshal data in the apport generated
Title.

Looking both of these up at errors.ubuntu.com using the bug parameter
(yeah!) I only found one example which appears to be the same report as
in bug 1178599.

--
Brian Murray
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <https://lists.ubuntu.com/archives/ubuntu-devel/attachments/20130515/df60715a/attachment.pgp>