[BUG] 0.15rc2: DeprecationWarning: struct integer overflow masking is deprecated

[Martin Pool]

On 3/27/07, John Arbash Meinel <john at arbash-meinel.com> wrote:
> Jari Aalto+usenet wrote:
> > * Mon 2007-03-26 John Arbash Meinel <john AT arbash-meinel.com>
> >>>>>  /usr/lib/python2.5/site-packages/bzrlib/dirstate.py:2302: DeprecationWarning: struct integer overflow masking is deprecated
> >>>>>   , st.st_dev, st.st_ino, st.st_mode))[:-1]
> >>>> struct.pack('>LLLLLL') instead of struct.pack('>llllll')
> >> And the change the lines around "struct.pack()" to catch the exception
> >> with something like:
> >>
> >> try:
> >>     return _encode(_pack('>llllll'
> >>         , st.st_size, int(st.st_mtime), int(st.st_ctime)
> >>         , st.st_dev, st.st_ino, st.st_mode))[:-1]
> >> except Exception, e:
> >>     print e
> >>     print 'size: %s %s' % (type(st.st_size), st.st_size)
> >>     print 'mtime: %s %s' % (type(st.st_mtime), st.st_mtime)
> >>     ...
> >>     # The same for all parameters, so we can figure out what it is
> >>     # complaining about
> >>
> >> python -Werror /usr/bin/bzr ci -m "..."
> >
> > Here is catch of: "return _encode(_pack('>llllll'"
> >
> > Jari
> >
> > /usr/lib/python2.5/site-packages/bzrlib python -Werror /usr/bin/bzr status dirstate.py
> > struct integer overflow masking is deprecated
> > size : <type 'long'> 107912
> > mtime: <type 'float'> 1174919444.0
> > ctime: <type 'float'> 1174919444.0
> > dev  : <type 'long'> 1085651769
> > ino  : <type 'long'> 4336604776193321356
> > mode : <type 'int'> 33188
> > modified:
> >   dirstate.py
>
> Well, it is pretty clear that you have 64-bit inodes, and those aren't
> going to fit in a 32-bit struct.pack.
>

> Now, there are 2 things we could do:
>
> 1) int(st.st_ino & 0xFFFFFFFF)
> This would ignore the warning, at the potential of having a erroneous
> cache hit. (If a files inode changed in the upper 32-bits, but the
> mtime, ctime, size, etc did not)
> The chance of that is pretty darn small, so I think it would be plenty
> safe do do in the short term.

So the exposure would be that we somehow get two files whose inodes
differ only in the top word, and which have their other fields the
same, but different contents.  Then you replace one with the other.
On Unix the act of switching the files would give you a new ctime, so
(barring clocks running backwards) it still wouldn't collide.  On
Windows ctime is (I think) the creation time, and unless renaming
resets the mtime the switch could happen.  So it would just hinge on
whether you are likely to ever get inodes that are the same in the
lower 32 bits.

> 2) Longer term, I think we are going for "%lX.%lX...." as the
> formatting, rather than base64.encode(struct.pack('>l...')). It seems to
> be a bit faster to generate, can handle 64 bit entries, and scales down
> to small entries, too. I think that is the proposed change for 0.16.
>
> Thoughts?

So I assume you're doing the &  because we can't change the length of
this field?  Does dirstate really care how long it is?
-- 
Martin