messages log file corrupted

[Smoot Carl-Mitchell]

On Fri, 2009-03-20 at 10:42 -0700, Ray Parrish wrote:
> Smoot Carl-Mitchell wrote:
> > On Fri, 2009-03-20 at 06:54 -0700, Ray Parrish wrote:

> > I speculate some non-printable characters may be in the logs which is
> > interfering with the display in the log viewer.  If you look at the log
> > with vim, these characters will show up in the editor.
> >   
> Thank you! After installing vim, and all of it's companion packages, I 
> used the gui version to edit my messages file, and it now loads in Gedit 
> with no complaints. There were about three lines of ^@ characters in the 
> file. Actually the @ was slightly different but I don't have any way to 
> represent it's actual appearance.

^@ represents ASCII NULL .e.g 0x00. Vim uses a consistent representation
for non-printable ASCII characters.  A character displayed as ^@ means
the character is the same as '@' (0x40) (see man ascii) with the high
order (7th bit) set to zero instead of 1. Please note, the character set
is 7-bit.  ASCII character encoding in a byte always have the 8th bit
set to 0. Linux uses ISO 8859-1 by default which is an 8-bit encoding
scheme.  However the lower half of ISO 8859-1 is just the ASCII
character set and those characters are the ones mostly commonly seen.
You can see the ISO 8859-1 characters with "man ISO 8859-1". Some folks
set the character encoding to UTF-8, which is an efficient variable byte
encoding scheme.  But ASCII is again a subset of UTF-8, so the same vim
display rules apply.

NULL in text files is problematic, since most programs think a NULL byte
means the end of a string.  That is why the log viewer is not displaying
the log correctly.  The NULL in strings problem should be fixed by
changing the representation of a string, but there is a lot of legacy
code which assumes strings are terminated with a NULL byte.
-- 
Smoot Carl-Mitchell
Computer Systems and
Network Consultant
smoot at tic.com
+1 480 922 7313
cell: +1 602 421 9005