[Bug 839609] Re: [11.10 beta1] UnicodeDecodeError crash on simplified chinese input of fake command

[Dennis Chua]

More effort put into this problem yielded a likely solution (i.e.
'hack'). First of all, the Python Exception was

     UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in
position 0: ordinal not in range(128)

This was with the bogus Simplified Chinese command 我。The expected way
command-not-found should have handled this would be something like:


root at u-VirtualBox:~# mgcc
未找到 'mgcc' 命令，您要输入的是否是：
 命令 'mlcc' 来自于包 'mlterm-tools' (universe)
 命令 'cgcc' 来自于包 'sparse' (multiverse)
 命令 'gcc' 来自于包 'gcc' (main)
 命令 'gcc' 来自于包 'pentium-builder' (universe)
mgcc：找不到命令


Now the hack involves updating two Python files in the package:

(1) /usr/lib/command-not-found (line 24) : 
    cnf.install(unicode=True)  ==> cnf.install(unicode=False)

(2) /usr/share/pyshared/CommandNotFound/util.py (line 9):
    _ = gettext.translation("command-not-found", fallback=True).ugettext ==>
    _ = gettext.translation("command-not-found", fallback=True).lgettext

With these edits in place, command-not-found can now handle the test
case:


root at u-VirtualBox:~# 我
我：找不到命令

root at u-VirtualBox:~# mgcc
未找到 'mgcc' 命令，您要输入的是否是：
 命令 'mlcc' 来自于包 'mlterm-tools' (universe)
 命令 'cgcc' 来自于包 'sparse' (multiverse)
 命令 'gcc' 来自于包 'gcc' (main)
 命令 'gcc' 来自于包 'pentium-builder' (universe)
mgcc：找不到命令

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to command-not-found in Ubuntu.
https://bugs.launchpad.net/bugs/839609

Title:
  [11.10 beta1] UnicodeDecodeError crash on simplified chinese input of
  fake command

Status in “command-not-found” package in Ubuntu:
  Confirmed

Bug description:
  The command-not-found package crashes on input of a simplified chinese
  character representing a bogus command. The problem was found with in
  11.10 beta1, for both the x86/i386 and amd64 systems. Debugging the
  python script in /usr/lib/command-not-found shows that a
  UnicodeDecodeError is thrown. The crash_guard() callback framework
  catches this and reports the error.

  Here are further observations.

  (1) With the same simplified chinese input, 11.04 handles the test case gracefully, returning a message
      explaining that the command is not found.

  (2) Between these two series, python has change: 11.04 (Python 2.7.1+)
  versus 11.10 beta1 (Python 2.7.2+).

  To elaborate on this problem, the following files have been included:

  (1) Screen shots showing step-by-step how to reproduce the bug. As switching to Simplified Chinese is
      difficult to explain in words, a video was taken to show how this process.

  (2) A screen shot showing /usr/lib/command-not-found script traced by means of the Python pdb module.
      This shows the zh_CN.UTF-8 byte stream input and the point where UnicodeDecodeError is thrown.

  This issue was investigated in 11.10 beta1 host running in VirtualBox.

  ===

  Taken from To_Reproduce_Bug.txt attachment.

  01_After_ISO_Installation.png   The VirtualBox VM with default English
  locale.

  02_Open_Lanugage_Support.png	Prepare to switch to Simplified Chinese locale.
  				See the accompanying video for this process.

  03_Enable_IBUS_Pinyin.png	After switching to Simplified Chinese. Note the locale 
  				environment variables. Click the IBUS keyboard icon 
  				and select Pinyin input.

  04_Pinyin_Enabled.png           Ready for Pinyin input. Note the blue
  IBUS icon.

  05_Type_Phonetic_Pinyin.png	Type in two letters: 'w' followed by 'o'. Phonetically these
  				correspond to the Chinese character representing 'I' or 'Myself'.
  				IBUS displays options. You want the first one. Hit the space
  				bar to choose it.

  06_Chinese_Input_Complete.png	Chinese 'wo' in zh_CN.UTF-8 is ready to be passed to the Bash.
                                  Hit the return key to do so.

  07_Crash_command_not_found.png  Bash calls command-not-found, which
  can't handle the input.

  08_Disable_Pinyin_Input.png     Instruct IBUS to disable Simplified
  Chinese input.

  ProblemType: Bug
  DistroRelease: Ubuntu 11.10
  Package: command-not-found 0.2.43ubuntu1 [modified: usr/lib/command-not-found]
  ProcVersionSignature: Ubuntu 3.0.0-9.15-generic 3.0.3
  Uname: Linux 3.0.0-9-generic x86_64
  Architecture: amd64
  Date: Fri Sep  2 10:23:08 2011
  InstallationMedia: Ubuntu 11.10 "Oneiric Ocelot" - Beta amd64 (20110901)
  PackageArchitecture: all
  SourcePackage: command-not-found
  UpgradeStatus: No upgrade log present (probably fresh install)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/command-not-found/+bug/839609/+subscriptions