[Bug 1440450] [NEW] GDB does not honor `set output-base 16`

[Jeffrey Walton]

Public bug reported:

>From below, GDB does not appear to honor `set output-base 16`.

In case there's any ambiguity, I want to see output like
9D53491512D5A24D58A6D71D14A371... or 9D 53 49 15 12 D5 A2 4D 58 A6 D7 1D
14 A3 71.... And I don't want to see
\023\066\236\255\370ř\376\206\215...

**********

$ gdb
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
...
(gdb) file cryptopp-test.exe 
Reading symbols from cryptopp-test.exe...done.
(gdb) set output-radix 16
Output radix now set to decimal 16, hex 10, octal 20.
(gdb) show output-radix
Default output radix for printing of values is 16.
(gdb) r
Starting program: /home/user/cryptopp-test.exe 
...

Program received signal SIGSEGV, Segmentation fault.
CryptoPP::AESNI_Dec_Block(long long __vector&, long long __vector const*, unsigned int) (block=..., subkeys=subkeys at entry=0x7fffffffdc60, 
    rounds=rounds at entry=0x7fff) at rijndael.cpp:1040
1040            block = _mm_aesdec_si128(block, subkeys[i+1]);
(gdb) where
#0  CryptoPP::AESNI_Dec_Block(long long __vector&, long long __vector const*, unsigned int) (block=..., subkeys=subkeys at entry=0x7fffffffdc60, 
    rounds=rounds at entry=0x7fff) at rijndael.cpp:1040
#1  0x000000000046dff2 in CryptoPP::AESNI_AdvancedProcessBlocks<void (*)(long long __vector&, long long __vector const*, unsigned int), void (*)(long long __vector&, long long __vector&, long long __vector&, long long __vector&, long long __vector const*, unsigned int)>(void (*)(long long __vector&, long long __vector const*, unsigned int), void (*)(long long __vector&, long long __vector&, long long __vector&, long long __vector&, long long __vector const*, unsigned int), long long __vector const*, unsigned int, unsigned char const*, unsigned char const*, unsigned char*, unsigned long, unsigned int) (
    func1=func1 at entry=0x46dad4 <CryptoPP::AESNI_Dec_Block(long long __vector&, long long __vector const*, unsigned int)>, 
    func4=func4 at entry=0x46db3c <CryptoPP::AESNI_Dec_4_Blocks(long long __vector&, long long __vector&, long long __vector&, long long __vector&, long long __vector const*, unsigned int)>, subkeys=<optimized out>, rounds=<optimized out>, 
    inBlocks=0x6ff908 "\023\066\236\255\370ř\376\206\215\324\332fEQ\342\310\337>;\017/\020\335\036\005U\240\060 ", <incomplete sequence \361>, ...
...

** Affects: gdb (Ubuntu)
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to gdb in Ubuntu.
https://bugs.launchpad.net/bugs/1440450

Title:
  GDB does not honor `set output-base 16`

Status in gdb package in Ubuntu:
  New

Bug description:
  From below, GDB does not appear to honor `set output-base 16`.

  In case there's any ambiguity, I want to see output like
  9D53491512D5A24D58A6D71D14A371... or 9D 53 49 15 12 D5 A2 4D 58 A6 D7
  1D 14 A3 71.... And I don't want to see
  \023\066\236\255\370ř\376\206\215...

  **********

  $ gdb
  GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
  Copyright (C) 2014 Free Software Foundation, Inc.
  ...
  (gdb) file cryptopp-test.exe 
  Reading symbols from cryptopp-test.exe...done.
  (gdb) set output-radix 16
  Output radix now set to decimal 16, hex 10, octal 20.
  (gdb) show output-radix
  Default output radix for printing of values is 16.
  (gdb) r
  Starting program: /home/user/cryptopp-test.exe 
  ...

  Program received signal SIGSEGV, Segmentation fault.
  CryptoPP::AESNI_Dec_Block(long long __vector&, long long __vector const*, unsigned int) (block=..., subkeys=subkeys at entry=0x7fffffffdc60, 
      rounds=rounds at entry=0x7fff) at rijndael.cpp:1040
  1040            block = _mm_aesdec_si128(block, subkeys[i+1]);
  (gdb) where
  #0  CryptoPP::AESNI_Dec_Block(long long __vector&, long long __vector const*, unsigned int) (block=..., subkeys=subkeys at entry=0x7fffffffdc60, 
      rounds=rounds at entry=0x7fff) at rijndael.cpp:1040
  #1  0x000000000046dff2 in CryptoPP::AESNI_AdvancedProcessBlocks<void (*)(long long __vector&, long long __vector const*, unsigned int), void (*)(long long __vector&, long long __vector&, long long __vector&, long long __vector&, long long __vector const*, unsigned int)>(void (*)(long long __vector&, long long __vector const*, unsigned int), void (*)(long long __vector&, long long __vector&, long long __vector&, long long __vector&, long long __vector const*, unsigned int), long long __vector const*, unsigned int, unsigned char const*, unsigned char const*, unsigned char*, unsigned long, unsigned int) (
      func1=func1 at entry=0x46dad4 <CryptoPP::AESNI_Dec_Block(long long __vector&, long long __vector const*, unsigned int)>, 
      func4=func4 at entry=0x46db3c <CryptoPP::AESNI_Dec_4_Blocks(long long __vector&, long long __vector&, long long __vector&, long long __vector&, long long __vector const*, unsigned int)>, subkeys=<optimized out>, rounds=<optimized out>, 
      inBlocks=0x6ff908 "\023\066\236\255\370ř\376\206\215\324\332fEQ\342\310\337>;\017/\020\335\036\005U\240\060 ", <incomplete sequence \361>, ...
  ...

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/gdb/+bug/1440450/+subscriptions