[Bug 1803327] [NEW] iconv no longer transforms UTF8 to CP1252 as it used to under Ubuntu 16.04 LTS.

[Bernard Moreton]

Public bug reported:

Under Ubuntu 16.04 LTS I used code in the form
   unix2dos < in.txt | iconv -f UTF-8 -t CP1252 > out.txt
(as in "man unix2dos", then and also still now under 18.04 LTS)
to transform UTF8 database extract into an RTF file, for onward processing.
This now fails.

inconv now gives output as follows:

echo abc àêü | iconv -f utf-8 -t cp1252
abc ���

echo abc àêü | iconv -f utf-8 -t cp1252//translit
abc ���

echo abc àêü | iconv -f utf-8 -t latin1//translit
abc ���

But inconv WILL transform UTF8 to basic ASCII, if called with TRANLIT:

echo abc àêü | iconv -f utf-8 -t ascii
abc iconv: illegal input sequence at position 4

echo abc àêü | iconv -f utf-8 -t ascii//translit
abc aeu

This is better than not working at all, but the transform to CP1252 used to change accented UTF8 characters to single-byte accented characters 
This is the appropriate behaviour, and should be restored..


(FWIW, recode now cannot transform to CP1252 either ...)

ProblemType: Bug
DistroRelease: Ubuntu 18.04
Package: libc-bin 2.27-3ubuntu1
ProcVersionSignature: Ubuntu 4.15.0-38.41-generic 4.15.18
Uname: Linux 4.15.0-38-generic x86_64
ApportVersion: 2.20.9-0ubuntu7.4
Architecture: amd64
CurrentDesktop: X-Cinnamon
Date: Wed Nov 14 10:53:57 2018
Dependencies:
 gcc-8-base 8.2.0-1ubuntu2~18.04
 libc6 2.27-3ubuntu1
 libgcc1 1:8.2.0-1ubuntu2~18.04
InstallationDate: Installed on 2017-03-09 (614 days ago)
InstallationMedia: Ubuntu 16.04 LTS "Xenial Xerus" - Release amd64 (20160420.1)
SourcePackage: glibc
UpgradeStatus: No upgrade log present (probably fresh install)
modified.conffile..etc.ld.so.conf:
 include /etc/ld.so.conf.d/*.conf
 
 /lib /lib/esql  /usr/local/lib
mtime.conffile..etc.ld.so.conf: 2017-03-21T19:11:53.466413

** Affects: glibc (Ubuntu)
     Importance: Undecided
         Status: New


** Tags: amd64 apport-bug bionic

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to glibc in Ubuntu.
https://bugs.launchpad.net/bugs/1803327

Title:
  iconv no longer transforms UTF8 to CP1252 as it used to under Ubuntu
  16.04 LTS.

Status in glibc package in Ubuntu:
  New

Bug description:
  Under Ubuntu 16.04 LTS I used code in the form
     unix2dos < in.txt | iconv -f UTF-8 -t CP1252 > out.txt
  (as in "man unix2dos", then and also still now under 18.04 LTS)
  to transform UTF8 database extract into an RTF file, for onward processing.
  This now fails.

  inconv now gives output as follows:

  echo abc àêü | iconv -f utf-8 -t cp1252
  abc ���

  echo abc àêü | iconv -f utf-8 -t cp1252//translit
  abc ���

  echo abc àêü | iconv -f utf-8 -t latin1//translit
  abc ���

  But inconv WILL transform UTF8 to basic ASCII, if called with TRANLIT:

  echo abc àêü | iconv -f utf-8 -t ascii
  abc iconv: illegal input sequence at position 4

  echo abc àêü | iconv -f utf-8 -t ascii//translit
  abc aeu

  This is better than not working at all, but the transform to CP1252 used to change accented UTF8 characters to single-byte accented characters 
  This is the appropriate behaviour, and should be restored..

  
  (FWIW, recode now cannot transform to CP1252 either ...)

  ProblemType: Bug
  DistroRelease: Ubuntu 18.04
  Package: libc-bin 2.27-3ubuntu1
  ProcVersionSignature: Ubuntu 4.15.0-38.41-generic 4.15.18
  Uname: Linux 4.15.0-38-generic x86_64
  ApportVersion: 2.20.9-0ubuntu7.4
  Architecture: amd64
  CurrentDesktop: X-Cinnamon
  Date: Wed Nov 14 10:53:57 2018
  Dependencies:
   gcc-8-base 8.2.0-1ubuntu2~18.04
   libc6 2.27-3ubuntu1
   libgcc1 1:8.2.0-1ubuntu2~18.04
  InstallationDate: Installed on 2017-03-09 (614 days ago)
  InstallationMedia: Ubuntu 16.04 LTS "Xenial Xerus" - Release amd64 (20160420.1)
  SourcePackage: glibc
  UpgradeStatus: No upgrade log present (probably fresh install)
  modified.conffile..etc.ld.so.conf:
   include /etc/ld.so.conf.d/*.conf
   
   /lib /lib/esql  /usr/local/lib
  mtime.conffile..etc.ld.so.conf: 2017-03-21T19:11:53.466413

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/1803327/+subscriptions