Re: How to make results of “localedef” survive “dpkg-reconfigure locales”?

[Tom H]

On Tue, Feb 26, 2019 at 8:31 AM Thorsten Schöning <tschoening at am-soft.de> wrote:
> am Montag, 25. Februar 2019 um 20:11 schrieben Sie:


>> if [ -z "$1" ] && [ -z "$KEEP" ]; then
>>         # Remove all old locale dir and locale-archive before generating new
>>         # locale data.
>>         rm -rf /usr/lib/locale/locale-archive || true
>>         for dir in /usr/lib/locale/*; do
>>                 [ -e "$dir" ] || continue
>>                 if [ "${dir#/usr/lib/locale/}" = C.UTF-8 ]; then
>>                         # owned by libc-bin
>>                         continue
>>                 fi
>>                 rm -rf "$dir" 2>/dev/null || true
>>         done
>> fi
>
> Thanks for investigation. Do you know who the maintainer for this
> script is? Should be best to talk to them why things are deleted,
> while successful recreation in my case simply doesn't work.

The maintainers are the maintainers of glibc.

The Debian version of "locale-gen" has

if [ -z "$KEEP" ]; then
        # Remove all old locale dir and locale-archive before generating new
        # locale data.
        rm -rf /usr/lib/locale/locale-archive || true
fi

so this is an Ubuntu "locale-gen" change after importing from Debian.

Now that I know that it's a shell script, I've checked a Funtoo
installation that I have and it has

if [[ -z ${KEEP} && -z ${UPDATE} ]] ; then
        # Remove all old locale dir and locale-archive before generating new
        # locale data.  Scrubbing via update is done elsewhere.
        rm -rf "${LOCALEDIR}"/* || true
fi

And I've downloaded and checked an Arch package and it has

# Remove all old locale dir and locale-archive before generating new
# locale data.
rm -rf /usr/lib/locale/* || true

So every distribution has its own script that purges currently-defined
locales by default (and, in Arch, only).


"locale-gen" has an option not to purge the existing locales (both in
"locale-archive" and in separate directories) but I don't see any such
option for "update-locale".


>> I added "de_DE.CP1252 CP1251" to "/etc/locale.gen".
>>
>> "de_DE.CP1252" is generated by "locale-gen" and "dpkg-reconfigure locales".
>
> Is the locale recreated as individual directory like with "localedef"
> or embedded into the archive in your case?

It's embedded. "locale-gen" has an option ("--no-archive") not to
embed locale definitions into an archive.

According to the "-h" output, "--no-archive" is the default, but it
isn't because of

ARCHIVE=yes
...
        --archive)
                ARCHIVE=yes
                ;;
        --no-archive)
                ARCHIVE=no
                ;;
...
no_archive=
[ "$ARCHIVE" = yes ] || no_archive="--no-archive"


> Because I did exactly the same, only as stated in "locale-gen" itself
> using the file /usr/local/share/i18n/SUPPORTED, and I as well have
> de_DE.CP1252 available in "dpkg-reconfigure locales" and "localdef
> --list-archive" shows it's in there.

I'm able to generate locales without "/usr/share/i18n/SUPPORTED",
"/usr/local/share/i18n/SUPPORTED" (which is not referred to in the
script), or "/var/lib/locales/supported.d/*" because "locale-gen" has

LOCALEGEN=/etc/locale.gen
...
SUPPORTED=/var/lib/locales/supported.d
...
if [ -z "$1" ]; then
  GENERATE="`cat $SUPPORTED/* $LOCALEGEN 2>/dev/null || true`"
else
  GENERATE=
  while [ -n "$1" ]; do
    if [ -f "$SUPPORTED/$1" ]; then
      GENERATE="$GENERATE\n`cat $SUPPORTED/$1`"
    elif [ $IS_LANG = no ] && L=`grep "^${1/%.utf8/.UTF-8} "
/usr/share/i18n/SUPPORTED`; then
      # convert utf8 to UTF-8 and check if the requested locale exists
      GENERATE="$GENERATE\n$L"
      add_to_locale_gen "$L"
    elif [ $IS_LANG = no ] && L=`grep "^${1%.[uU][tT][fF]*} UTF-8"
/usr/share/i18n/SUPPORTED`; then
      # recognize 'll_CC.UTF-8' or 'll_CC.utf8' as ll_CC without codeset
      GENERATE="$GENERATE\n$L"
      add_to_locale_gen "$L"
    else
      # try to come up with a sensible default
      L=`grep -E "^${1}( |[._@][^[:space:]]* )UTF-8"
/usr/share/i18n/SUPPORTED || true`
      if [ -z "$L" ]; then
        echo "Error: '$1' is not a supported language or locale" >&2
        exit 1
      fi
      GENERATE="$GENERATE\n$L"
      add_to_locale_gen "$L"
    fi
    shift
  done
fi

So you can edit "/etc/locale.gen", "/usr/share/i18n/SUPPORTED", or
"/var/lib/locales/supported.d/some_file" to ensure that your locale's
generated.


> BUT Postgres simply still doesn't work, that only works if I use
> "localedef" to create the new locale as an individual archive. And
> that's why it's problematic if that directory gets deleted without
> properly recreation.
>
> Might this be some Postgres-specific handling of locales for some
> reason? I'm gonna to ask them if they behave specially.

Are you saying that if you create an unembedded locale, postgres picks
it up (with or without restarting it?!), but if you create an embedded
locale, postgres doesn't pick it up?