[Bug 1905393] Re: Ubuntu 20.04: opal-prd fails to start on 20.04

[Andrew Cloke]

** Changed in: ubuntu-power-systems
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to skiboot in Ubuntu.
Matching subscriptions: foundations-bugs-skiboot
https://bugs.launchpad.net/bugs/1905393

Title:
  Ubuntu 20.04: opal-prd fails to start on 20.04

Status in The Ubuntu-power-systems project:
  Fix Released
Status in skiboot package in Ubuntu:
  Fix Released
Status in skiboot source package in Focal:
  Fix Released
Status in skiboot source package in Groovy:
  Fix Released
Status in skiboot source package in Hirsute:
  Fix Released

Bug description:
  [Impact]
  opal-prd (the daemon on a power system that listens for hw diagnostic system events at the /dev/opal-prd device) fails to start.
  The reason is that opal-prd is not able to properly handle devtmpfs, mounted with noexec in /dev, which is the case on recent versions of systemd (like used in focal or newer)..
  Currently such a mount prevents mapping HBRT image code region as 'rwx' from /dev.

  [Fix]
  This patch/commit attempts to work around the situation by copying HBRT image to a non mmapped memory region and sets mprotect rwx on it, allowing opal-prd to successfully execute the code region (as suggested here: https://github.com/open-power/skiboot/issues/258):
  47005e8d4c9aeda5826c17c4a013cfbda1a3f2de 47005e8 "opal-prd: handle devtmpfs mounted with noexec"

  [Test Case]
  Since the opal-prd daemon must be running in the background as a separate process, the test is to:
  - install the updated package that includes the patched opal-prd daemon (e.g. from the PPA mentioned below)
  - double check the installed package version (dpkg -l) and maybe the opal-pd version that's in place (opal-prd --version)
  - start opal-prd as daemon: 'service opal-prd start' (if not started automatically)
  - verify the opal-prd status and check if it's running or not, by for example 'service opal-prd status'

  [What could go wrong]
  Things can go wrong in case the HBRT image copy is done wrong; in case it's accidentally copied to a wrong memory area (e.g. to an already mapped range, or erroneously calculated address/size), a seg. fault will happen and the system would core dump.
  The mprotect code is pretty straight forward, but the fact that mprotect rwx is set on it, allows opal-prd to successfully execute the code region. It's not generally a perfect approach to map memory as RWX, but HBRT requires the ability to write into the image at runtime - and it got upstream accepted that way with skiboot v6.7.
  The fix was released back in October and was pre-tested by the IBM Power team.
  On top a patched Ubuntu package was build and shared in a PPA (see comment #1) and again successfully validated on focal and groovy.
  __________

  [Original Description]

  == Comment: #0 - VASANT HEGDE <hegdevasant at in.ibm.com> - 2020-11-23 23:23:22 ==
  ---Problem Description---
  opal-prd fails to start on 20.04

  Contact Information = Vasant hegde <hegdevasant at linux.vnet.ibm.com>

  ---uname output---
  Ubuntu 20.04

  Machine Type = All Power System

  ---Steps to Reproduce---
   opal-prd fails to start on 20.04

  Userspace tool common name: opal-prd

  The userspace tool has the following bit modes: 64bit

  Userspace rpm: opal-prd

  This is fixed in upstream by below commit. Please backport this patch
  to 20.04 LTS release. Also applicable for 20.10.

  commit 47005e8d4c9aeda5826c17c4a013cfbda1a3f2de
  Author: Georgy Yakovlev <gyakovlev at gentoo.org>
  Date:   Mon Oct 12 14:29:17 2020 -0700

      opal-prd: handle devtmpfs mounted with noexec

      On systems using recent versions of systemd /dev (devtmpfs) is mounted with
      noexec option. Such mount prevents mapping HBRT image code region as RWX
      from /dev. This commit, as suggested in github PR linked below, attempts to
      work around the situation by copying HBRT image to anon mmaped memory
      region and sets mprotect rwx on it, allowing opal-prd to sucessfully
      execute the code region.

      Having memory region set as RWX is not ideal for security, but fixing that
      is a separate and hard to solve problem. Original code also mmaped region
      as RWX, so this PR does not make things worse at least.

      Closes: https://github.com/open-power/skiboot/issues/258
      Signed-off-by: Georgy Yakovlev <gyakovlev at gentoo.org>
      Reviewed-by: Vasant Hegde <hegdevasant at linux.vnet.ibm.com>
      [oliver: whitespace fix, add a comment, reflow commit message]
      Signed-off-by: Oliver O'Halloran <oohall at gmail.com>

  -Vasant

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-power-systems/+bug/1905393/+subscriptions