[Bug 1785026] Re: [LTCTest][OPAL][OP920] OPAL PRD generated logs is not available in /var/log/opal-prd.log file

[Andrew Cloke]

** Changed in: ubuntu-power-systems
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to skiboot in Ubuntu.
Matching subscriptions: foundations-bugs-skiboot
https://bugs.launchpad.net/bugs/1785026

Title:
  [LTCTest][OPAL][OP920] OPAL PRD generated logs is not available in
  /var/log/opal-prd.log file

Status in The Ubuntu-power-systems project:
  Fix Released
Status in skiboot package in Ubuntu:
  Fix Released
Status in skiboot source package in Bionic:
  Fix Released

Bug description:
  [Impact]
  Without this fix we end up creating /var/log/opal-prd.log file with root user. Hence rsyslogd will not be able to write data to this file. Hence we will not have enough opal-prd logs for debugging.

  [Test Case]
  - install fixed version of opal-prd package
  - stop opal-prd daemon : sudo systemctl stop opal-prd
  - force log rotation : sudo /usr/sbin/logrotate -f /etc/logrotate.conf
  - check whether opal-prd logs /var/log/opal-prd.log file has 644 mode and non root owner.
  - start opal-prd daemon : sudo systemctl start opal-prd : at that point opal-prd should just re-use the log with the proper mode.

  
  [Regression Potential]
  This is just changing owner for /var/log/opal-prd.log file.
  Without this fix, file won't be created and prd logs gets dropped.
  I do not see any possible regression with this fix.

  ----------------

  Original description follows:

  == Comment: #0 - PAVAMAN SUBRAMANIYAM - 2018-06-27 05:00:15 ==

  Install a P9 Open Power Hardware with the latest OP920 Firmware images provided in the following link:
  http://pfd.austin.ibm.com/releasenotes/openpower9/OP920/OP920_1824A/OP920_1824A_RelNote_Main.html

  root at witherspoon:~# cat /etc/os-release
  ID="openbmc-phosphor"
  NAME="Phosphor OpenBMC (Phosphor OpenBMC Project Reference Distro)"
  VERSION="ibm-v2.1"
  VERSION_ID="ibm-v2.1-438-g0030304-r15-0-g19832d3"
  PRETTY_NAME="Phosphor OpenBMC (Phosphor OpenBMC Project Reference Distro) ibm-v2.1"
  BUILD_ID="ibm-v2.1-438-g0030304-r15"
  root at witherspoon:~# cat /var/lib/phosphor-software-manager/pnor/ro/VERSION
  IBM-witherspoon-ibm-OP9-v2.0.3-2.17
          op-build-v2.0.3-2-g6279a04-dirty
          buildroot-2018.02.1-6-ga8d1126
          skiboot-v6.0.4
          hostboot-9df0950-p378630b
          occ-90208bb
          linux-4.16.13-openpower1-pc10362e
          petitboot-v1.7.1-pcc0e2c7
          machine-xml-7cd20a6
          hostboot-binaries-8097b96
          capp-ucode-p9-dd2-v4
          sbe-0b006e7
          hcode-fbe2c7c

  Then check if the OPAL prd generated logs are captured in /var/log
  /opal-prd.log file.

  root at ltc-wspoon11:/var/log# cat opal-prd.log
  root at ltc-wspoon11:/var/log# echo $?
  0

  == Comment: #4 - VASANT HEGDE - 2018-06-27 09:12:48 ==
  root at ltc-wspoon11:~# grep prd /var/log/syslog
  Jun 27 08:00:44 ltc-wspoon11 rsyslogd: file '/var/log/opal-prd.log': open error: Permission denied [v8.32.0 try http://www.rsyslog.com/e/2433 ]

  root at ltc-wspoon11:~# ls -l /var/log
  total 83332
  -rw-r--r--   1 root      root               11833 Jun 15 03:15 alternatives.log
  ....
  -rw-r--r--   1 root      root                   0 May  1 06:25 opal-prd.log
  -rw-r-----   1 syslog    adm              5084615 May  1 02:12 opal-prd.log.1

  Notice that opal-prd.log file is owned by root user while opal-prd.log.1 is owned by syslog.
  That means someone has manually created that log file?

  I deleted opal-prd.log file and restart opal-prd daemon.

  rm /var/log/opal-prd.log

  systemctl restart opal-prd

  head /var/log/opal-prd.log
  Jun 27 08:02:41 ltc-wspoon11 opal-prd: CTRL: Starting PRD daemon
  Jun 27 08:02:41 ltc-wspoon11 opal-prd: I2C: Found Chip: 00000000 engine 3 port 0
  Jun 27 08:02:41 ltc-wspoon11 opal-prd: I2C: Found Chip: 00000000 engine 1 port 2
  Jun 27 08:02:41 ltc-wspoon11 opal-prd: I2C: Found Chip: 00000008 engine 3 port 1
  Jun 27 08:02:41 ltc-wspoon11 opal-prd: I2C: Found Chip: 00000008 engine 1 port 2
  Jun 27 08:02:41 ltc-wspoon11 opal-prd: I2C: Found Chip: 00000000 engine 3 port 1
  Jun 27 08:02:41 ltc-wspoon11 opal-prd: I2C: Found Chip: 00000000 engine 2 port 0
  Jun 27 08:02:41 ltc-wspoon11 opal-prd: I2C: Found Chip: 00000000 engine 1 port 0
  Jun 27 08:02:41 ltc-wspoon11 opal-prd: I2C: Found Chip: 00000008 engine 3 port 0
  Jun 27 08:02:41 ltc-wspoon11 opal-prd: I2C: Found Chip: 00000008 engine 1 port 0

  root at ltc-wspoon11:~# ls -l /var/log
  ...
  -rw-r-----   1 syslog    syslog             80804 Jun 27 08:02 opal-prd.log
  ...

  So looks like its user error.

  -Vasant

  == Comment: #6 - VASANT HEGDE - 2018-07-30 23:16:39 ==
  Looks like logrorate stuff is creating new file with root user. Below change should fix this.

  :/etc/logrotate.d# diff -Naurp opal-prd.org opal-prd
  --- opal-prd.org	2018-07-30 13:04:08.814168026 -0400
  +++ opal-prd	2018-07-31 00:09:04.145154122 -0400
  @@ -5,5 +5,5 @@
    delaycompress
    missingok
    notifempty
  -	create 644 root root
  +	create 644 syslog adm
   }

  -Vasant

  == Comment: #7 - VASANT HEGDE  - 2018-07-30 23:17:38 ==
  Fred,

  Can you look into the proposed change?

  -Vasant

  == Comment: #8 - Frederic Bonnard <FREDERIC at fr.ibm.com> - 2018-07-31 01:18:28 ==
  Hi all,
  thanks Vasant for the investigation. The fix looks good,
  I'm just going to check that on Debian as well.
  I'll let you know asap,

  F.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-power-systems/+bug/1785026/+subscriptions