[Bug 2116751] Re: openscap probe_file process consumes excessive resources during CIS scan

Fri Sep 5 09:25:33 UTC 2025

Heather had added the specific targets before, but ended up removing both plucky and questing, instead of setting a status. 
As mentioned at the beginning of the description, this was fixed in 1.3 and both plucky and questing are already in the newer 1.4 version.
I've set it to Invalid (let me know if it doesn't make sense).
hopefully it is easier to understand now.

** Changed in: openscap (Ubuntu)
       Status: In Progress => Won't Fix

** Changed in: openscap (Ubuntu)
       Status: Won't Fix => Fix Released

** Changed in: openscap (Ubuntu)
       Status: Fix Released => Invalid

** Changed in: openscap (Ubuntu)
     Assignee: Heather Lemon (hypothetical-lemon) => (unassigned)

** Changed in: openscap (Ubuntu Noble)
     Assignee: Heather Lemon (hypothetical-lemon) => (unassigned)

-- 
You received this bug notification because you are a member of Ubuntu
Sponsors, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/2116751

Title:
  openscap probe_file process consumes excessive resources during CIS
  scan

Status in openscap package in Ubuntu:
  Invalid
Status in openscap source package in Jammy:
  In Progress
Status in openscap source package in Noble:
  Won't Fix

Bug description:
  [ Impact ]

  probe_file executable consumes all the RAM of the system during a CIS scan with openscap tool.
  excessive resource usage running a specific rule which is related to this bug [1]. This has been fixed in OpenSCAP 1.3, while Jammy runs 1.2.17. A fix for this patch has been made [2].

  [ Test Plan ]

  Steps to Reproduce:
  In a Jammy VM:

  Have SSSD installed. This is crucial to files being opened and scanned
  causing error. [3]

  # Download the latest Scap Security Guide
  wget https://github.com/ComplianceAsCode/content/releases/tag/v0.1.77/scap-security-guide-0.1.77.tar.gz

  gunzip -d scap-security-guide-0.1.77.tar.gz
  tar -xf scap-security-guide-0.1.77.tar
  cd scap-security-guide-0.1.77/

  sudo mkdir -p mkdir /usr/share/xml/scap/ssg/content

  cp ssg-ubuntu2204-ds.xml /usr/share/xml/scap/ssg/content/

  Consume a portion of RAM with python script. My testing was 1/2 capacity.
  ```
  import time
  # Set a target memory usage in gigabytes
  target_gigabytes = 4
  # Calculate the number of bytes
  target_bytes = target_gigabytes * (1024 ** 3)

  # Create a list to hold the allocated memory
  memory_hog = []

  print(f"Allocating {target_gigabytes}GB of memory. This might freeze
  your system.")

  try:
      while True:
          # Append a large block of bytes to the list
          # Adjust the size of the block (e.g., 1024*1024) to control allocation speed
          memory_hog.append(b" " * (1024 * 1024 * 100)) # Allocate 100MB at a time
          if sum(len(x) for x in memory_hog) >= target_bytes:
              break
      print("Allocation complete. Memory held.")
      # The script will now hold the allocated memory until you close it.
      while True:
          time.sleep(1)
  except MemoryError:
      print("Memory allocation failed. The system ran out of resources.")
  except Exception as e:
      print(f"An error occurred: {e}")
  ```
  CTRL+C to quit script once finished.

  # create 100 users
  for i in $(seq 1 100); do sudo useradd -N -g users user$i; echo "user-ubu" | sudo passwd  user$i; done
  # create 1000 text files
  for i in $(seq 1 100); do echo "This is test file number $i." > file$i.txt; 1000 $(id -u user$i); done
  # each user opens 100 files and reads it
  for i in $(seq 1 1000); do -u user1 file_1.txt 1000 100 & done
    --> this will start 100 processes having 100 threads each, which are opening 1000 files each (shared between threads)

  # Run oscap in a new terminal at the same time as executing the third step.
  oscap xccdf eval --rule xccdf_org.ssgproject.content_rule_file_permissions_ungroupowned  --results-arf /tmp/oscap_results.xml /usr/share/xml/scap/ssg/content/ssg-ubuntu2204-ds.xml

  # While oscap runs, strace probe_file for some time in a new terminal
  timeout 10s strace -fttTvyy -o oscap_10s.strace -s 64 -p <pid of probe_file>

  # find probe_file executable
  ps -aux | grep probe_file
  ```
  root        6165  0.0  0.0 288064 11264 pts/3    Sl+  09:12   0:00 /usr/lib/x86_64-linux-gnu/openscap/probe_file
  root        6197 52.4  0.1 288064 22248 pts/3    Sl+  09:12   0:09 /usr/lib/x86_64-linux-gnu/openscap/probe_file
  ubuntu      6408  0.0  0.0   9212  2560 pts/5    R+   09:12   0:00 grep --color=auto probe_file
  ```
  # attach strace
  sudo timeout 10s strace -fttTvyy -o oscap_10s20250823.strace -s 64 -p 6197
  strace: Process 6197 attached with 5 threads

  You can also pull up system monitor and watch memory be consumed.
  Or just run htop and sort by memory. [see screenshot]
  Once this happens, it becomes laggy and program is slow.

  look at logs for errors specifically lstat in strace output.

  A crash occurs, but the program still succeeds.

  Title   Ensure All Files Are Owned by a Group
  Rule    xccdf_org.ssgproject.content_rule_file_permissions_ungroupowned
  FAIL: 304:pthread_timedjoin_np: 0, Success
  W: oscap:     Can't receive message: 103, Software caused connection abort.
  E: probe_file: Invalid value of the `recurse_direction' attribute: -1
  E: probe_file: Invalid value of the `recurse_direction' attribute: -1
  E: probe_file: Invalid value of the `recurse_direction' attribute: -1
  E: probe_file: Invalid value of the `recurse_direction' attribute: -1
  E: probe_file: Invalid value of the `recurse_direction' attribute: -1
  E: probe_file: Invalid value of the `recurse_direction' attribute: -1
  E: probe_file: Invalid value of the `recurse_direction' attribute: -1
  E: probe_file: Invalid value of the `recurse_direction' attribute: -1
  E: probe_file: Invalid value of the `recurse_direction' attribute: -1
  E: probe_file: Invalid value of the `recurse_direction' attribute: -1
  E: probe_file: Invalid value of the `recurse_direction' attribute: -1
  E: probe_file: Invalid value of the `recurse_direction' attribute: -1
  E: probe_file: Invalid value of the `recurse_direction' attribute: -1
  E: probe_file: Invalid value of the `recurse_direction' attribute: -1
  E: probe_file: Invalid value of the `recurse_direction' attribute: -1
  Result  error

  or something like
  Title   Ensure All Files Are Owned by a Group
  Rule    xccdf_org.ssgproject.content_rule_file_permissions_ungroupowned
  W: oscap:     Obtrusive data from probe!
  Result  fail

  [ Where Problems Could Occur ]

  Calculating max ratio could be incorrect.
  Return memory check code could fail and as a result not run the scan or exit early.

  [ Other Info ]

  We will not be fixing noble as there was not a substantial difference
  when the patch was applied or not. It was difficult to tell visually
  if the patch fixed the memory consumption.

  Backport from upstream.

  Commands
  # show guide for ubuntu
  oscap info /usr/share/xml/scap/ssg/content/ssg-ubuntu2204-ds.xml

  Openscap test packages
  https://launchpad.net/~hypothetical-lemon/+archive/ubuntu/lp2116751-openscap-bugfix/+packages

  [1] https://bugzilla.redhat.com/show_bug.cgi?id=1932833
  [2] https://github.com/OpenSCAP/openscap/pull/1803
  [3] https://documentation.ubuntu.com/server/how-to/sssd/with-active-directory/index.html

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/openscap/+bug/2116751/+subscriptions