[Bug 2116751] Re: openscap probe_file process consumes excessive resources during CIS scan
Heather Lemon
2116751 at bugs.launchpad.net
Tue Sep 16 19:47:32 UTC 2025
Hi Chris,
In answering your questions -
What are the consequences of this?
- From upstream comments here: https://github.com/OpenSCAP/openscap/pull/1803
"That means that on systems with a small amount of memory it stops sooner that needed and on systems with a very large amount of memory it stops later than needed."
There is later discussions on Github and the conclusion is introducing this new environment variable OSCAP_PROBE_MEMORY_USAGE_RATIO will resolve the concerns here. And it lets the user customize the needed memory ratio, BUT this is something that we didn't do as apart of this patch because we only want the mimimal amount of changes. This is apart of this patch (1/3) https://github.com/OpenSCAP/openscap/pull/1803/commits/bb26be6ffcf2a88c09e4f237516ad71db555158f. This can be updated and added.
Will it generate a backlog of work to process, will it drop events, will it be slower, etc?
- It will throw a warning and fail that specific scanned directory and then it normally continues and finishes successfully.
W: probe_file: Memory usage ratio limit reached! limit=0.100000, current=0.312033, used=573 MB, free=131 MB, total=1837 MB, count of items=1001
Result fail
... continue scanning..
redhat upstream bug report
https://bugzilla.redhat.com/show_bug.cgi?id=1932833
>From this article I will summarize openscap memory-consumption problem.
https://access.redhat.com/articles/6999111
On systems with limited memory, openscap might terminate early or not generate the results file. OpenSCAP stores all collected results in the memory until the scan finishes. In general, we recommend having a system with a memory of at least 2 GB to complete a successful scan. There are multiple factors why memory consumption is so high.
Multiple various ways of dealing with over-consumption of memory listed in this redhat article as a workaround.
** Bug watch added: Red Hat Bugzilla #1932833
https://bugzilla.redhat.com/show_bug.cgi?id=1932833
--
You received this bug notification because you are a member of Ubuntu
Sponsors, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/2116751
Title:
openscap probe_file process consumes excessive resources during CIS
scan
Status in openscap package in Ubuntu:
Fix Released
Status in openscap source package in Jammy:
Incomplete
Status in openscap source package in Noble:
Fix Released
Bug description:
[ Impact ]
probe_file executable consumes all the RAM of the system during a CIS scan with openscap tool.
excessive resource usage running a specific rule which is related to this bug [1]. This has been fixed in OpenSCAP 1.3, while Jammy runs 1.2.17. A fix for this patch has been made [2].
[ Test Plan ]
Steps to Reproduce:
In a Jammy VM:
Have SSSD installed. This is crucial to files being opened and scanned
causing error. [3]
# Download the latest Scap Security Guide
wget https://github.com/ComplianceAsCode/content/releases/tag/v0.1.77/scap-security-guide-0.1.77.tar.gz
gunzip -d scap-security-guide-0.1.77.tar.gz
tar -xf scap-security-guide-0.1.77.tar
cd scap-security-guide-0.1.77/
sudo mkdir -p mkdir /usr/share/xml/scap/ssg/content
cp ssg-ubuntu2204-ds.xml /usr/share/xml/scap/ssg/content/
Consume a portion of RAM with python script. My testing was 1/2 capacity.
```
import time
# Set a target memory usage in gigabytes
target_gigabytes = 4
# Calculate the number of bytes
target_bytes = target_gigabytes * (1024 ** 3)
# Create a list to hold the allocated memory
memory_hog = []
print(f"Allocating {target_gigabytes}GB of memory. This might freeze
your system.")
try:
while True:
# Append a large block of bytes to the list
# Adjust the size of the block (e.g., 1024*1024) to control allocation speed
memory_hog.append(b" " * (1024 * 1024 * 100)) # Allocate 100MB at a time
if sum(len(x) for x in memory_hog) >= target_bytes:
break
print("Allocation complete. Memory held.")
# The script will now hold the allocated memory until you close it.
while True:
time.sleep(1)
except MemoryError:
print("Memory allocation failed. The system ran out of resources.")
except Exception as e:
print(f"An error occurred: {e}")
```
CTRL+C to quit script once finished.
# create 100 users
for i in $(seq 1 100); do sudo useradd -N -g users user$i; echo "user-ubu" | sudo passwd user$i; done
# create 1000 text files
for i in $(seq 1 100); do echo "This is test file number $i." > file$i.txt; 1000 $(id -u user$i); done
# each user opens 100 files and reads it
for i in $(seq 1 1000); do -u user1 file_1.txt 1000 100 & done
--> this will start 100 processes having 100 threads each, which are opening 1000 files each (shared between threads)
# Run oscap in a new terminal at the same time as executing the third step.
oscap xccdf eval --rule xccdf_org.ssgproject.content_rule_file_permissions_ungroupowned --results-arf /tmp/oscap_results.xml /usr/share/xml/scap/ssg/content/ssg-ubuntu2204-ds.xml
# While oscap runs, strace probe_file for some time in a new terminal
timeout 10s strace -fttTvyy -o oscap_10s.strace -s 64 -p <pid of probe_file>
# find probe_file executable
ps -aux | grep probe_file
```
root 6165 0.0 0.0 288064 11264 pts/3 Sl+ 09:12 0:00 /usr/lib/x86_64-linux-gnu/openscap/probe_file
root 6197 52.4 0.1 288064 22248 pts/3 Sl+ 09:12 0:09 /usr/lib/x86_64-linux-gnu/openscap/probe_file
ubuntu 6408 0.0 0.0 9212 2560 pts/5 R+ 09:12 0:00 grep --color=auto probe_file
```
# attach strace
sudo timeout 10s strace -fttTvyy -o oscap_10s20250823.strace -s 64 -p 6197
strace: Process 6197 attached with 5 threads
You can also pull up system monitor and watch memory be consumed.
Or just run htop and sort by memory. [see screenshot]
Once this happens, it becomes laggy and program is slow.
look at logs for errors specifically lstat in strace output.
A crash occurs, but the program still succeeds.
Title Ensure All Files Are Owned by a Group
Rule xccdf_org.ssgproject.content_rule_file_permissions_ungroupowned
FAIL: 304:pthread_timedjoin_np: 0, Success
W: oscap: Can't receive message: 103, Software caused connection abort.
E: probe_file: Invalid value of the `recurse_direction' attribute: -1
E: probe_file: Invalid value of the `recurse_direction' attribute: -1
E: probe_file: Invalid value of the `recurse_direction' attribute: -1
E: probe_file: Invalid value of the `recurse_direction' attribute: -1
E: probe_file: Invalid value of the `recurse_direction' attribute: -1
E: probe_file: Invalid value of the `recurse_direction' attribute: -1
E: probe_file: Invalid value of the `recurse_direction' attribute: -1
E: probe_file: Invalid value of the `recurse_direction' attribute: -1
E: probe_file: Invalid value of the `recurse_direction' attribute: -1
E: probe_file: Invalid value of the `recurse_direction' attribute: -1
E: probe_file: Invalid value of the `recurse_direction' attribute: -1
E: probe_file: Invalid value of the `recurse_direction' attribute: -1
E: probe_file: Invalid value of the `recurse_direction' attribute: -1
E: probe_file: Invalid value of the `recurse_direction' attribute: -1
E: probe_file: Invalid value of the `recurse_direction' attribute: -1
Result error
or something like
Title Ensure All Files Are Owned by a Group
Rule xccdf_org.ssgproject.content_rule_file_permissions_ungroupowned
W: oscap: Obtrusive data from probe!
Result fail
[ Where Problems Could Occur ]
Calculating max ratio could be incorrect.
Return memory check code could fail and as a result not run the scan or exit early.
[ Other Info ]
We will not be fixing noble as there was not a substantial difference
when the patch was applied or not. It was difficult to tell visually
if the patch fixed the memory consumption.
Backport from upstream.
Commands
# show guide for ubuntu
oscap info /usr/share/xml/scap/ssg/content/ssg-ubuntu2204-ds.xml
Openscap test packages
https://launchpad.net/~hypothetical-lemon/+archive/ubuntu/lp2116751-openscap-bugfix/+packages
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1932833
[2] https://github.com/OpenSCAP/openscap/pull/1803
[3] https://documentation.ubuntu.com/server/how-to/sssd/with-active-directory/index.html
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/openscap/+bug/2116751/+subscriptions
More information about the Ubuntu-sponsors
mailing list