ONTAP Mediator 1.9.1 installation hangs on RHEL 8.2 (Azure VM) with "writing RSA key"

[Samet]

Hello, 
I am having installation hang problem while installing ONTAP Mediator 1.9.1

How I try to install:
Extracted installer archive /opt/ontap/
and on /opt/ontap/ontap-mediator-1.9.1 run:
./ontap-mediator-1.9.1

last part of the installation log:

[2025-06-26 12:37:53]
[2025-06-26 12:37:53] If you want SCST to start automatically at boot time, run the following command:
[2025-06-26 12:37:53] systemctl enable scst.service
[2025-06-26 12:37:53]
[2025-06-26 12:37:53] make[1]: Leaving directory '/opt/ontap/ontap-mediator-1.9.1/ontap_mediator.Y5g7HJ/ontap-mediator-1.9.1/ontap-mediator-1.9.1/dist/scst/scst-3.8.0/scstadmin'
[2025-06-26 12:37:56] Saving SCST mod keys
[2025-06-26 12:37:56] Installing ONTAP Mediator server packages
[2025-06-26 12:37:56] Installing dist/rpm/ontap-mediator-*el8*
[2025-06-26 12:37:56] Preparing packages...
[2025-06-26 12:37:56] ontap-mediator-1.9.1-16.el8.noarch
[2025-06-26 12:38:25] Finalizing ontap_mediator module
[2025-06-26 12:38:25] Creating mediator service acount + group: netapp:netapp
[2025-06-26 12:38:25] Modifying ownership and permissions to mediator service acount + group: netapp:netapp
[2025-06-26 12:38:25] Configuring and setting ownership for mailbox_directory: /mnt/iscsi_space
[2025-06-26 12:38:26] Checking for pre-existing modification to pyenv/bin/uwsgi SElinux context...
[2025-06-26 12:38:26] No pre-existing modification found. Modifying context...
[2025-06-26 12:38:29] pyenv/bin/uwsgi context modified!
[2025-06-26 12:38:29] Adding Subject Alternative Names to the self-signed server certificate
[2025-06-26 12:38:29] #
[2025-06-26 12:38:29] # OpenSSL example configuration file.
[2025-06-26 12:38:29] Generating self-signed certificates
[2025-06-26 12:38:29] For root_ca.key:
[2025-06-26 12:38:29] Generating RSA private key, 4096 bit long modulus (2 primes)
[2025-06-26 12:38:30] .................................................................................................................++++
[2025-06-26 12:38:30] .........................................++++
[2025-06-26 12:38:30] e is 65537 (0x010001)
[2025-06-26 12:38:30] writing RSA key
[2025-06-26 12:38:30] For intermediate.key:
[2025-06-26 12:38:30] Generating RSA private key, 4096 bit long modulus (2 primes)
[2025-06-26 12:38:30] ..................++++
[2025-06-26 12:38:30] ..........................................................................................................................++++
[2025-06-26 12:38:30] e is 65537 (0x010001)
[2025-06-26 12:38:30] writing RSA key

What I've Already Tried

• Fresh RHEL 8.2 Azure VM deployment (multiple times)

• Installed all pre-req RPMs offline:

  • python39, python39-libs, python39-devel

  • python39-pip-wheel, python39-setuptools-wheel

  • chkconfig-1.19.2 (since alternatives was a missing dep)

• Verified entropy is not the issue (e.g., /dev/random, getrandom() syscall monitored via strace)

• getenforce was set to Enforcing, later changed to Permissive

• Still, full installer hangs again during the "writing RSA key" phase

 Questions for the Community

• Has anyone else seen the RSA generation hang on RHEL 8.2 / Azure specifically?

• Is there a known bug in Mediator 1.9.1 related to the cert gen script?

• Could there be an issue with OpenSSL or Azure's VM entropy model, even though it's not /dev/random related?

• Is there a fully manual install method that bypasses the RSA keygen part?
  
  

Thank you in advance.

[Samet · Accepted solution]

Hi all,

I'd like to share detailed findings and progress on the ONTAP Mediator 1.9.1 installation issue I previously posted about, where the installer was hanging indefinitely during RSA key generation on a RHEL 8.2 Azure VM.

Root Issue Observed:

The installation process repeatedly hangs at the openssl genrsa phase, specifically:

"Generating a RSA private key
................................................................................++++
.....................................++++
writing new private key to 'ontap_mediator_server.key'"

During this time, system CPU usage spikes, and the following processes run continuously:

cat

tr

fold

These are related to the certificate-generation logic in make_self_signed_certs.sh.

System Environment:

Fresh RHEL 8.2 Azure VM

OpenSSL version: OpenSSL 1.1.1c FIPS 28 May 2019

Python 3.9 and dependencies installed manually via RPMs (offline install)

SELinux set to enforcing=0 before installation

No active entropy starvation; confirmed key generation works outside installer

Steps Taken Before Final Success:

Removed all previous mediator remnants:

rm -rf /opt/netapp /opt/ontap

sudo /usr/local/bin/uninstall_ontap_mediator 

Rebooted VM

Re-ran the installer with:

./ontap-mediator-1.9.1 --skip-code-signature-check --skip-yum-dependencies -y

Installer initially reported ontap_mediator.service failure, but auto-retries eventually started the service successfully.

Final State:

ontap_mediator.service: running

mediator-scst.service: running

Web server started (uWSGI logs present)

Installer finished and moved logs to:
/opt/netapp/lib/ontap_mediator/log/install_<timestamp>.log

RSA certificate generation eventually passed (possibly due to retries)

Workaround:
Observed and discovered running processes which was stuck during installation with "top" command 
Retrieved and terminated processes & installation script continued with next installation steps
ps aux | grep -E 'cat|fold|tr'
kill -9 <PID>

Remaining Concerns:

The fact that the installation hangs for indefinitely with high CPU usage indicates a potential bug or inefficiency in make_self_signed_certs.sh — especially the handling of I/O with cat | tr | fold pipe chains.

The script does not fail gracefully, making it hard to debug without killing and re-attempting the installation.

This issue is reproducible across freshly provisioned RHEL 8.2 Azure VMs.

Request:

I’d appreciate if a NetApp engineer could:

Confirm whether this is a known bug in 1.9.1 (and fixed in a later version).

Suggest a clean workaround to avoid the hang without manual intervention.

Clarify whether the certificate generation script could be pre-patched or executed independently prior to install.

Thanks to anyone who reads this.