Skip to content

Best Practices Checklist

A consolidated checklist of recommendations for deploying and operating an Omnia cluster. Each item links to the relevant documentation section for detailed procedures.

Playbook execution

Best Practice Reference
☐ Run playbooks from their directory using cd. Omnia playbooks use relative paths for roles and configuration files. Always cd /omnia before running ansible-playbook. Index
☐ Review prerequisites before running playbooks. Each playbook has specific input files and environment requirements. Check the corresponding how-to guide before execution. Index

BIOS and firmware

Best Practice Reference
☐ Keep iDRAC firmware current on all nodes. Firmware updates fix bugs, improve Redfish API reliability, and patch security vulnerabilities. Use Dell Repository Manager or racadm for updates.
☐ Disable PowerCap on all compute nodes to ensure maximum CPU performance. PowerCap can throttle compute-intensive HPC workloads. Prerequisites Checklist
☐ Set BIOS to Performance mode on all compute nodes. The Performance power profile maximizes clock speeds and disables power-saving C-states that introduce latency. Prerequisites Checklist

Storage and telemetry

Best Practice Reference
☐ Provision sufficient NFS storage for telemetry. The telemetry pipeline (VictoriaMetrics, Kafka) can generate significant data volumes. Allocate at least 500 GB of NFS-backed persistent storage for telemetry retention. Setup Telemetry
☐ Prefer the PowerScale CSI driver over external NFS for Kubernetes persistent volumes. The CSI driver provides dynamic provisioning, better performance, and snapshot support compared to static NFS mounts. Deploy Powerscale Csi
☐ Ensure external NFS is accessible via the admin network. If using external NFS (not PowerScale), verify that the NFS server is reachable from all nodes on the admin network and that firewall rules allow NFS traffic (ports 2049, 111). Configure Mounts

System administration

Best Practice Reference
☐ Minimize OIM reboots. The OIM hosts critical Podman containers that provide provisioning, DHCP, and management services. Rebooting the OIM interrupts these services and may cause hostname or IP changes. Plan reboots during maintenance windows only. General
☐ Run yum update --security routinely. Apply security patches on the OIM and all cluster nodes on a regular schedule (monthly recommended). Drain Slurm nodes before applying updates.
☐ Sync system time across the OIM and all nodes. Use chrony or ntpd to maintain synchronized clocks. Time drift causes Kerberos authentication failures, Slurm scheduling issues, and inaccurate telemetry timestamps. Prerequisites Checklist

Pre-deployment summary

Use this condensed list as a quick pre-flight check before running any Omnia deployment:

  1. ☐ BIOS: PowerCap disabled, Performance mode set.
  2. ☐ iDRAC firmware updated to the latest supported version.
  3. ☐ Network: admin and BMC VLANs configured, NFS accessible.
  4. ☐ System clocks synchronized (chrony/NTP).
  5. ☐ OIM meets minimum requirements (4+ cores, 32+ GB RAM, 256+ GB disk).
  6. ☐ Input files reviewed and populated (mapping file, config files).
  7. ☐ Credentials encrypted with Ansible Vault.
  8. ☐ Sufficient NFS storage allocated for telemetry.
  9. ☐ Security updates applied: yum update --security.

Info