Omnia BuildStreaM: Catalog-Driven Build Automation

Omnia BuildStreaM provides a comprehensive automation solution for managing infrastructure build workflows. It uses a catalog-driven approach where you define your build requirements in a structured catalog file, and BuildStreaM executes automated pipelines to create and deploy images according to your specifications.

BuildStreaM supports three pipeline types that can be executed through GitLab:

• Build Pipeline: Creates diskless images based on catalog specifications. This pipeline is automatically triggered when the catalog is committed, but can also be executed manually.

• Deploy Pipeline: Deploys built images to target cluster nodes. This pipeline is automatically triggered when the PXE mapping file is updated, but can also be executed manually.

• Clean Pipeline: Removes old Image Groups based on retention policy. This pipeline can be executed only manually.

Note

Do not cancel a running GitLab pipeline or stage. Cancellation prevents some pipeline steps from executing, which leaves the BuildStreaM job in an intermediate, inconsistent state. Note that backend BuildStreaM tasks already in progress will continue running to completion regardless of the cancellation.

Note

BuildStreaM does not support execution of multiple pipelines in parallel or concurrently. Only one pipeline can be executed at a time. Attempting to run multiple pipelines simultaneously may result in unexpected behavior or failures.

BuildStreaM addresses the key challenges in HPC cluster image management:

• Automation: Eliminates manual build and deployment processes

• Integration: Works seamlessly with existing Omnia deployments

• Traceability: Provides complete audit trails for all build operations

To build your own custom workflows, you can use the BuildStreaM REST API. The BuildStreaM API documentation is available at Omnia BuildStreaM API Documentation.

BuildStreaM Documentation

• Step 1: Deploy Omnia Core Container
• Step 2: Create Mapping File with Node Information
• Step 3: Prepare the Omnia Infrastructure Manager
• Step 4: Deploy GitLab for BuildStream
• Step 5: Execute Build Pipeline
• Step 6: Execute Deploy Pipeline
• Step 7: Initialize and Verify Telemetry
• Step 8: Verify Telemetry Services Deployed on the Cluster
• Perform Cleanup Operations
• Retry Pipeline Operations
• Omnia Configuration Tables
• Troubleshooting BuildStreaM Pipeline Issues

If you have any feedback about Omnia documentation, please reach out at omnia.readme@dell.com.