Directory Structure of Omnia1.1 Appliance

[DeepikaKrishnaiah]

The team has decided to have the following directory structure in appliance,

• input_params
• roles
• test
• tools
• appliance.yml
• appliance_inventory.yml
• host_inventory.yml
• idrac.yml
• ethernet.yml
• powervault.yml
• infiniband.yml

The directory input_params contains files for accepting user inputs specific to the hardware or tool.
The files in input_params are as follows,

• base_vars.yml
• login_vars.yml (This file accepts only usernames and passwords. This file will be encrypted.)
• idrac_vars.yml
• ethernet_vars.yml
• ib_vars.yml
• powervault_vars.yml

The directory structure of the roles in appliance is as follows,

• appliance_common
• appliance_k8s
• appliance_inventory
• mgmt_network
• host_network
• custom_iso
• web_ui
• offline_repo
• ethernet_config
• idrac_config
• ib_config
• powervault_config

[sujit-jadhav]

@lwilson and @j0hnL Please share your opinion on this.

[lwilson]

@DeepikaKrishnaiah, I'm trying to understand the top-level playbook structure.

What will ethernet.yml, idrac.yml, infiniband.yml, and powerstore.yml be responsible for?

Also, what is the difference between appliance_inventory.yml and host_inventory.yml?

[j0hnL]

I agree we should change appliance -> control_plane. Also like the idea to clearly state what the inventory files are for by using idrac_inentory.yaml and switch_inventory.yaml. The rest of the directory structure looks good.

[sujit-jadhav]

@lwilson changing from appliance-> control_plane is fine. In that case should we rename top level appliance.yml to control_plane.yml ?
Appliance is responsible for building management station and managing varies hardware like server, switches and powervaults. We can think of rename like appliance -> management and appliance.yml to management.yml ? However I am still ok with control_plane

appliance_inventory.yml will fetch list of the IP from the dhcp leases. This list of IP will be combination of management IP's of idrac, ethernet, infiniband and powervault . AWX will use appliance_inventory.yml to fetch IP and it will categorize into idrac, ethernet, infiniband and powervault inventories. We cannot rename it to idrac_inventory.yml and a switch_inventory.yml, can we think of some other name? collect_device.yml and collect_host.yml

[DeepikaKrishnaiah]

@lwilson host_inventory.yml is scheduled from awx to create dynamic inventory. If user wants he can also invoke the same.

[lwilson]

@lwilson changing from appliance-> control_plane is fine. In that case should we rename top level appliance.yml to control_plane.yml ?
Appliance is responsible for building management station and managing varies hardware like server, switches and powervaults. We can think of rename like appliance -> management and appliance.yml to management.yml ? However I am still ok with control_plane

appliance_inventory.yml will fetch list of the IP from the dhcp leases. This list of IP will be combination of management IP's of idrac, ethernet, infiniband and powervault . AWX will use appliance_inventory.yml to fetch IP and it will categorize into idrac, ethernet, infiniband and powervault inventories. We cannot rename it to idrac_inventory.yml and a switch_inventory.yml, can we think of some other name? collect_device.yml and collect_host.yml

Thanks for the explanation @sujit-jadhav. Since appliance_inventory is a playbook and not an inventory file, let's rename it so that there isn't confusion. I like collect_device over collect_host, since host is too specific. Could we consider making it more descriptive, maybe collect_device_info.yml or collect_info.yml?

[DeepikaKrishnaiah]

Based on the review comments, we have come up with the below mentioned structure,
Omnia
control_plane (previously appliance)
 input_params
 roles
 test
 tools
 control_plane.yml (previously appliance.yml)
 ethernet.yml
 idrac.yml
 infiniband.yml
 powervault.yml
 collect_device_info.yml (previously appliance_inventory.yml)
 collect_node_info.yml (previously host_inventory.yml)

The directory structure of the roles in control_plane is as follows,
• common (appliance_common)
• control_plane_k8s (appliance_k8s)
• collect_device_info (appliance_inventory)
• mgmt_network
• host_network
• custom_iso
• web_ui
• offline_repo
• ethernet_config
• idrac_config
• ib_config
• powervault_config
• collect_node_info (host_inventory)

Please review the same @lwilson @j0hnL

[sujit-jadhav]

@j0hnL and @lwilson
Here expanded version of omnia/control_plane/roles

• awx_ui --> previously web_ui (wanted to have this in case if we come up with our own ui then we need this)
• cobbler_config --> previously provision
• collect_node_info
• control_plane_k8s
• ethernet_config
• ib_config
• mgmt_network
• powervault_config
• warewulf_config --> if we add warewulf support
• cli_config --> if we add cli workflow (this is extensive work and might need python coding, can be handled in 1.2)
• collect_device_info
• common
• custom_iso
• host_network --> Our own container if other provisioning tools fails
• idrac_config --> iDRAC provisioning
• offline_repo
• tinkerbell_config --> if we add Tinkerbell support

This whole structure will change once move to Galaxy. But we have time for it. We can keep this structure till 1.4
Please share your opinion.

[lwilson]

@sujit-jadhav would it make sense to have a provisioner_config directory, with cobbler, warewulf, and tinkerbell etc. within? That way the directory list doesn't continue to grow. Same thing for network (ethernet and ib)

[sujit-jadhav]

@lwilson we need to finalize the directory structure as soon as possible. All dependent components are holding their changes.

Ansible Documentation

Adhering to roles documentation and best practices of ansible, the team finds the above mentioned structure as the accepted one (rather than having directories within roles).

• common
• collect_device_info
• collect_node_info
• control_plane_cli
• control_plane_device (mgmt_network)
• control_plane_iso (custom_iso)
• control_plane_k8s
• control_plane_repo (offline_repo)
• network_ethernet (ethernet_config)
• network_ib (ib_config)
• network_subnet_manager
• provision_cobbler
• provision_idrac
• provision_tinkerbell
• provision_warewulf
• storage_me4 (powervault ME4)
• storage_me5 (powervault ME5 - future support)
• webui_awx (note we replaced web_ui to webui_awx)
• webui_native (Proposed Dell UI )

[lwilson]

@sujit-jadhav if these directories don't exist under roles, then how will Ansible know how to invoke them? Perhaps I'm missing something about the proposed structure overall.

[sujit-jadhav]

@lwilson, mentioned directories are under roles folder, below is tree format for the same. As per Ansible guidelines we prefer not to have directory within directory inside roles folder e.g. omnia->control_plane->roles->provision->cobbler is not recommended.

Are you recommending some thing like this: omnia->control_plane->provision->roles->cobbler, and omnia->control_plane->network->roles->ethernet with this directory structure omnia->control_plane->control_plane.yml will not detect roles automatically, we will have to give relative path inside control_plane.yml which increases overhead.

Below is our proposal:

[root@localhost omnia_gitlab]# tree -d
.
└── omnia
├── control_plane
│   ├── input_params
│   ├── roles
│   │   ├── collect_device_info
│   │   │   ├── files
│   │   │   ├── tasks
│   │   │   └── vars
│   │   ├── collect_node_info
│   │   │   ├── files
│   │   │   ├── tasks
│   │   │   └── vars
│   │   ├── common
│   │   │   ├── files
│   │   │   ├── tasks
│   │   │   └── vars
│   │   ├── control_plane_cli
│   │   ├── control_plane_device
│   │   ├── control_plane_iso
│   │   │   ├── files
│   │   │   ├── tasks
│   │   │   └── vars
│   │   ├── control_plane_k8s
│   │   │   ├── files
│   │   │   ├── tasks
│   │   │   └── vars
│   │   ├── control_plane_repo
│   │   │   ├── tasks
│   │   │   └── vars
│   │   ├── network_ethernet
│   │   │   ├── tasks
│   │   │   └── vars
│   │   ├── network_ib
│   │   ├── network_subnet_manager
│   │   ├── provision_cobbler
│   │   ├── provision_idrac
│   │   │   ├── files
│   │   │   ├── tasks
│   │   │   └── vars
│   │   ├── provision_tinkerbell
│   │   ├── provision_warewulf
│   │   ├── storage_me4
│   │   ├── storage_me5
│   │   ├── webui_awx
│   │   │   ├── tasks
│   │   │   └── vars
│   │   └── webui_native
│   ├── test
│   │   └── test_vars
│   └── tools
│   └── roles
│   ├── cluster_preperation
│   │   ├── tasks
│   │   └── vars
│   ├── fetch_password
│   │   └── tasks
│   └── hpc_cluster_report
│   ├── tasks
│   └── templates

[lwilson]

@sujit-jadhav I think that will work just fine. Let's go ahead with it.

[sujit-jadhav]

Here is final structure we are going ahead:

• collect_device_info (host_inventory)
• collect_node_info (appliance_inventory)
• control_plane_cli
• control_plane_common (common)
• control_plane_customiso (created by control plane role, custom_iso)
• control_plane_device (mgmnt_network)
• control_plane_ib (ifiniband DHCP container)
• control_plane_k8s
• control_plane_repo (offline_repo)
• control_plane_sm (subnet_manager container under control plane)
• network_ethernet (ethernet_config)
• network_ib (ib_config)
• provision_cobbler
• provision_idrac
• provision_tinkerbell
• provision_warewulf
• powervault_me4 (powervault, we removed storage as in future we might have storage_lustre and storage_beefgs
• powervault_me5
• webui_awx
• webui_native