• VMware

    Learn about VMware virtualization for its products like vsphere ESX and ESXi, vCenter Server, VMware View, VMware P2V and many more

  • Linux

    Step by step configuration tutorials for many of the Linux services like DNS, DHCP, FTP, Samba4 etc including many tips and tricks in Red Hat Linux.

  • Database

    Learn installation and configuration of databases like Oracle, My SQL, Postgresql, etc including many other related tutorials in Linux.

  • Final Part 3: Openstack TripleO Architecture and Step By Step Guide for installation of undercloud and overcloud nodes (compute, controller, ceph-storage)

    Below list of topics are covered in this article

    Bring up undercloud director



    Prepare Undercloud for overcloud deployment






    Phases of overcloud node deployment

    Registration:
    • The "stack" user uploads information about the proposed overcloud nodes
    • The information includes credentials for power management
    • The information is saved in the ironic database and used during the introspection phase.

    Introspection:
    • Ironic connects to the registered nodes to gather more details about the hardware resources.
    • The discovery kernel and ramdisk images are used during this process.

    Deployment:
    • The "stack" user deploys the overcloud nodes, allocating resources and nodes that were discovered during the introspection phase.
    • Hardware profiles and Heat templates are used during the phase.

    Registering an overcloud node

    This consists of adding it to an ironic list for the possible nodes for the overcloud. The undercloud needs the following information to register a node.
    • The type of power  management being used (such as IPMI or PXE over SHH). The various power management drivers supported by iroic can be listed using "ironic driver -list".
    • The node's IP address on the power management network.
    • The credentials to be used for the power management interface.
    • The MAC address for the NIC on the PXE/provisioning network.
    • The kernel and ramdisk that will be used for introspection.

    All of this information can be passed using a JSON file or using a CSV file. The "openstack baremetal import" command imports this file into the ironic database.




    Configure Virtual Bare Metal Controller (VBMC)

    The director can use virtual machines as nodes on a KVM host. It controls their power management through emulated IPMI devices. SInce we have a lab setup using KVM as my setup we will use VBMC to help register the nodes.

    Since we are using virtual machines for our setup which does not has any iLO or similar utlity for power management we will use VBMC.

    Enable the below repo on your "KVM Host" to get the vbmc package.
    [root@openstack ~]# subscription-manager repos --enable=rhel-7-server-openstack-11-rpms

    NOTE: For CentOS you can get the VBMC from the below git repository
    https://git.openstack.org/cgit/openstack/virtualbmc

    Create a virtual bare metal controller (BMC) for each virtual machine using the vbmc command.
    [root@openstack ~]# vbmc add overcloud-ceph.example --port 6320 --username admin --password redhat
    [root@openstack ~]# vbmc add overcloud-controller.example --port 6321 --username admin --password redhat
    [root@openstack ~]# vbmc add overcloud-compute.example --port 6322 --username admin --password redhat

    NOTE: Use a different port for each virtual machine. Port numbers lower than 1025 require root privileges in the system.

    To list the available domains
    [root@openstack ~]# vbmc list
    +------------------------------+--------+---------+------+
    |         Domain name          | Status | Address | Port |
    +------------------------------+--------+---------+------+
    |    overcloud-ceph.example    |  down  |    ::   | 6320 |
    |  overcloud-compute.example   |  down  |    ::   | 6322 |
    | overcloud-controller.example |  down  |    ::   | 6321 |
    +------------------------------+--------+---------+------+

    Next start all the virtual BMCs:
    [root@openstack ~]# vbmc start overcloud-ceph.example
    [root@openstack ~]# vbmc start overcloud-compute.example
    [root@openstack ~]# vbmc start overcloud-controller

    Check the status again
    [root@openstack ~]# vbmc list
    +------------------------------+---------+---------+------+
    |         Domain name          |  Status | Address | Port |
    +------------------------------+---------+---------+------+
    |    overcloud-ceph.example    | running |    ::   | 6320 |
    |  overcloud-compute.example   | running |    ::   | 6322 |
    | overcloud-controller.example | running |    ::   | 6321 |
    +------------------------------+---------+---------+------+

    Now all our domains are in running state.
    NOTE: With VBCM we will use "pxe_ipmitool" as the driver for executing all the IPMI commands so make sure this is loaded and available on your undercloud

    To get the list of supported drivers for IMPI connection
    [stack@undercloud-director ~]$ openstack baremetal driver list
    +---------------------+-----------------------+
    | Supported driver(s) | Active host(s)        |
    +---------------------+-----------------------+
    | pxe_drac            | localhost.localdomain |
    | pxe_ilo             | localhost.localdomain |
    | pxe_ipmitool        | localhost.localdomain |
    | pxe_ssh             | localhost.localdomain |
    +---------------------+-----------------------+

    To check the power status of all our virtual hosts to make sure they are reachable from our undercloud.
    NOTE: Here -H represents our KVM host where all the virtual machines are running
    [stack@undercloud-director ~]$ ipmitool -I lanplus -H 10.43.138.12 -L ADMINISTRATOR -p 6320 -U admin -R 3 -N 5 -P redhat power status
    Chassis Power is off

    [stack@undercloud-director ~]$ ipmitool -I lanplus -H 10.43.138.12 -L ADMINISTRATOR -p 6321 -U admin -R 3 -N 5 -P redhat power status
    Chassis Power is off

    [stack@undercloud-director ~]$ ipmitool -I lanplus -H 10.43.138.12 -L ADMINISTRATOR -p 6320 -U admin -R 3 -N 5 -P redhat power status
    Chassis Power is off



    Configuration Files

    Create a JSON file describing your Overcloud baremetal nodes, call it "instack-threenodes.json" and place in your home directory. The file should contain a JSON object with the only field nodes containing list of node descriptions.

    Each node description should contains required fields:
    • pm_type - driver for Ironic nodes, see Ironic Hardware Types for details
    • pm_addr - node BMC IP address (hypervisor address in case of virtual environment)
    • pm_user, pm_password - node BMC credentials

    These credentials will be used to control the power of the overcloud hypervisors.

    Below is my input json file which I will use to import the nodes into the ironic database
    NOTE: get the MAC Address from your virtual machines and place them respectively in their section in the below json file

    [stack@undercloud-director ~]$ cat instack-threenodes.json
    {
        "nodes":[
            {
                "mac":[
                    "52:54:00:87:37:1f"
                ],
                "name":"overcloud-controller.example",
                "cpu":"4",
                "memory":"10240",
                "disk":"50",
                "arch":"x86_64",
                "pm_type":"pxe_ipmitool",
                "pm_user":"admin",
                "pm_addr": "10.43.138.12",
                "pm_password": "redhat",
                "pm_port": "6321"
            },
            {
                "mac":[
                    "52:54:00:64:36:c6"
                ],
                "name":"overcloud-compute.example",
                "cpu":"4",
                "memory":"10240",
                "disk":"50",
                "arch":"x86_64",
                "pm_type":"pxe_ipmitool",
                "pm_user":"admin",
                "pm_addr": "10.43.138.12",
                "pm_password": "redhat",
                "pm_port": "6322"
            },
            {
                "mac":[
                    "52:54:00:6f:3f:47"
                ],
                "name":"overcloud-ceph.example",
                "cpu":"4",
                "memory":"20240",
                "disk":"50",
                "arch":"x86_64",
                "pm_type":"pxe_ipmitool",
                "pm_user":"admin",
                "pm_addr": "10.43.138.12",
                "pm_port": "6320",
                "pm_password": "redhat"
            }
        ]
    }


    Register Nodes

    Register and configure nodes for your deployment with Ironic:
    [stack@undercloud-director ~]$ openstack baremetal import --json instack-threenodes.json
    Started Mistral Workflow. Execution ID: 3c5dc807-b797-4b09-b6c1-3f6902cd4a26
    Successfully registered node UUID 7995f1f2-4af7-4c5d-9099-fc928c4c73b3
    Successfully registered node UUID 7995f1f2-4af7-4c5d-9099-fc928c4c73b3
    Successfully registered node UUID 7c84cdf2-c5b2-47fb-a741-30c025b54183
    Started Mistral Workflow. Execution ID: 62e4c913-a15e-438f-bedd-5648e2ba1aa0
    Successfully set all nodes to available.

    NOTE: The file to be imported may be either JSON, YAML or CSV format, and the type is detected via the file extension (json, yaml, csv).



    Introspection of overcloud nodes

    For the introspection/discovery of overcloud nodes. Ironic uses PXE provided by the undercloud. The "dnsmasq" is used to provide DHCP and PXE capabilities to the ironic service. The PXE directory images are delivered over HTTP. Prior to introspection, the registered nodes have a valid kernel and ramdisk assigned to them and every node for introspection should have:

    • Power State should be power off
    • Provision State should be available
    • Maintenance should be False
    • Instance UUID likely set to None.

    Use below command to make sure our nodes meet the above requirement
    [stack@undercloud-director ~]$ openstack baremetal node list
    +--------------------------------------+------------------------------+---------------+-------------+--------------------+-------------+
    | UUID                                 | Name                         | Instance UUID | Power State | Provisioning State | Maintenance |
    +--------------------------------------+------------------------------+---------------+-------------+--------------------+-------------+
    | ece1651a-6adc-4826-9f77-5d47891c6c9b | overcloud-controller.example | None          | power off   | available          | False       |
    | d19a1bce-3792-428e-b242-fab2bab6213d | overcloud-compute.example    | None          | power off   | available          | False       |
    | 74d4151f-03b5-4c6a-badc-d2cbf6bba7af | overcloud-ceph.example       | None          | power off   | available          | False       |
    +--------------------------------------+------------------------------+---------------+-------------+--------------------+-------------+

    The "openstack baremetal introspection" command is used to start the introspection and "bulk start" can be used to proceed with introspection of all nodes. The two nodes that will be checked are the controller and compute nodes

    [stack@undercloud-director ~]$ openstack baremetal configure boot

    [stack@undercloud-director ~]$ for i in overcloud-controller.example overcloud-compute.example  overcloud-ceph.example; do ironic node-show $i| grep -1 deploy; done
    | driver                 | pxe_ipmitool                                                          |
    | driver_info            | {u'ipmi_port': u'6321', u'ipmi_username': u'admin', u'deploy_kernel': |
    |                        | u'e16bd471-2bab-4bef-9e5f-c2b88def647f', u'ipmi_address':             |
    |                        | u'10.43.138.12', u'deploy_ramdisk': u'fdca9579-5a68-42a4-9ebf-        |
    |                        | fe6b605e6ae5', u'ipmi_password': u'******'}                           |
    | driver                 | pxe_ipmitool                                                          |
    | driver_info            | {u'ipmi_port': u'6322', u'ipmi_username': u'admin', u'deploy_kernel': |
    |                        | u'e16bd471-2bab-4bef-9e5f-c2b88def647f', u'ipmi_address':             |
    |                        | u'10.43.138.12', u'deploy_ramdisk': u'fdca9579-5a68-42a4-9ebf-        |
    |                        | fe6b605e6ae5', u'ipmi_password': u'******'}                           |
    | driver                 | pxe_ipmitool                                                          |
    | driver_info            | {u'ipmi_port': u'6320', u'ipmi_username': u'admin', u'deploy_kernel': |
    |                        | u'e16bd471-2bab-4bef-9e5f-c2b88def647f', u'ipmi_address':             |
    |                        | u'10.43.138.12', u'deploy_ramdisk': u'fdca9579-5a68-42a4-9ebf-        |
    |                        | fe6b605e6ae5', u'ipmi_password': u'******'}                           |

    Change the provision state of the nodes to "manageable"
    [stack@undercloud-director ~]$ for node in $(openstack baremetal node list -c UUID -f value) ; do openstack baremetal node manage $node ; done

    NOTE: After the introspection, the node's provisioning state will switch back to "available" but if it fails then you can also manually change the state of the node's from "manage" to "available" by using below the command
    $ openstack baremetal node provide <UUID>


    Check the provisioning state

    [stack@undercloud-director ~]$ openstack baremetal node list
    +--------------------------------------+------------------------------+---------------+-------------+--------------------+-------------+
    | UUID                                 | Name                         | Instance UUID | Power State | Provisioning State | Maintenance |
    +--------------------------------------+------------------------------+---------------+-------------+--------------------+-------------+
    | ece1651a-6adc-4826-9f77-5d47891c6c9b | overcloud-controller.example | None          | power off   | manageable         | False       |
    | d19a1bce-3792-428e-b242-fab2bab6213d | overcloud-compute.example    | None          | power off   | manageable         | False       |
    | 74d4151f-03b5-4c6a-badc-d2cbf6bba7af | overcloud-ceph.example       | None          | power off   | manageable         | False       |
    +--------------------------------------+------------------------------+---------------+-------------+--------------------+-------------+

    Run the following command to inspect the hardware attributes of individual nodes:
    NOTE: The --provide option resets all nodes to an active state after introspection.
    [stack@undercloud-director ~]$ for node in $(openstack baremetal node list -c UUID -f value) ; do openstack overcloud node introspect $node --provide; done
    Started Mistral Workflow. Execution ID: 1fc5064a-40c9-4471-9ca4-36577474f4ae
    Waiting for introspection to finish...
    Successfully introspected all nodes.
    Introspection completed.
    Started Mistral Workflow. Execution ID: 790bd02a-514f-4781-90e5-269b250876f8
    Successfully set all nodes to available.
    Started Mistral Workflow. Execution ID: d3c2e58a-54ab-48d8-87d8-58b84e5f6b7e
    Waiting for introspection to finish...
    Successfully introspected all nodes.
    Introspection completed.
    Started Mistral Workflow. Execution ID: 90ec9f5b-da54-4e32-a5fc-52c9e89b65e9
    Successfully set all nodes to available.
    Started Mistral Workflow. Execution ID: 04b8ec73-6440-432d-bcbf-72c6ee4c554a
    Waiting for introspection to finish...
    Successfully introspected all nodes.
    Introspection completed.
    Started Mistral Workflow. Execution ID: fa832937-b473-47ff-99ca-574ae2dd0bd1
    Successfully set all nodes to available.

    You can check the progress of the introspection using below comman from a different terminal
    [stack@undercloud-director ~]$ sudo journalctl -l -u openstack-ironic-inspector -u openstack-ironic-inspector-dnsmasq -u openstack-ironic-conductor -f
    OR
    You can monitor the console of the VM to check the progress of the introspection


    To check the exit status of introspection
    [stack@undercloud-director ~]$ for node in $(openstack baremetal node list -c UUID -f value) ; do echo -e "\n"$node;openstack baremetal introspection status $node; done

    ece1651a-6adc-4826-9f77-5d47891c6c9b
    +----------+-------+
    | Field    | Value |
    +----------+-------+
    | error    | None  |
    | finished | True  |
    +----------+-------+

    d19a1bce-3792-428e-b242-fab2bab6213d
    +----------+-------+
    | Field    | Value |
    +----------+-------+
    | error    | None  |
    | finished | True  |
    +----------+-------+

    74d4151f-03b5-4c6a-badc-d2cbf6bba7af
    +----------+-------+
    | Field    | Value |
    +----------+-------+
    | error    | None  |
    | finished | True  |
    +----------+-------+

    To get the disk related inventory from the introspection data
    [stack@undercloud-director ~]$ openstack baremetal introspection data save ece1651a-6adc-4826-9f77-5d47891c6c9b | jq ".inventory.disks"
    [
      {
        "size": 53687091200,
        "rotational": true,
        "vendor": "0x1af4",
        "name": "/dev/vda",
        "wwn_vendor_extension": null,
        "wwn_with_extension": null,
        "model": "",
        "wwn": null,
        "serial": null
      }
    ]

    HINT: This data can be used to assign root disks if required.



    Once the introspection is completed you can check the resources calculated by the nova at the introspection stage for all the 3 hypervisors
    [stack@undercloud-director ~]$ openstack  hypervisor stats show
    +----------------------+-------+
    | Field                | Value |
    +----------------------+-------+
    | count                | 3     |
    | current_workload     | 0     |
    | disk_available_least | 117   |
    | free_disk_gb         | 117   |
    | free_ram_mb          | 30720 |
    | local_gb             | 117   |
    | local_gb_used        | 0     |
    | memory_mb            | 30720 |
    | memory_mb_used       | 0     |
    | running_vms          | 0     |
    | vcpus                | 12    |
    | vcpus_used           | 0     |
    +----------------------+-------+


    Flavor Details

    The undercloud will have a number of default flavors created at install time. In most cases these flavors do not need to be modified, but they can be if desired. By default, all overcloud instances will be booted with the baremetal flavor, so all baremetal nodes must have at least as much memory, disk, and cpu as that flavor.

    In addition, there are profile-specific flavors created which can be used with the profile-matching feature
    [stack@undercloud-director ~]$ openstack flavor list
    +--------------------------------------+---------------+-------+------+-----------+-------+-----------+
    | ID                                   | Name          |   RAM | Disk | Ephemeral | VCPUs | Is Public |
    +--------------------------------------+---------------+-------+------+-----------+-------+-----------+
    | 30f44bc0-cad4-48a3-8b99-195ac0ccdb71 | ceph-storage  |  4096 |   40 |         0 |     2 | True      |
    | 33ca9562-a9d5-457e-b4af-98188c6eef5c | compute       |  4096 |   40 |         0 |     2 | True      |
    | 57cb3b2c-5d07-4aba-8606-fb9a5b033351 | baremetal     |  4096 |   40 |         0 |     2 | True      |
    | 5883f1c8-8d56-42b3-b1a1-45e3e2957314 | block-storage |  4096 |   40 |         0 |     2 | True      |
    | 9989fb29-5796-4597-98e1-efe981358659 | swift-storage |  4096 |   40 |         0 |     2 | True      |
    | ecf3a3ff-a885-4bad-a05d-cba8f0125498 | control       |  4096 |   40 |         0 |     2 | True      |
    +--------------------------------------+---------------+-------+------+-----------+-------+-----------+


    Check the property of the ironic nodes

    [stack@undercloud-director ~]$ for node in $(openstack baremetal node list -c UUID -f value) ; do  echo -e "\n"$node; openstack baremetal node show $node --fields properties ; done

    ece1651a-6adc-4826-9f77-5d47891c6c9b
    +------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
    | Field      | Value                                                                                                                                           |
    +------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
    | properties | {u'memory_mb': u'10240', u'cpu_arch': u'x86_64', u'local_gb': u'49', u'cpus': u'4', u'capabilities':                                            |
    |            | u'cpu_aes:true,cpu_hugepages:true,boot_option:local'}                                                                                           |
    +------------+-------------------------------------------------------------------------------------------------------------------------------------------------+

    d19a1bce-3792-428e-b242-fab2bab6213d
    +------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
    | Field      | Value                                                                                                                                           |
    +------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
    | properties | {u'memory_mb': u'10240', u'cpu_arch': u'x86_64', u'local_gb': u'49', u'cpus': u'4', u'capabilities':                                            |
    |            | u'cpu_aes:true,cpu_hugepages:true,boot_option:local'}                                                                                           |
    +------------+-------------------------------------------------------------------------------------------------------------------------------------------------+

    74d4151f-03b5-4c6a-badc-d2cbf6bba7af
    +------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
    | Field      | Value                                                                                                                                           |
    +------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
    | properties | {u'memory_mb': u'10240', u'cpu_arch': u'x86_64', u'local_gb': u'19', u'cpus': u'4', u'capabilities':                                            |
    |            | u'cpu_aes:true,cpu_hugepages:true,boot_option:local'}                                                                                           |
    +------------+-------------------------------------------------------------------------------------------------------------------------------------------------+



    IMPORTANT NOTE: Make sure that the extra specs in your flavor starting with capability: should match ones in node.properties['capabilities']. These fields will be checked by nova-scheduler and if they don't match you will get "NoValidHost: No valid host was found. There are not enough hosts available." error message during the overcloud deployment.

    Follow below article for more detail:
    How to fix "NoValidHost: No valid host was found. There are not enough hosts available" during overcloud deployment (openstack)

    Modify your flavors to add the capabilities similar to as the ironic nodes
    [stack@undercloud-director ~]$ openstack flavor set --property "capabilities:profile"="control"  --property "capabilities:cpu_aes"="true"  --property "capabilities:cpu_hugepages"="true" --property "capabilities:boot_option"="local" control

    [stack@undercloud-director ~]$ openstack flavor set --property "capabilities:profile"="compute"  --property "capabilities:cpu_aes"="true"  --property "capabilities:cpu_hugepages"="true" --property "capabilities:boot_option"="local" compute

    [stack@undercloud-director ~]$ openstack flavor set --property "capabilities:profile"="ceph-storage"  --property "capabilities:cpu_aes"="true"  --property "capabilities:cpu_hugepages"="true" --property "capabilities:boot_option"="local"  ceph-storage

    Check the properties filed of the flavors again which we plan to use with our ironic nodes
    [stack@undercloud-director ~]$ openstack flavor show compute -c properties
    +------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
    | Field      | Value                                                                                                                                           |
    +------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
    | properties | capabilities:boot_option='local', capabilities:cpu_aes='true', capabilities:cpu_hugepages='true', capabilities:profile='compute',               |
    |            | cpu_arch='x86_64'                                                                                                                               |
    +------------+-------------------------------------------------------------------------------------------------------------------------------------------------+

    [stack@undercloud-director ~]$ openstack flavor show control -c properties
    +------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
    | Field      | Value                                                                                                                                           |
    +------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
    | properties | capabilities:boot_option='local', capabilities:cpu_aes='true', capabilities:cpu_hugepages='true', capabilities:profile='control',               |
    |            | cpu_arch='x86_64'                                                                                                                               |
    +------------+-------------------------------------------------------------------------------------------------------------------------------------------------+

    [stack@undercloud-director ~]$ openstack flavor show ceph-storage -c properties
    +------------+---------------------------------------------------------------------------------------------------------------------------------------+
    | Field      | Value                                                                                                                                 |
    +------------+---------------------------------------------------------------------------------------------------------------------------------------+
    | properties | capabilities:boot_option='local', capabilities:cpu_aes='true', capabilities:cpu_hugepages='true', capabilities:profile='ceph-storage' |
    +------------+---------------------------------------------------------------------------------------------------------------------------------------+


    Tagging node into profiles

    After registering and inspecting the hardware of each node, you will tag them into specific profiles. These profile tags match your nodes to flavors, and in turn the flavors are assigned to a deployment role. Default profile flavors compute, control, swift-storage, ceph-storage, and block-storage are created during Undercloud installation and are usable without modification in most environments.

    Currently there is no profile assigned to any of the nodes
    [stack@undercloud-director ~]$ openstack overcloud profiles list
    +--------------------------------------+------------------------------+-----------------+-----------------+-------------------+
    | Node UUID                            | Node Name                    | Provision State | Current Profile | Possible Profiles |
    +--------------------------------------+------------------------------+-----------------+-----------------+-------------------+
    | ece1651a-6adc-4826-9f77-5d47891c6c9b | overcloud-controller.example | available       | None            |                   |
    | d19a1bce-3792-428e-b242-fab2bab6213d | overcloud-compute.example    | available       | None            |                   |
    | 74d4151f-03b5-4c6a-badc-d2cbf6bba7af | overcloud-ceph.example       | available       | None            |                   |
    +--------------------------------------+------------------------------+-----------------+-----------------+-------------------+

    Make sure you modify the "properties/capabilities" as per your node's properties and add the profile
    [stack@undercloud-director ~]$ ironic node-update overcloud-compute.example add properties/capabilities='profile:compute,cpu_aes:true,cpu_hugepages:true,boot_option:local'
    +------------------------+-----------------------------------------------------------------------+
    | Property               | Value                                                                 |
    +------------------------+-----------------------------------------------------------------------+
    | chassis_uuid           |                                                                       |
    | clean_step             | {}                                                                    |
    | console_enabled        | False                                                                 |
    | created_at             | 2018-08-14T05:45:36+00:00                                             |
    | driver                 | pxe_ipmitool                                                          |
    | driver_info            | {u'ipmi_port': u'6322', u'ipmi_username': u'admin', u'deploy_kernel': |
    |                        | u'e16bd471-2bab-4bef-9e5f-c2b88def647f', u'ipmi_address':             |
    |                        | u'10.43.138.12', u'deploy_ramdisk': u'fdca9579-5a68-42a4-9ebf-        |
    |                        | fe6b605e6ae5', u'ipmi_password': u'******'}                           |
    | driver_internal_info   | {}                                                                    |
    | extra                  | {u'hardware_swift_object': u'extra_hardware-                          |
    |                        | d19a1bce-3792-428e-b242-fab2bab6213d'}                                |
    | inspection_finished_at | None                                                                  |
    | inspection_started_at  | None                                                                  |
    | instance_info          | {}                                                                    |
    | instance_uuid          | None                                                                  |
    | last_error             | None                                                                  |
    | maintenance            | False                                                                 |
    | maintenance_reason     | None                                                                  |
    | name                   | overcloud-compute.example                                             |
    | network_interface      |                                                                       |
    | power_state            | power off                                                             |
    | properties             | {u'memory_mb': u'10240', u'cpu_arch': u'x86_64', u'local_gb': u'49',  |
    |                        | u'cpus': u'4', u'capabilities':                                       |
    |                        | u'profile:compute,cpu_aes:true,cpu_hugepages:true,boot_option:local'} |
    | provision_state        | available                                                             |
    | provision_updated_at   | 2018-08-14T05:55:10+00:00                                             |
    | raid_config            |                                                                       |
    | reservation            | None                                                                  |
    | resource_class         |                                                                       |
    | target_power_state     | None                                                                  |
    | target_provision_state | None                                                                  |
    | target_raid_config     |                                                                       |
    | updated_at             | 2018-08-14T05:55:16+00:00                                             |
    | uuid                   | d19a1bce-3792-428e-b242-fab2bab6213d                                  |
    +------------------------+-----------------------------------------------------------------------+

    [stack@undercloud-director ~]$ ironic node-update overcloud-controller.example add properties/capabilities='profile:control,cpu_aes:true,cpu_hugepages:true,boot_option:local'
    +------------------------+-----------------------------------------------------------------------+
    | Property               | Value                                                                 |
    +------------------------+-----------------------------------------------------------------------+
    | chassis_uuid           |                                                                       |
    | clean_step             | {}                                                                    |
    | console_enabled        | False                                                                 |
    | created_at             | 2018-08-14T05:45:36+00:00                                             |
    | driver                 | pxe_ipmitool                                                          |
    | driver_info            | {u'ipmi_port': u'6321', u'ipmi_username': u'admin', u'deploy_kernel': |
    |                        | u'e16bd471-2bab-4bef-9e5f-c2b88def647f', u'ipmi_address':             |
    |                        | u'10.43.138.12', u'deploy_ramdisk': u'fdca9579-5a68-42a4-9ebf-        |
    |                        | fe6b605e6ae5', u'ipmi_password': u'******'}                           |
    | driver_internal_info   | {}                                                                    |
    | extra                  | {u'hardware_swift_object': u'extra_hardware-ece1651a-                 |
    |                        | 6adc-4826-9f77-5d47891c6c9b'}                                         |
    | inspection_finished_at | None                                                                  |
    | inspection_started_at  | None                                                                  |
    | instance_info          | {}                                                                    |
    | instance_uuid          | None                                                                  |
    | last_error             | None                                                                  |
    | maintenance            | False                                                                 |
    | maintenance_reason     | None                                                                  |
    | name                   | overcloud-controller.example                                          |
    | network_interface      |                                                                       |
    | power_state            | power off                                                             |
    | properties             | {u'memory_mb': u'10240', u'cpu_arch': u'x86_64', u'local_gb': u'49',  |
    |                        | u'cpus': u'4', u'capabilities':                                       |
    |                        | u'profile:control,cpu_aes:true,cpu_hugepages:true,boot_option:local'} |
    | provision_state        | available                                                             |
    | provision_updated_at   | 2018-08-14T05:52:50+00:00                                             |
    | raid_config            |                                                                       |
    | reservation            | None                                                                  |
    | resource_class         |                                                                       |
    | target_power_state     | None                                                                  |
    | target_provision_state | None                                                                  |
    | target_raid_config     |                                                                       |
    | updated_at             | 2018-08-14T05:52:59+00:00                                             |
    | uuid                   | ece1651a-6adc-4826-9f77-5d47891c6c9b                                  |
    +------------------------+-----------------------------------------------------------------------+



    [stack@undercloud-director ~]$ ironic node-update overcloud-ceph.example add properties/capabilities='profile:ceph-storage,cpu_aes:true,cpu_hugepages:true,boot_option:local'
    +------------------------+-----------------------------------------------------------------------+
    | Property               | Value                                                                 |
    +------------------------+-----------------------------------------------------------------------+
    | chassis_uuid           |                                                                       |
    | clean_step             | {}                                                                    |
    | console_enabled        | False                                                                 |
    | created_at             | 2018-08-14T05:45:36+00:00                                             |
    | driver                 | pxe_ipmitool                                                          |
    | driver_info            | {u'ipmi_port': u'6320', u'ipmi_username': u'admin', u'deploy_kernel': |
    |                        | u'e16bd471-2bab-4bef-9e5f-c2b88def647f', u'ipmi_address':             |
    |                        | u'10.43.138.12', u'deploy_ramdisk': u'fdca9579-5a68-42a4-9ebf-        |
    |                        | fe6b605e6ae5', u'ipmi_password': u'******'}                           |
    | driver_internal_info   | {}                                                                    |
    | extra                  | {u'hardware_swift_object': u'extra_hardware-74d4151f-03b5-4c6a-badc-  |
    |                        | d2cbf6bba7af'}                                                        |
    | inspection_finished_at | None                                                                  |
    | inspection_started_at  | None                                                                  |
    | instance_info          | {}                                                                    |
    | instance_uuid          | None                                                                  |
    | last_error             | None                                                                  |
    | maintenance            | False                                                                 |
    | maintenance_reason     | None                                                                  |
    | name                   | overcloud-ceph.example                                                |
    | network_interface      |                                                                       |
    | power_state            | power off                                                             |
    | properties             | {u'memory_mb': u'10240', u'cpu_arch': u'x86_64', u'local_gb': u'49',  |
    |                        | u'cpus': u'4', u'capabilities': u'profile:ceph-                       |
    |                        | storage,cpu_aes:true,cpu_hugepages:true,boot_option:local'}           |
    | provision_state        | available                                                             |
    | provision_updated_at   | 2018-08-14T05:57:50+00:00                                             |
    | raid_config            |                                                                       |
    | reservation            | None                                                                  |
    | resource_class         |                                                                       |
    | target_power_state     | None                                                                  |
    | target_provision_state | None                                                                  |
    | target_raid_config     |                                                                       |
    | updated_at             | 2018-08-14T05:57:59+00:00                                             |
    | uuid                   | 74d4151f-03b5-4c6a-badc-d2cbf6bba7af                                  |
    +------------------------+-----------------------------------------------------------------------+

    Once all the above commands are executed sucessfully, re-check the profile assignment. Here we see respective profiles are assigned to the nodes
    [stack@undercloud-director ~]$ openstack overcloud profiles list
    +--------------------------------------+------------------------------+-----------------+-----------------+-------------------+
    | Node UUID                            | Node Name                    | Provision State | Current Profile | Possible Profiles |
    +--------------------------------------+------------------------------+-----------------+-----------------+-------------------+
    | ece1651a-6adc-4826-9f77-5d47891c6c9b | overcloud-controller.example | available       | control         |                   |
    | d19a1bce-3792-428e-b242-fab2bab6213d | overcloud-compute.example    | available       | compute         |                   |
    | 74d4151f-03b5-4c6a-badc-d2cbf6bba7af | overcloud-ceph.example       | available       | ceph-storage    |                   |
    +--------------------------------------+------------------------------+-----------------+-----------------+-------------------+


    Heat Templates

    The director uses Heat Orchestration Templates (HOT) as a template format for its Overcloud deployment plan. Templates in HOT format are mostly expressed in YAML format. The purpose of a template is to define and create a stack, which is a collection of resources that Heat creates and the configuration per resources. Resources are objects in OpenStack and can include compute resources, network configuration, security groups, scaling rules, and custom resources.

    The structure of a Heat template has three main sections:
    Parameters - These are settings passed to Heat, which provides a way to customize a stack, and any default values for parameters without passed values. These are defined in the parameters section of a template.

    Resources - These are the specific objects to create and configure as part of a stack. OpenStack contains a set of core resources that span across all components. These are defined in the resources section of a template.

    Output - These are values passed from Heat after the stack's creation. You can access these values either through the Heat API or client tools. These are defined in the output section of a template.

    Copy the default templates
    [stack@undercloud-director ~]$ cp -rf /usr/share/openstack-tripleo-heat-templates/* ~/templates/

    [stack@undercloud-director ~]$ cd templates/

    [stack@undercloud-director templates]$ ls -l
    total 160
    -rw-r--r--. 1 stack stack  1039 Aug 13 11:15 all-nodes-validation.yaml
    -rw-r--r--. 1 stack stack   583 Aug 13 11:15 bootstrap-config.yaml
    -rw-r--r--. 1 stack stack 21256 Aug 13 11:15 capabilities-map.yaml
    drwxr-xr-x. 5 stack stack  4096 Aug 13 11:15 ci
    -rw-r--r--. 1 stack stack   681 Aug 13 11:15 default_passwords.yaml
    drwxr-xr-x. 3 stack stack  4096 Aug 13 11:15 deployed-server
    drwxr-xr-x. 4 stack stack  4096 Aug 13 11:15 docker
    drwxr-xr-x. 4 stack stack  4096 Aug 13 11:15 environments
    drwxr-xr-x. 6 stack stack  4096 Aug 13 11:15 extraconfig
    drwxr-xr-x. 2 stack stack  4096 Aug 13 11:15 firstboot
    -rw-r--r--. 1 stack stack   735 Aug 13 11:15 hosts-config.yaml
    -rw-r--r--. 1 stack stack   325 Aug 13 11:15 j2_excludes.yaml
    -rw-r--r--. 1 stack stack  2594 Aug 13 11:15 net-config-bond.yaml
    -rw-r--r--. 1 stack stack  1895 Aug 13 11:15 net-config-bridge.yaml
    -rw-r--r--. 1 stack stack  2298 Aug 13 11:15 net-config-linux-bridge.yaml
    -rw-r--r--. 1 stack stack  1244 Aug 13 11:15 net-config-noop.yaml
    -rw-r--r--. 1 stack stack  3246 Aug 13 11:15 net-config-static-bridge-with-external-dhcp.yaml
    -rw-r--r--. 1 stack stack  2838 Aug 13 11:15 net-config-static-bridge.yaml
    -rw-r--r--. 1 stack stack  2545 Aug 13 11:15 net-config-static.yaml
    drwxr-xr-x. 5 stack stack  4096 Aug 13 11:15 network
    -rw-r--r--. 1 stack stack 26967 Aug 13 11:15 overcloud.j2.yaml
    -rw-r--r--. 1 stack stack 14608 Aug 13 11:15 overcloud-resource-registry-puppet.j2.yaml
    drwxr-xr-x. 5 stack stack  4096 Aug 13 11:15 puppet
    -rw-r--r--. 1 stack stack  6832 Aug 13 11:15 roles_data.yaml
    drwxr-xr-x. 2 stack stack  4096 Aug 13 11:15 validation-scripts

    The default mapping uses the root disk for Ceph Storage. However, most production environments use multiple separate disks for storage and partitions for journaling. In this situation, you define a storage map as part of the storage-environment.yaml file copied previously.



    Edit the storage-environment.yaml file and add an additional section that contains the following:
    [stack@undercloud-director ~]$ vim templates/environments/storage-environment.yaml
    resource_registry:
      OS::TripleO::Services::CephMon: ../puppet/services/ceph-mon.yaml
      OS::TripleO::Services::CephOSD: ../puppet/services/ceph-osd.yaml
      OS::TripleO::Services::CephClient: ../puppet/services/ceph-client.yaml
    parameter_defaults:
      CinderEnableIscsiBackend: false
      CinderEnableRbdBackend: true
      CinderBackupBackend: ceph
      NovaEnableRbdBackend: true
      GlanceBackend: rbd
      GnocchiBackend: rbd
      ExtraConfig:
        ceph::profile::params::osds:
          '/dev/vdc':
            journal: '/dev/vdb'

    This adds extra Hiera data to the Overcloud, which Puppet uses as custom parameters during configuration. Use the ceph::profile::params::osds parameter to map the relevant disks and journal partitions. For example, a Ceph node with four disks might have the following assignments:

    /dev/vda - The root disk containing the Overcloud image
    /dev/vdb - The disk containing the journal partitions. This is usually a solid state disk (SSD) to aid with system performance.
    /dev/vdc - The OSD disks


    Overcloud Deployment

    After introspection the undercloud knows which nodes are used for the deployment of the overcloud, but it may not know what overcloud node types are to be deployed.
    [stack@undercloud-director ~]$ openstack overcloud  deploy --templates  ~/templates/ --control-scale 1 --compute-scale 1 --ceph-storage-scale 1 --control-flavor control --compute-flavor compute --ceph-storage-flavor ceph-storage --neutron-tunnel-types vxlan --neutron-network-type vxlan -e ~/templates/environments/storage-environment.yaml
    ********output trimmed*********

    2018-08-13 14:00:31Z [overcloud]: CREATE_COMPLETE  Stack CREATE completed successfully

     Stack overcloud CREATE_COMPLETE

    Host 192.168.122.151 not found in /home/stack/.ssh/known_hosts
    Overcloud Endpoint: http://192.168.122.151:5000/v2.0
    Overcloud Deployed

    HINT: The entire deployment may take ~45-60 minutes

    Below list explains the terminology used in the above command.
    --control-scale : The number of Controller nodes to scale out

    --compute-scale: The number of Compute nodes to scale out

    --ceph-storage-scale: The number of Ceph Storage nodes to scale out

    --templates [TEMPLATES]: The directory containing the Heat templates to deploy. If blank, the command uses the default template location at /usr/share/openstack-tripleo-heat-templates/

    -e [EXTRA HEAT TEMPLATE], --extra-template [EXTRA HEAT TEMPLATE]: Extra environment files to pass to the overcloud deployment. Can be specified more than once. Note that the order of environment files passed to the openstack overcloud deploy command is important.

    --neutron-tunnel-types: The tunnel types for the Neutron tenant network. To specify multiple values, use a comma separated string

    --compute-flavor: The flavor to use for Compute nodes

    --ceph-storage-flavor: The flavor to use for Ceph Storage nodes

    --control-flavor: The flavor to use for Controller nodes

    To check the status of overcloud deployment
    $ heat resource-list overcloud

    To check the status of individual resource
    $ heat resource-show overcloud <Resource>

    To delete the existing overcloud stack and re-deploy
    $ openstack stack-delete overcloud

    [stack@undercloud-director ~]$ openstack server list
    +--------------------------------------+-------------------------+--------+------------+-------------+--------------------------+
    | ID                                   | Name                    | Status | Task State | Power State | Networks                 |
    +--------------------------------------+-------------------------+--------+------------+-------------+--------------------------+
    | 9651eac2-f5de-410f-a5bf-a98772cd6790 | overcloud-cephstorage-0 | ACTIVE | -          | Running     | ctlplane=192.168.122.153 |
    | 7ba43618-c52a-4dbd-b82d-726207011e0e | overcloud-compute-0     | ACTIVE | -          | Running     | ctlplane=192.168.122.157 |
    | 609cb38a-6f14-42f3-b392-cfa1bcee3769 | overcloud-controller-0  | ACTIVE | -          | Running     | ctlplane=192.168.122.152 |
    +--------------------------------------+-------------------------+--------+------------+-------------+--------------------------+

    Below file will be created once the overcloud deployment is complete
    [stack@undercloud-director ~]$ cat overcloudrc
    # Clear any old environment that may conflict.
    for key in $( set | awk '{FS="="}  /^OS_/ {print $1}' ); do unset $key ; done
    export OS_USERNAME=admin
    export OS_TENANT_NAME=admin
    export NOVA_VERSION=1.1
    export OS_PROJECT_NAME=admin
    export OS_PASSWORD=fdk8YmUHE9ujb7GXpqDAxsT7g
    export OS_NO_CACHE=True
    export COMPUTE_API_VERSION=1.1
    export no_proxy=,192.168.122.151,192.168.122.151
    export OS_CLOUDNAME=overcloud
    export OS_AUTH_URL=http://192.168.122.151:5000/v2.0
    export PYTHONWARNINGS="ignore:Certificate has no, ignore:A true SSLContext object is not available"



    Now you can login to your nodes from the undercloud
    [stack@undercloud-director ~]$ ssh heat-admin@192.168.122.157
    The authenticity of host '192.168.122.157 (192.168.122.157)' can't be established.
    ECDSA key fingerprint is SHA256:yaMnJm7HTfOrMeOiYT6PR6nm7DO7SQQgX1Bh4bNwaSU.
    ECDSA key fingerprint is MD5:b6:f1:9b:f5:ac:93:b7:9d:2e:a9:9c:cb:0c:6c:a7:b7.
    Are you sure you want to continue connecting (yes/no)? yes
    Warning: Permanently added '192.168.122.157' (ECDSA) to the list of known hosts.
    [heat-admin@overcloud-compute-0 ~]$ logout
    Connection to 192.168.122.157 closed.

    [stack@undercloud-director ~]$ ssh heat-admin@192.168.122.153
    The authenticity of host '192.168.122.153 (192.168.122.153)' can't be established.
    ECDSA key fingerprint is SHA256:goMTxDPCmNlYfmdQdZ44nw5iXWyQNtB5dbAnfVwGDqA.
    ECDSA key fingerprint is MD5:e4:1d:32:b3:d0:e2:1f:44:ed:6d:0b:80:61:09:8f:4d.
    Are you sure you want to continue connecting (yes/no)? yes
    Warning: Permanently added '192.168.122.153' (ECDSA) to the list of known hosts.
    [heat-admin@overcloud-controller-0 ~]$ logout
    Connection to 192.168.122.153 closed.

    [stack@undercloud-director ~]$ ssh heat-admin@192.168.122.152
    The authenticity of host '192.168.122.152 (192.168.122.152)' can't be established.
    ECDSA key fingerprint is SHA256:EajVihYirMBljOnA7eNPY0lVd1TGpMQRgvmqNUqrrQk.
    ECDSA key fingerprint is MD5:d0:4b:40:ab:56:6f:f8:c2:fe:ee:61:0b:10:84:2b:c8.
    Are you sure you want to continue connecting (yes/no)? yes
    Warning: Permanently added '192.168.122.152' (ECDSA) to the list of known hosts.
    [heat-admin@overcloud-cephstorage-0 ~]$

    I hope the article was useful.

    Please let me know your views and feedback in the comment section below.

    Deepak Prasad

    is a techie and an author who is still trying to survive in this IT generation with very little knowledge he has on Linux/Unix, VMware, SAN Storage, Automation, networking etc

    You can follow him on Facebook or Google+


    Do you also have something to share here?
    Join GoLinuxHub Team as an Author, Click here for more information
    Final Part 3: Openstack TripleO Architecture and Step By Step Guide for installation of undercloud and overcloud nodes (compute, controller, ceph-storage) Final Part 3: Openstack TripleO Architecture and Step By Step Guide for installation of undercloud and overcloud nodes (compute, controller, ceph-storage) Reviewed by Deepak Prasad on Wednesday, August 15, 2018 Rating: 5

    No comments:

    Powered by Blogger.