• VMware

    Learn about VMware virtualization for its products like vsphere ESX and ESXi, vCenter Server, VMware View, VMware P2V and many more

  • Linux

    Step by step configuration tutorials for many of the Linux services like DNS, DHCP, FTP, Samba4 etc including many tips and tricks in Red Hat Linux.

  • Database

    Learn installation and configuration of databases like Oracle, My SQL, Postgresql, etc including many other related tutorials in Linux.

  • How to fix "NoValidHost: No valid host was found. There are not enough hosts available" during overcloud deployment (openstack)


    ERROR:

    You get below error message on the console while performing an overcloud deployment from the undercloud director
    2018-08-14 06:42:59Z [overcloud.Controller.0.Controller]: CREATE_FAILED  ResourceInError: resources.Controller: Went to status ERROR due to "Message: No valid host was found. There are not enough hosts available., Code: 500"
    2018-08-14 06:42:59Z [overcloud.Controller.0]: CREATE_FAILED  Resource CREATE failed: ResourceInError: resources.Controller: Went to status ERROR due to "Message: No valid host was found. There are not enough hosts available., Code: 500"
    2018-08-14 06:42:59Z [overcloud.Controller.0]: CREATE_FAILED  ResourceInError: resources[0].resources.Controller: Went to status ERROR due to "Message: No valid host was found. There are not enough hosts available., Code: 500"
    2018-08-14 06:42:59Z [overcloud.Controller]: CREATE_FAILED  Resource CREATE failed: ResourceInError: resources[0].resources.Controller: Went to status ERROR due to "Message: No valid host was found. There are not enough hosts available., Code: 500"
    2018-08-14 06:42:59Z [overcloud.Controller]: CREATE_FAILED  ResourceInError: resources.Controller.resources[0].resources.Controller: Went to status ERROR due to "Message: No valid host was found. There are not enough hosts available., Code: 500"
    2018-08-14 06:43:00Z [overcloud]: CREATE_FAILED  Resource CREATE failed: ResourceInError: resources.Controller.resources[0].resources.Controller: Went to status ERROR due to "Message: No valid host was found. There are not enough hosts available., Code: 500"

    In the nova-conductor.log, you may find below messages
    nova-conductor.log:2018-08-12 22:50:24.855 11372 WARNING nova.scheduler.utils [req-1bcec69e-4ad1-4460-bdd0-506e701dd81b a557b3e88e20446d921adbea6667e9ec d7f4df43c5db41f08572f582344b4e6f - - -] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available.
    nova-conductor.log:2018-08-12 22:50:26.995 11373 WARNING nova.scheduler.utils [req-0070ba9f-5eed-441d-a762-16bb9eaf9e8a a557b3e88e20446d921adbea6667e9ec d7f4df43c5db41f08572f582344b4e6f - - -] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available.

    In your nova-scheduler.log, below message may appear
    nova-scheduler.log:2018-08-13 16:22:48.035 1475 INFO nova.filters [req-291df51f-aac0-42dc-bc72-84dfdb3e4a0c a557b3e88e20446d921adbea6667e9ec d7f4df43c5db41f085744b4e6f - - -] Filtering removed all hosts for the request with instance ID 'f6ec9c8e-4dd9-4995-92b7-b570bb06790f'. Filter results: ['RetryFilter: (start: 3, en, 'TripleOCapabilitiesFilter: (start: 3, end: 3)', 'ComputeCapabilitiesFilter: (start: 3, end: 1)', 'AvailabilityZoneFilter: (start: 1, end: 1)', 'RamFilter: (s1, end: 1)', 'DiskFilter: (start: 1, end: 0)']


    ANALYSIS AND SOLUTION:



    Scenario 1:
    Make sure that enough nodes are in available state, not in maintenance mode and not already used by an existing instance.
    [stack@undercloud-director ~]$ openstack baremetal node list
    +--------------------------------------+---------------------------+---------------+-------------+--------------------+-------------+
    | UUID                                 | Name                      | Instance UUID | Power State | Provisioning State | Maintenance |
    +--------------------------------------+---------------------------+---------------+-------------+--------------------+-------------+
    | 8ef9b862-9a4f-4961-813c-4b81be7c8e48 | overcloud-controller      | None          | power off   | available          | False       |
    | 7995f1f2-4af7-4c5d-9099-fc928c4c73b3 | overcloud-compute.example | None          | power off   | available          | False       |
    | 7c84cdf2-c5b2-47fb-a741-30c025b54183 | overcloud-ceph.example    | None          | power off   | available          | False       |
    +--------------------------------------+---------------------------+---------------+-------------+--------------------+-------------+

    Scenario 2:
    Make sure the undercloud is able to connect to the overcloud hypervisors using the power credentials (e.g. ipmi_address, ipmi_username and ipmi_password)
    [stack@undercloud-director ~]$ ipmitool -I lanplus -H 10.43.138.12 -L ADMINISTRATOR -p 6320 -U admin -R 3 -N 5 -P redhat power status
    Chassis Power is off

    [stack@undercloud-director ~]$ ipmitool -I lanplus -H 10.43.138.12 -L ADMINISTRATOR -p 6321 -U admin -R 3 -N 5 -P redhat power status
    Chassis Power is off

    [stack@undercloud-director ~]$ ipmitool -I lanplus -H 10.43.138.12 -L ADMINISTRATOR -p 6320 -U admin -R 3 -N 5 -P redhat power status
    Chassis Power is off

    Many times it is observed that ipmitool fails to connect to the hypervisor hence the add node functionality breaks during overcloud deployment

    Scenario 3:
    Make sure that you have Compute services running and enabled:
    [stack@undercloud-director ~]$ openstack compute service list --service nova-compute
    +----+--------------+-----------------------------+------+---------+-------+----------------------------+
    | ID | Binary       | Host                        | Zone | Status  | State | Updated At                 |
    +----+--------------+-----------------------------+------+---------+-------+----------------------------+
    |  4 | nova-compute | undercloud-director.example | nova | enabled | up    | 2018-08-14T12:51:23.000000 |
    +----+--------------+-----------------------------+------+---------+-------+----------------------------+

    By default, a Compute service is disabled after 10 consecutive build failures on it. This is to ensure that new build requests are not routed to a broken Compute service. If it is the case, make sure to fix the source of the failures, then re-enable it:
    $ openstack compute service set --enable <COMPUTE HOST> nova-compute





    Scenario 4: (Most possible)
    Make sure the node's properties matches completely with the assigned flavor's properties field. The node's property is populated at the introspection stage

    For exacmple my node's property
    [stack@undercloud-director ~]$ openstack baremetal node show ece1651a-6adc-4826-9f77-5d47891c6c9b -c properties
    +------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
    | Field      | Value                                                                                                                                           |
    +------------+-------------------------------------------------------------------------------------------------------------------------------------------------+
    | properties | {u'memory_mb': u'10240', u'cpu_arch': u'x86_64', u'local_gb': u'49', u'cpus': u'4', u'capabilities':                                            |
    |            | u'profile:control,cpu_aes:true,cpu_hugepages:true,boot_option:local'}                                                                           |
    +------------+-------------------------------------------------------------------------------------------------------------------------------------------------+

    This node is assigned to "control" profile as highlighted, so check the properties of "control" flavor
    [stack@undercloud-director ~]$ openstack flavor show control -c properties -f value
    capabilities:boot_option='local', capabilities:cpu_aes='true', capabilities:cpu_hugepages='true', capabilities:profile='control', cpu_arch='x86_64'

    Here as you see both the property field match exactly.

    How to update "node's" property content?

    You can manually update the node's property content although I would not recommend that because these are automatically populated by the introspection. You should modify your flavor's property to match the node's content. But still we can update the node's property using the below command
    $ ironic node-update overcloud-controller.example add properties/capabilities='profile:control,cpu_aes:true,cpu_hugepages:true,boot_option:local'



    How to update flavor's property field?

    To update the property section of a flavor use the below command
    $ openstack flavor set --property "capabilities:profile"="control"  --property "capabilities:cpu_aes"="true"  --property "capabilities:cpu_hugepages"="true" --property "capabilities:boot_option"="local" control


    There can be many more possibilities, these were few which I faced during my testing stage. If the above didn't helped you then for more information on the you can check below log files

    /var/log/ironic/*
    /var/log/nova/*

    I hope the article was useful.

    Deepak Prasad

    is a techie and an author who is still trying to survive in this IT generation with very little knowledge he has on Linux/Unix, VMware, SAN Storage, Automation, networking etc

    You can follow him on Facebook or Google+


    Do you also have something to share here?
    Join GoLinuxHub Team as an Author, Click here for more information
    How to fix "NoValidHost: No valid host was found. There are not enough hosts available" during overcloud deployment (openstack) How to fix "NoValidHost: No valid host was found. There are not enough hosts available" during overcloud deployment (openstack) Reviewed by Deepak Prasad on Thursday, August 16, 2018 Rating: 5

    No comments:

    Powered by Blogger.