Skip to main content

Fixing Openstack VM spawning issue: No suitable host found/vif_type=binding_failed error

Once in a while, on a new setup, when you try to spawn a VM on Openstack and it may fail with the error :  No Suitable Host Found, it points to the fact the the Nova Scheduler failed to filter out a host to spawn the VM on.

[The following trouble shooting is related to my setup. There are a lot of other factors that may lead to this error. So google a bit and check them in addition. This article can help you in figuring out the way to do troubleshooting.]

A quick look at /var/log/nova/scheduler.log and compute.log shows an error:
  • NovaException: Unexpected vif_type=binding_failed
  • Filter RetryFilter returned 0 hosts
This points to the fact that something is wrong with the Neutron setup. So a quick look into the neutron logs show the following:


ERROR neutron.openstack.common.rpc.common [-] AMQP server on ***** is unreachable: [Errno 111] ECONNREFUSED. Trying again in 5 seconds.
ERROR neutron.agent.linux.ovsdb_monitor [-] Error received from ovsdb monitor: ovsdb-client: unix:/var/run/openvswitch/db.sock: receive failed (End of file)
ERROR neutron.agent.linux.ovs_lib [-] Unable to execute ['ovs-ofctl', 'dump-flows', 'br-int', 'table=22']. Exception:
Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-ofctl', 'dump-flows', 'br-int', 'table=22']
Exit code: 1
Stdout: ''
Stderr: 'ovs-ofctl: br-int is not a bridge or a socket\n'


INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-d0057d1b-79fd-41ac-8616-7b86b5c900a6 None] Mapping physical network physnet1 to bridge br-eth1
ERROR neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-d0057d1b-79fd-41ac-8616-7b86b5c900a6 None] Bridge br-eth1 for physical network physnet1 does not exist. Agent terminated!

 Whoa!! My agent looks dead. A quick status check [ /etc/init.d/neutron-openvswitch-agent status] confirms this.

The problem now seems to be that my OVS agent is trying to bind a port on physnet1 to br-eth1 bridge and is failing to find it. So, to fix the problem, create the bridge [which got missed out in the initial setup]:

ovs-vsctl add-br br-eth1
ovs-vsctl add-port br-eth1 eth1

Restarting the ovs-agent fixes the issue.

Hope this helps someone.
Labels:

Comments

Post a Comment

Popular posts from this blog

Openstack : Fixing Failed to create network. No tenant network is available for allocation issue.

Assumptions : You are using ML2 plugin configured to use Vlans If you try to create a network for a tenant and it fails with the following error: Error: Failed to create network "Test": 503-{u'NeutronError': {u'message': u'Unable to create the network. No tenant network is available for allocation.', u'type': u'NoNetworkAvailable', u'detail': u''}} The problem can be due to missing configuration in the below files: In /etc/neutron/plugins/ml2/ml2_conf.ini network_vlan_ranges =physnet1:1000:2999 (1000:2999 is the Vlan range allocation) In /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini bridge_mappings = physnet1:br-eth1 (in OVS we map the physical network to the OVS bridge) Note You should have created a bridge br-eth1 manually and mapped it to a port ovs-vsctl add-br br-eth1 ovs-vsctl add-port br-eth1 eth1 Once configuration is done, restart the neutron ovs agent on the compute node(s):
1 comment
Read more

Solved: Fix for Git clone failure due to GnuTLS recv error (-9)

My devstack installation was failing with an error reported by the GnuTLS module as shown below: $ git clone https://github.com/openstack/horizon.git /opt/stack/horizon --branch master Cloning into '/opt/stack/horizon'... remote: Counting objects: 154213, done. remote: Compressing objects: 100% (11/11), done. error: RPC failed; curl 56 GnuTLS recv error (-9): A TLS packet with unexpected length was received. fatal: The remote end hung up unexpectedly fatal: early EOF fatal: index-pack failed The following Git config changes fixed the issue for me. Am hoping it will be useful for someone out there: $ git config http.sslVerify false $ git config --global http.postBuffer 1048576000
14 comments
Read more

QuickBite: Tap Vs Veth

Linux supports virtual networking via various artifacts such as: Soft Switches (Linux Bridge, OpenVSwitch) Virtual Network Adapters (tun, tap, veth and a few more) In this blog, we will look at the virtual network adapters tap and veth. From a practical view point, both seem to be having the same functionality and its a bit confusing as to where to use what. A quick definition of tap/veth is as follows: TAP A TAP is a simulated interface which exists only in the kernel and has no physical component associated with it. It can be viewed as a simple Point-to-Point or Ethernet device, which instead of receiving packets from a physical media, receives them from user space program and instead of sending packets via physical media writes them to the user space program. When a user space program (in our case the VM) gets attached to the tap interface it gets hold of a file descriptor, reading from which gives it the data being sent on the tap interface. Writing to the file descri
5 comments
Read more