Cinder fails to create volume with eseries OpenStack icehouse

[JULIANDIAZ]

Hello!

I'm trying to integrate our new E5524 in one of our test clouds and I keep running against an error when I try to create a volume using cinder (ver: 1.0.8). I have already installed the Proxy as stated in the document: https://library.netapp.com/ecm/ecm_get_file/ECMP1506075

This is my cinder.conf file:

[DEFAULT]

enabled_backends=iscsi_emc_vmax_1,NetApp_ESeries

rootwrap_config = /etc/cinder/rootwrap.conf

api_paste_config = /etc/cinder/api-paste.ini

iscsi_helper = tgtadm

volume_name_template = volume-%s

volume_group = cinder-volumes

verbose = True

debug = True

auth_strategy = keystone

state_path = /var/lib/cinder

lock_path = /var/lock/cinder

volumes_dir = /var/lib/cinder/volumes

osapi_volume_workers = 6

rpc_backend = cinder.openstack.common.rpc.impl_kombu

rabbit_host = xxx.xxx.xxx.xxx

rabbit_port = 5672

rabbit_userid = guest

rabbit_password = guest

notification_driver=cinder.openstack.common.notifier.rpc_notifier

periodic_interval=60

report_interval=60

osapi_volume_extension=cinder.api.contrib.standard_extensions

my_ip=xxx.xxx.xxx.xxx

[database]

connection = mysql://cinder:cinder@localhost/cinder

[keystone_authtoken]

auth_uri = http://xxx.xxx.xxx.xxx:5000

auth_host = xxx.xxx.xxx.xxx

auth_port = 35357

auth_protocol = http

admin_tenant_name = service

admin_user = cinder

admin_password = xxxxxxxx

[iscsi_emc_vmax_1]

volume_backend_name=emc_vmax_1

iscsi_target_prefix=iqn.1992-04.com.emc

# ALTERNATIVE PORT --> iscsi_ip_address=xxx.xxx.xxx.xxx

iscsi_ip_address=xxx.xxx.xxx.xxx

volume_driver=cinder.volume.drivers.emc.emc_smis_iscsi.EMCSMISISCSIDriver

cinder_emc_config_file=/etc/cinder/cinder_emc_vmax_1.xml

[NetApp_ESeries]

volume_driver=cinder.volume.drivers.netapp.common.NetAppDriver

netapp_storage_family=eseries

netapp_storage_protocol=iscsi

netapp_server_hostname=netapp-proxy.despexds.net

netapp_transport_type=http

netapp_webservice_path=/devmgr/v2

netapp_server_port=8080

netapp_login=rw

netapp_password=rw

netapp_controller_ips=10.1.5.22,10.1.5.23

netapp_sa_password=*********

netapp_storage_pools=Disk_Pool_1,Disk_Pool_2

These are the commands I issued to create the backend type:

# cinder type-create NetApp_ESeries

# cinder type-key NetApp_ESeries set volume_backend_name=NetApp_ESeries

# ./restart_all.sh

And to create the test volume:

# cinder create --volume-type NetApp_ESeries --display_name nettest 5

The cinder list shows the error codition associated to the newly created volume:

+------------------------------------------------------------+-------------+----------------------+------+--------------------------+--------------+--------------------------------------+

|                  ID                                        |   Status  | Display Name | Size |  Volume Type      | Bootable |         Attached to             |

+------------------------------------------------------------+--------------+---------------------+-------+--------------------------+-------------+-------------------------------------+

| 5d7e7ea6-27c9-412e-beb5-b59b5c105da2 |     error   |      nettest       |  5    | NetApp_ESeries  |  false      |                                      |

+------------------------------------------------------------+-------------+----------------------+------+--------------------------+--------------+--------------------------------------+

This is the output of some information commands:

# cinder type-list

+-------------------------------------------------------------------+-----------------------------------------------+

|                              ID                                   |                      Name                 |

+-------------------------------------------------------------------+-----------------------------------------------+

| 9b78963c-cfa6-4dc3-8d85-d1bd2f6b5446       |              iscsiEmcVmax1          |

| dad8851f-c8bf-4c0c-88a1-d337a29e8508        |              NetApp_ESeries         |

+-------------------------------------------------------------------+------------------------------------------------+

# cinder extra-specs-list

+-------------------------------------------------------------------------+------------------------------+--------------------------------------------------------------------------+

|                                   ID                                    |             Name         |                            extra_specs                            |

+-------------------------------------------------------------------------+------------------------------+--------------------------------------------------------------------------+

| 9b78963c-cfa6-4dc3-8d85-d1bd2f6b5446             | iscsiEmcVmax1      |   {u'volume_backend_name': u'emc_vmax_1'}   |

| dad8851f-c8bf-4c0c-88a1-d337a29e8508              | NetApp_ESeries     | {u'volume_backend_name': u'NetApp_ESeries'}|

+-------------------------------------------------------------------------+------------------------------+--------------------------------------------------------------------------+

The scheduler log shows the following, I highlighted what caught my attention):

2014-08-08 11:08:51.827 12632 DEBUG oslo.messaging._drivers.amqpdriver [-] received {u'_context_domain': None, u'_context_request_id': u'req-28594cd1-7285-4049-baa6-faaa81653409', u'_context_quota_class': None, u'_context_service_catalog': [{u'endpoints': [{u'adminURL': u'http://10.111.81.1:8774/v2/3ab47e19d21c4672973e57e7a8c85c83', u'region': u'RegionOne', u'internalURL': u'http://10.111.81.1:8774/v2/3ab47e19d21c4672973e57e7a8c85c83', u'id': u'302b287bedf54a6fa1dd51a73793ee57', u'publicURL': u'http://10.111.81.1:8774/v2/3ab47e19d21c4672973e57e7a8c85c83'}], u'endpoints_links': [], u'type': u'compute', u'name': u'nova'}], u'_context_auth_token': '<SANITIZED>', u'_context_user': u'790e3caa38984fccb158b05571e1bc6c', u'_context_user_id': u'790e3caa38984fccb158b05571e1bc6c', u'_context_is_admin': True, u'version': u'1.2', u'_context_project_domain': None, u'_context_timestamp': u'2014-08-08T15:08:51.274902', u'method': u'create_volume', u'_context_remote_address': u'10.111.81.20', u'_context_roles': [u'KeystoneAdmin', u'KeystoneServiceAdmin', u'admin'], u'args': {u'request_spec': {u'volume_properties': {u'status': u'creating', u'volume_type_id': u'dad8851f-c8bf-4c0c-88a1-d337a29e8508', u'user_id': u'790e3caa38984fccb158b05571e1bc6c', u'availability_zone': u'nova', u'reservations': [u'1236dfef-ea96-4dd2-b196-6a397e22d974', u'8c0cfc24-9188-4592-8dfa-9760fd5171e1', u'65d8f43d-a17b-4088-bd12-3ab9887cce92', u'8f6cda2f-300f-4e94-88c9-fde4ba53679f'], u'snapshot_id': None, u'attach_status': u'detached', u'display_description': None, u'volume_metadata': [], u'encryption_key_id': None, u'source_volid': None, u'volume_admin_metadata': [], u'display_name': u'nettest', u'project_id': u'3ab47e19d21c4672973e57e7a8c85c83', u'id': u'5d7e7ea6-27c9-412e-beb5-b59b5c105da2', u'size': 5, u'metadata': {}}, u'volume_type': {u'name': u'NetApp_ESeries', u'qos_specs_id': None, u'deleted': False, u'created_at': u'2014-08-08T14:17:56.000000', u'updated_at': None, u'extra_specs': {u'volume_backend_name': u'NetApp_ESeries'}, u'deleted_at': None, u'id': u'dad8851f-c8bf-4c0c-88a1-d337a29e8508'}, u'image_id': None, u'snapshot_id': None, u'source_volid': None, u'volume_id': u'5d7e7ea6-27c9-412e-beb5-b59b5c105da2'}, u'volume_id': u'5d7e7ea6-27c9-412e-beb5-b59b5c105da2', u'filter_properties': {}, u'topic': u'cinder-volume', u'image_id': None, u'snapshot_id': None}, u'_unique_id': u'cd189dbaada94d98ab458439cb818b62', u'_context_project_name': u'admin', u'_context_read_deleted': u'no', u'_context_user_identity': u'790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -', u'_context_tenant': u'3ab47e19d21c4672973e57e7a8c85c83', u'_context_project_id': u'3ab47e19d21c4672973e57e7a8c85c83', u'_context_user_domain': None} _safe_log /usr/lib/python2.7/dist-packages/oslo/messaging/_drivers/common.py:290

2014-08-08 11:08:51.829 12632 DEBUG oslo.messaging._drivers.amqp [-] unpacked context: {'tenant': u'3ab47e19d21c4672973e57e7a8c85c83', 'domain': None, 'project_name': u'admin', 'user_id': u'790e3caa38984fccb158b05571e1bc6c', 'roles': [u'KeystoneAdmin', u'KeystoneServiceAdmin', u'admin'], 'user_identity': u'790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -', 'project_domain': None, 'timestamp': u'2014-08-08T15:08:51.274902', 'auth_token': '<SANITIZED>', 'remote_address': u'10.111.81.20', 'quota_class': None, 'user_domain': None, 'is_admin': True, 'user': u'790e3caa38984fccb158b05571e1bc6c', 'service_catalog': [{u'endpoints_links': [], u'endpoints': [{u'adminURL': u'http://10.111.81.1:8774/v2/3ab47e19d21c4672973e57e7a8c85c83', u'region': u'RegionOne', u'publicURL': u'http://10.111.81.1:8774/v2/3ab47e19d21c4672973e57e7a8c85c83', u'internalURL': u'http://10.111.81.1:8774/v2/3ab47e19d21c4672973e57e7a8c85c83', u'id': u'302b287bedf54a6fa1dd51a73793ee57'}], u'type': u'compute', u'name': u'nova'}], 'request_id': u'req-28594cd1-7285-4049-baa6-faaa81653409', 'project_id': u'3ab47e19d21c4672973e57e7a8c85c83', 'read_deleted': u'no'} _safe_log /usr/lib/python2.7/dist-packages/oslo/messaging/_drivers/common.py:290

2014-08-08 11:08:51.833 12632 WARNING cinder.context [-] Arguments dropped when creating context: {'user': u'790e3caa38984fccb158b05571e1bc6c', 'tenant': u'3ab47e19d21c4672973e57e7a8c85c83', 'user_identity': u'790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -'}

2014-08-08 11:08:51.833 12632 DEBUG stevedore.extension [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] found extension EntryPoint.parse('default = taskflow.engines.action_engine.engine:SingleThreadedActionEngine') _load_plugins /usr/lib/python2.7/dist-packages/stevedore/extension.py:156

2014-08-08 11:08:51.834 12632 DEBUG stevedore.extension [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] found extension EntryPoint.parse('serial = taskflow.engines.action_engine.engine:SingleThreadedActionEngine') _load_plugins /usr/lib/python2.7/dist-packages/stevedore/extension.py:156

2014-08-08 11:08:51.834 12632 DEBUG stevedore.extension [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] found extension EntryPoint.parse('parallel = taskflow.engines.action_engine.engine:MultiThreadedActionEngine') _load_plugins /usr/lib/python2.7/dist-packages/stevedore/extension.py:156

2014-08-08 11:08:51.835 12632 DEBUG taskflow.storage [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] No backend provided, not calling functor 'taskflow.storage.save_both' _with_connection /usr/lib/python2.7/dist-packages/taskflow/storage.py:88

2014-08-08 11:08:51.835 12632 DEBUG taskflow.storage [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] No backend provided, not calling functor 'taskflow.storage.SingleThreadedStorage._save_task_detail' _with_connection /usr/lib/python2.7/dist-packages/taskflow/storage.py:88

2014-08-08 11:08:51.835 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] Starting to flatten 'taskflow.patterns.linear_flow.Flow: volume_create_scheduler; 2' _pre_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:154

2014-08-08 11:08:51.836 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] Starting to flatten 'cinder.scheduler.flows.create_volume.ExtractSchedulerSpecTask;volume:create==1.0' _pre_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:154

2014-08-08 11:08:51.836 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] Finished flattening 'cinder.scheduler.flows.create_volume.ExtractSchedulerSpecTask;volume:create==1.0' _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:159

2014-08-08 11:08:51.837 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] Translated 'cinder.scheduler.flows.create_volume.ExtractSchedulerSpecTask;volume:create==1.0' into a graph: _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:164

2014-08-08 11:08:51.837 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -]  Name: cinder.scheduler.flows.create_volume.ExtractSchedulerSpecTask;volume:create _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:167

2014-08-08 11:08:51.837 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -]  Type: DiGraph _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:167

2014-08-08 11:08:51.838 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -]  Frozen: False _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:167

2014-08-08 11:08:51.838 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -]  Nodes: 1 _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:167

2014-08-08 11:08:51.838 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -]    - cinder.scheduler.flows.create_volume.ExtractSchedulerSpecTask;volume:create==1.0 _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:167

2014-08-08 11:08:51.839 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -]  Edges: 0 _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:167

2014-08-08 11:08:51.839 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -]  Density: 0.000 _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:167

2014-08-08 11:08:51.839 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -]  Cycles: 0 _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:167

2014-08-08 11:08:51.840 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] Starting to flatten 'cinder.scheduler.flows.create_volume.schedule_create_volume==1.0' _pre_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:154

2014-08-08 11:08:51.840 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] Finished flattening 'cinder.scheduler.flows.create_volume.schedule_create_volume==1.0' _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:159

2014-08-08 11:08:51.841 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] Translated 'cinder.scheduler.flows.create_volume.schedule_create_volume==1.0' into a graph: _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:164

2014-08-08 11:08:51.841 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -]  Name: cinder.scheduler.flows.create_volume.schedule_create_volume _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:167

2014-08-08 11:08:51.841 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -]  Type: DiGraph _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:167

2014-08-08 11:08:51.842 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -]  Frozen: False _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:167

2014-08-08 11:08:51.842 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -]  Nodes: 1 _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:167

2014-08-08 11:08:51.842 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -]    - cinder.scheduler.flows.create_volume.schedule_create_volume==1.0 _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:167

2014-08-08 11:08:51.843 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -]  Edges: 0 _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:167

2014-08-08 11:08:51.843 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -]  Density: 0.000 _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:167

2014-08-08 11:08:51.843 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -]  Cycles: 0 _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:167

2014-08-08 11:08:51.844 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] Finished flattening 'taskflow.patterns.linear_flow.Flow: volume_create_scheduler; 2' _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:159

2014-08-08 11:08:51.844 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] Translated 'taskflow.patterns.linear_flow.Flow: volume_create_scheduler; 2' into a graph: _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:164

2014-08-08 11:08:51.845 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -]  Name: volume_create_scheduler _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:167

2014-08-08 11:08:51.845 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -]  Type: DiGraph _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:167

2014-08-08 11:08:51.845 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -]  Frozen: False _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:167

2014-08-08 11:08:51.845 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -]  Nodes: 2 _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:167

2014-08-08 11:08:51.846 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -]    - cinder.scheduler.flows.create_volume.ExtractSchedulerSpecTask;volume:create==1.0 _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:167

2014-08-08 11:08:51.846 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -]    - cinder.scheduler.flows.create_volume.schedule_create_volume==1.0 _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:167

2014-08-08 11:08:51.846 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -]  Edges: 1 _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:167

2014-08-08 11:08:51.847 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -]    cinder.scheduler.flows.create_volume.ExtractSchedulerSpecTask;volume:create==1.0 -> cinder.scheduler.flows.create_volume.schedule_create_volume==1.0 ({'flatten': True}) _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:167

2014-08-08 11:08:51.847 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -]  Density: 0.500 _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:167

2014-08-08 11:08:51.847 12632 DEBUG taskflow.utils.flow_utils [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -]  Cycles: 0 _post_item_flatten /usr/lib/python2.7/dist-packages/taskflow/utils/flow_utils.py:167

2014-08-08 11:08:51.848 12632 DEBUG taskflow.storage [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] No backend provided, not calling functor 'taskflow.storage.save_both' _with_connection /usr/lib/python2.7/dist-packages/taskflow/storage.py:88

2014-08-08 11:08:51.849 12632 DEBUG taskflow.storage [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] No backend provided, not calling functor 'taskflow.storage.save_both' _with_connection /usr/lib/python2.7/dist-packages/taskflow/storage.py:88

2014-08-08 11:08:51.850 12632 DEBUG taskflow.storage [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] No backend provided, not calling functor 'taskflow.storage.SingleThreadedStorage._save_flow_detail' _with_connection /usr/lib/python2.7/dist-packages/taskflow/storage.py:88

2014-08-08 11:08:51.850 12632 DEBUG taskflow.storage [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] No backend provided, not calling functor 'taskflow.storage.SingleThreadedStorage._save_task_detail' _with_connection /usr/lib/python2.7/dist-packages/taskflow/storage.py:88

2014-08-08 11:08:51.851 12632 DEBUG taskflow.storage [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] No backend provided, not calling functor 'taskflow.storage.SingleThreadedStorage._save_task_detail' _with_connection /usr/lib/python2.7/dist-packages/taskflow/storage.py:88

2014-08-08 11:08:51.851 12632 DEBUG taskflow.storage [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] No backend provided, not calling functor 'taskflow.storage.SingleThreadedStorage._save_task_detail' _with_connection /usr/lib/python2.7/dist-packages/taskflow/storage.py:88

2014-08-08 11:08:51.852 12632 DEBUG taskflow.storage [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] No backend provided, not calling functor 'taskflow.storage.SingleThreadedStorage._save_task_detail' _with_connection /usr/lib/python2.7/dist-packages/taskflow/storage.py:88

2014-08-08 11:08:51.852 12632 DEBUG taskflow.storage [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] No backend provided, not calling functor 'taskflow.storage.SingleThreadedStorage._save_task_detail' _with_connection /usr/lib/python2.7/dist-packages/taskflow/storage.py:88

2014-08-08 11:08:51.852 12632 DEBUG taskflow.storage [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] No backend provided, not calling functor 'taskflow.storage.SingleThreadedStorage._save_task_detail' _with_connection /usr/lib/python2.7/dist-packages/taskflow/storage.py:88

2014-08-08 11:08:51.857 12632 WARNING cinder.scheduler.host_manager [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] volume service is down or disabled. (host: cinder-dev01@ceph)

2014-08-08 11:08:51.857 12632 WARNING cinder.scheduler.host_manager [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] volume service is down or disabled. (host: cinder-dev01@e5524iscsi)

2014-08-08 11:08:51.858 12632 DEBUG cinder.openstack.common.scheduler.filters.capabilities_filter [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] extra_spec requirement 'NetApp_ESeries' does not match '3250vs1' _satisfies_extra_specs /usr/lib/python2.7/dist-packages/cinder/openstack/common/scheduler/filters/capabilities_filter.py:55

2014-08-08 11:08:51.858 12632 DEBUG cinder.openstack.common.scheduler.filters.capabilities_filter [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] host 'cinder-dev01@3250direct1': free_capacity_gb: 149 fails resource_type extra_specs requirements host_passes /usr/lib/python2.7/dist-packages/cinder/openstack/common/scheduler/filters/capabilities_filter.py:68

2014-08-08 11:08:51.859 12632 DEBUG cinder.openstack.common.scheduler.filters.capabilities_filter [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] extra_spec requirement 'NetApp_ESeries' does not match 'emc_vmax_1' _satisfies_extra_specs /usr/lib/python2.7/dist-packages/cinder/openstack/common/scheduler/filters/capabilities_filter.py:55

2014-08-08 11:08:51.859 12632 DEBUG cinder.openstack.common.scheduler.filters.capabilities_filter [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] host 'cinder-dev01@iscsi_emc_vmax_1': free_capacity_gb: unknown fails resource_type extra_specs requirements host_passes /usr/lib/python2.7/dist-packages/cinder/openstack/common/scheduler/filters/capabilities_filter.py:68

2014-08-08 11:08:51.860 12632 DEBUG cinder.openstack.common.scheduler.filters.capabilities_filter [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] extra_spec requirement 'NetApp_ESeries' does not match '3250vs3' _satisfies_extra_specs /usr/lib/python2.7/dist-packages/cinder/openstack/common/scheduler/filters/capabilities_filter.py:55

2014-08-08 11:08:51.860 12632 DEBUG cinder.openstack.common.scheduler.filters.capabilities_filter [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] host 'cinder-dev01@3250direct3': free_capacity_gb: 399 fails resource_type extra_specs requirements host_passes /usr/lib/python2.7/dist-packages/cinder/openstack/common/scheduler/filters/capabilities_filter.py:68

2014-08-08 11:08:51.861 12632 DEBUG cinder.openstack.common.scheduler.filters.capabilities_filter [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] extra_spec requirement 'NetApp_ESeries' does not match '3250vs2' _satisfies_extra_specs /usr/lib/python2.7/dist-packages/cinder/openstack/common/scheduler/filters/capabilities_filter.py:55

2014-08-08 11:08:51.861 12632 DEBUG cinder.openstack.common.scheduler.filters.capabilities_filter [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] host 'cinder-dev01@3250direct2': free_capacity_gb: 377 fails resource_type extra_specs requirements host_passes /usr/lib/python2.7/dist-packages/cinder/openstack/common/scheduler/filters/capabilities_filter.py:68

2014-08-08 11:08:51.861 12632 DEBUG cinder.scheduler.filter_scheduler [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] Filtered [host 'cinder-dev01@NetApp_ESeries': free_capacity_gb: 34276] _get_weighted_candidates /usr/lib/python2.7/dist-packages/cinder/scheduler/filter_scheduler.py:259

2014-08-08 11:08:51.862 12632 DEBUG cinder.scheduler.filter_scheduler [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] Choosing cinder-dev01@NetApp_ESeries _choose_top_host /usr/lib/python2.7/dist-packages/cinder/scheduler/filter_scheduler.py:276

2014-08-08 11:08:51.902 12632 DEBUG oslo.messaging._drivers.amqp [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] UNIQUE_ID is 87e50cc925184c788a711fd6c127f493. _add_unique_id /usr/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqp.py:338

2014-08-08 11:08:51.911 12632 DEBUG taskflow.storage [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] No backend provided, not calling functor 'taskflow.storage.SingleThreadedStorage._save_task_detail' _with_connection /usr/lib/python2.7/dist-packages/taskflow/storage.py:88

2014-08-08 11:08:51.913 12632 DEBUG taskflow.storage [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] No backend provided, not calling functor 'taskflow.storage.SingleThreadedStorage._save_task_detail' _with_connection /usr/lib/python2.7/dist-packages/taskflow/storage.py:88

2014-08-08 11:08:51.914 12632 DEBUG taskflow.storage [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] No backend provided, not calling functor 'taskflow.storage.SingleThreadedStorage._save_flow_detail' _with_connection /usr/lib/python2.7/dist-packages/taskflow/storage.py:88

2014-08-08 11:08:51.915 12632 DEBUG oslo.messaging._drivers.amqp [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] UNIQUE_ID is 68d7b06ec351464a943e7c4990ed91ae. _add_unique_id /usr/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqp.py:338

2014-08-08 11:08:51.917 12632 DEBUG oslo.messaging._drivers.amqp [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] UNIQUE_ID is 995a17dcfb63479098cf05b9901f0768. _add_unique_id /usr/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqp.py:338

At the end of the third try it adds:

2014-08-08 11:09:02.085 12632 ERROR cinder.scheduler.flows.create_volume [req-28594cd1-7285-4049-baa6-faaa81653409 790e3caa38984fccb158b05571e1bc6c 3ab47e19d21c4672973e57e7a8c85c83 - - -] Failed to schedule_create_volume: No valid host was found. Exceeded max scheduling attempts 3 for volume None

I have already tried changing the port in the cinder.conf to 80 (I disabled SSL and 8443 port edditing the wsconfig.xml file in the proxy:

<?xml version="1.0" encoding="UTF-8"?>^M

<config version="1">^M

^M

    <port>8080</port>^M

    <workingdir>working</workingdir>^M

    <env-entries>

        <env key="enable-basic-auth">true</env>

    </env-entries>

</config>

I don't know what else to do... Any help will be greatly appreciated.

Thanks in advance!

[rcallawa]

What does the cinder-volume log say?

[JULIANDIAZ · Accepted solution]

Actually the problem was solved by modifying the cinder.conf, I added this line: volume_backend_name=<name used to define the backend type>

And I also had to remove the Admin password on the storage (via santricity) and remove this line on the cinder.conf: netapp_sa_password=<arrayPassword>

I'm not sure if the password I set using Santricity is the password I should use here because it's very weird that it didn't authenticate with it...

[rcallawa]

netapp_sa_password is only authenticated when privileged operations are executed (create volume, delete volume, etc).

[JULIANDIAZ]

Hmmm, that's interesting, do you know if that password is the one that is configured using SANtricity under "Setup"/"Set a storage array password" because as far as I could see there are only 2 users already set up using SANtricity, one is "Administrator" and the other is "Monitor" and the only way I could get it to work is clearing the "Administrator" pwd and errasing the: netapp_sa_password=<arrayPassword> line in cinder.conf

I think there may be another user/pwd somewhere else but I'm not sure...