Hello,
I am trying to debug this situation: our openstack people deployed kilo a while ago. Since apparently last Friday that they noticed that after resizing a lun they couldnt mount/attach it back to same VM.
In production I am running kilo on a clustered ONTAP 8.2.2P1, we get this error while trying to reattach:
2015-11-02 09:34:36.493 21340 ERROR oslo_messaging._drivers.common [req-62d7c63f-efea-4417-893f-b81c7cc5c8cd bukowiec 4d679467-f828-41bc-90fa-ef8633594a6f - - -] Returning exception Bad or unexpected response from the storage volume backend API: Unable to fetch connection information from backend: NetApp API failed. Reason - 13003:Insufficient privileges: user 'cinder_admin' does not have read access to this resource to caller
cinder_admin have following privilages as per doc:http://netapp.github.io/openstack-deploy-ops-guide/kilo/openstack-deployment-ops-guide.pdf
rac51::*> security login role show -vserver rac51 -role cinder_cluster
Role Command/ Access
Vserver Name Directory Query Level
---------- ------------- --------- ----------------------------------- --------
rac51 cinder_cluster
DEFAULT none
rac51 cinder_cluster
event all
rac51 cinder_cluster
lun readonly
rac51 cinder_cluster
lun create all
rac51 cinder_cluster
lun delete all
rac51 cinder_cluster
lun igroup readonly
rac51 cinder_cluster
lun igroup add all
rac51 cinder_cluster
lun igroup create all
rac51 cinder_cluster
lun igroup modify all
rac51 cinder_cluster
lun igroup show all
rac51 cinder_cluster
lun map all
Role Command/ Access
Vserver Name Directory Query Level
---------- ------------- --------- ----------------------------------- --------
rac51 cinder_cluster
lun mapped readonly
rac51 cinder_cluster
lun modify all
rac51 cinder_cluster
lun resize all
rac51 cinder_cluster
lun show all
rac51 cinder_cluster
lun unmap all
rac51 cinder_cluster
security readonly
rac51 cinder_cluster
snapmirror readonly
rac51 cinder_cluster
storage aggregate readonly
rac51 cinder_cluster
storage disk readonly
rac51 cinder_cluster
version all
Role Command/ Access
Vserver Name Directory Query Level
---------- ------------- --------- ----------------------------------- --------
rac51 cinder_cluster
volume readonly
rac51 cinder_cluster
volume efficiency readonly
rac51 cinder_cluster
volume file clone create all
rac51 cinder_cluster
vserver readonly
rac51 cinder_cluster
vserver iscsi readonly
rac51 cinder_cluster
vserver iscsi interface readonly
27 entries were displayed.
Sorry for the formating.
One of my colleagues has done a test setup of openstack and I have done the same on a test ONTAP cluster, this one running ONTAP 8.3.1. I get similar errors:
2015-11-02 16:53:56.222 17313 WARNING cinder.volume.drivers.netapp.dataontap.client.client_base [req-eec18ed9-147e-4a8d-8943-9ad895f69497 9ad073f097c347509aee2414c0021f27 81b42e48d34446da9688d691078cbcd6 - - -] Error mapping LUN. Code :13003, Message:Insufficient privileges: user 'ois_admin' does not have write access to this resource
2015-11-02 16:53:56.459 17313 ERROR cinder.volume.manager [req-eec18ed9-147e-4a8d-8943-9ad895f69497 9ad073f097c347509aee2414c0021f27 81b42e48d34446da9688d691078cbcd6 - - -] Unable to fetch connection information from backend: NetApp API failed. Reason - 13003:Insufficient privileges: user 'ois_admin' does not have write access to this resource
2015-11-02 16:53:56.461 17313 ERROR oslo_messaging.rpc.dispatcher [req-eec18ed9-147e-4a8d-8943-9ad895f69497 9ad073f097c347509aee2414c0021f27 81b42e48d34446da9688d691078cbcd6 - - -] Exception during message handling: Bad or unexpected response from the storage volume backend API: Unable to fetch connection information from backend: NetApp API failed. Reason - 13003:Insufficient privileges: user 'ois_admin' does not have write access to this resource
Again while trying to attach the volume to a vm:
[root@ps-kilo-temp cinder(keystone_admin)]# nova volume-attach c5db89c5-7dbf-4d39-88ff-59cc16a85c57 49196581-99df-4351-b689-9351af0a4f4f auto
+----------+--------------------------------------+
| Property | Value |
+----------+--------------------------------------+
| device | /dev/vdb |
| id | 49196581-99df-4351-b689-9351af0a4f4f |
| serverId | c5db89c5-7dbf-4d39-88ff-59cc16a85c57 |
| volumeId | 49196581-99df-4351-b689-9351af0a4f4f |
+----------+--------------------------------------+
[root@ps-kilo-temp cinder(keystone_admin)]# cinder list
+--------------------------------------+----------------+--------------+------+-------------+----------+-------------+
| ID | Status | Display Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+----------------+--------------+------+-------------+----------+-------------+
| 1f4464d2-f954-4874-93c7-90622dd7ba0a | available | cephv01 | 1 | ceph | false | |
| 400b4ea6-08f2-427a-88b6-221d638a8902 | available | testvol01 | 1 | netapp | false | |
| 49196581-99df-4351-b689-9351af0a4f4f | available | testvol04 | 1 | netapp | false | |
| 63b1b607-7f40-4b5f-b9b5-c2768d93e1a7 | deleting | testvol03 | 2 | netapp | false | |
| f8cfc64c-a9c9-4588-b1d3-0ed69fa41c74 | error_deleting | testvol02 | 2 | netapp | false | |
+--------------------------------------+----------------+--------------+------+-------------+----------+-------------+
[root@ps-kilo-temp cinder(keystone_admin)]# nova volume-attach c5db89c5-7dbf-4d39-88ff-59cc16a85c57 49196581-99df-4351-b689-9351af0a4f4f auto
+----------+--------------------------------------+
| Property | Value |
+----------+--------------------------------------+
| device | /dev/vdb |
| id | 49196581-99df-4351-b689-9351af0a4f4f |
| serverId | c5db89c5-7dbf-4d39-88ff-59cc16a85c57 |
| volumeId | 49196581-99df-4351-b689-9351af0a4f4f |
+----------+--------------------------------------+
The security is as per doc again:
c02::*> security login role show -vserver c02 -role openstach
Role Command/ Access
Vserver Name Directory Query Level
---------- ------------- --------- ----------------------------------- --------
c02 openstach DEFAULT none
c02 openstach event all
c02 openstach lun readonly
c02 openstach lun create all
c02 openstach lun delete all
c02 openstach lun igroup readonly
c02 openstach lun igroup add all
c02 openstach lun igroup create all
c02 openstach lun igroup modify all
c02 openstach lun igroup show all
c02 openstach lun map all
c02 openstach lun mapped readonly
c02 openstach lun modify all
c02 openstach lun resize all
c02 openstach lun show all
c02 openstach lun unmap all
c02 openstach security readonly
c02 openstach snapmirror readonly
c02 openstach storage aggregate readonly
c02 openstach storage disk readonly
c02 openstach version all
c02 openstach volume readonly
c02 openstach volume efficiency readonly
c02 openstach volume file clone create all
c02 openstach vserver readonly
c02 openstach vserver iscsi readonly
c02 openstach vserver iscsi interface readonly
27 entries were displayed.
Please could you let me know what do you think, which permissions I am missing. Still astonished this started to fail now, this operations has been done plenty of times in the past.
THank you