OpenStack Discussions

cinder extend error with iscsi

youngs
4,755 Views

We have been doing Openstack POC..

we have an Error during trying to extend cinder volume in cDOT iSCSI FlexVol. ..(  ubuntu 12.04 KVM .. Havana version .. cDOT 8.2P3, flexclone enabled, iscsi FlexVol 3TB , available size 2.9TB )

Cinder volume in cDOT NFS FlexVol was OK  . Also manual LUN extend was OK in FAS.

It is very critical issue and I can’t find any same cases in google.

I think that it is maybe our Openstack iSCSI Driver bug. ( I also tested it with devstack in our LAB and got a same problem..  )

Please help..

2014-02-17 16:00:09.679 9716 ERROR cinder.volume.manager [req-00241cbd-6535-4459-8e95-451cc01b3bb5 3712ec7a451f452ebf0903bbff517c85 ca966fc251414a5caf81f3c62a994527] volume 1a2ff3ac-df8e-45a4-81d8-dbef238e39f2: Error trying to extend volume
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager Traceback (most recent call last):
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager   File "/opt/openstack/cinder/cinder/volume/manager.py", line 877, in extend_volume
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager     self.driver.extend_volume(volume, new_size)
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager   File "/opt/openstack/cinder/cinder/volume/drivers/netapp/iscsi.py", line 609, in extend_volume
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager     self._do_sub_clone_resize(path, new_size_bytes)
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager   File "/opt/openstack/cinder/cinder/volume/drivers/netapp/iscsi.py", line 713, in _do_sub_clone_resize
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager     self._destroy_lun(new_path)
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager   File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager     self.gen.next()
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager   File "/opt/openstack/cinder/cinder/volume/drivers/netapp/iscsi.py", line 708, in _do_sub_clone_resize
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager     self._clone_lun(name, new_lun, block_count=block_count)
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager   File "/opt/openstack/cinder/cinder/volume/drivers/netapp/iscsi.py", line 988, in _clone_lun
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager     self.client.invoke_successfully(clone_create, True)
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager   File "/opt/openstack/cinder/cinder/volume/drivers/netapp/api.py", line 222, in invoke_successfully
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager     raise NaApiError(code, msg)
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager NaApiError: NetApp api failed. Reason - 13115:Integer value 46137344 out of range [1..2^24] for block-count
2014-02-17 16:00:09.679 9716 TRACE cinder.volume.manager

1 ACCEPTED SOLUTION

akerr
4,755 Views

Youngs,

I have identified the bug and I have submitted a patch for review in the community.  Initially this is going into master (icehouse) but I think we should be able to back port easily enough to Havana.  In the mean time, you can view the patch and make the code changes manually if you would like to test it out: https://review.openstack.org/74524

View solution in original post

4 REPLIES 4

akerr
4,755 Views

How are you attempting to extend the volume(command line cinder client, Horizon GUI, or custom).  Can you provide the exact steps to reproduce?  What was the original size of the volume and the requested extended size of the volume?  I have attempted this myself using the cinder client with stable/Havana devstack on ubuntu 12.04.4LTS and could not reproduce.  My flexvol was 50GB in size, but allowed to automatically grow, and my cDOT is running 8.2P3.

akerr@openstack4:~$ tail /etc/cinder/cinder.conf

[cmodeiSCSI]

netapp_password = xxx

netapp_login = admin

netapp_vserver = test

netapp_storage_family = ontap_cluster

netapp_storage_protocol = iscsi

netapp_server_port = 80

netapp_server_hostname = x.x.x.x

volume_driver = cinder.volume.drivers.netapp.common.NetAppDriver

akerr@openstack4:~/devstack$ cinder create 1

+---------------------+--------------------------------------+

|       Property      |                Value                 |

+---------------------+--------------------------------------+

|     attachments     |                  []                  |

|  availability_zone  |                 nova                 |

|       bootable      |                false                 |

|      created_at     |      2014-02-17T15:22:47.170709      |

| display_description |                 None                 |

|     display_name    |                 None                 |

|          id         | d8531963-525a-4d24-94da-e04a8c3be1d3 |

|       metadata      |                  {}                  |

|         size        |                  1                   |

|     snapshot_id     |                 None                 |

|     source_volid    |                 None                 |

|        status       |               creating               |

|     volume_type     |                 None                 |

+---------------------+--------------------------------------+

akerr@openstack4:~/devstack$ cinder list

+--------------------------------------+-----------+--------------+------+-------------+----------+-------------+

|                  ID                  |   Status  | Display Name | Size | Volume Type | Bootable | Attached to |

+--------------------------------------+-----------+--------------+------+-------------+----------+-------------+

| d8531963-525a-4d24-94da-e04a8c3be1d3 | available |     None     |  1   |     None    |  false   |             |

+--------------------------------------+-----------+--------------+------+-------------+----------+-------------+

akerr@openstack4:~/devstack$ cinder extend d8531963-525a-4d24-94da-e04a8c3be1d3 2

akerr@openstack4:~/devstack$ cinder list

+--------------------------------------+-----------+--------------+------+-------------+----------+-------------+

|                  ID                  |   Status  | Display Name | Size | Volume Type | Bootable | Attached to |

+--------------------------------------+-----------+--------------+------+-------------+----------+-------------+

| d8531963-525a-4d24-94da-e04a8c3be1d3 | available |     None     |  2   |     None    |  false   |             |

+--------------------------------------+-----------+--------------+------+-------------+----------+-------------+

akerr
4,755 Views

I have been able to reproduce this issue as well and have opened https://bugs.launchpad.net/cinder/+bug/1281279 to track the issue.

The failure will appear after a pseudo-random number of extends against the same volume.  The exact number of times it takes seems somewhat dependent on the original size of the volume and the size of the extensions.  The following sequence appears to always cause the failure:

akerr@openstack4:~/devstack$ cinder create 4

+---------------------+--------------------------------------+

|       Property      |                Value                 |

+---------------------+--------------------------------------+

|     attachments     |                  []                  |

|  availability_zone  |                 nova                 |

|       bootable      |                false                 |

|      created_at     |      2014-02-17T20:25:44.961265      |

| display_description |                 None                 |

|     display_name    |                 None                 |

|          id         | 5ec14ba8-2da0-4e19-8594-9bee9a6ab641 |

|       metadata      |                  {}                  |

|         size        |                  4                   |

|     snapshot_id     |                 None                 |

|     source_volid    |                 None                 |

|        status       |               creating               |

|     volume_type     |                 None                 |

+---------------------+--------------------------------------+

akerr@openstack4:~/devstack$ cinder extend 5ec14ba8-2da0-4e19-8594-9bee9a6ab641 8

akerr@openstack4:~/devstack$ cinder extend 5ec14ba8-2da0-4e19-8594-9bee9a6ab641 20

akerr@openstack4:~/devstack$ cinder extend 5ec14ba8-2da0-4e19-8594-9bee9a6ab641 30

akerr@openstack4:~/devstack$ cinder list

+--------------------------------------+-----------------+--------------+------+-------------+----------+-------------+

|                  ID                  |      Status     | Display Name | Size | Volume Type | Bootable | Attached to |

+--------------------------------------+-----------------+--------------+------+-------------+----------+-------------+

| 5ec14ba8-2da0-4e19-8594-9bee9a6ab641 | error_extending |     None     |  20  |     None    |  false   |             |

+--------------------------------------+-----------------+--------------+------+-------------+----------+-------------+

alexmc
4,755 Views

16777216 is 2^24 blocks. The API is being asked for something that’s way too large at 46137344 blocks.  What is the space request you’re making for the LUN extend?

akerr
4,756 Views

Youngs,

I have identified the bug and I have submitted a patch for review in the community.  Initially this is going into master (icehouse) but I think we should be able to back port easily enough to Havana.  In the mean time, you can view the patch and make the code changes manually if you would like to test it out: https://review.openstack.org/74524

Public