OpenStack Discussions

I try CopyOffload

takeshik
10,355 Views

I can set CopyOffload.

However, I do not know how to check if they could be set correctly.

What should I check or test?

Regards,

Takeshi.K

1 ACCEPTED SOLUTION

rcallawa
8,744 Views

It doesn't appear that Red Hat has created a new RPM for RDO on Fedora 21 yet:

https://repos.fedorapeople.org/repos/openstack/openstack-icehouse/fedora-20/

You could manually update the file /usr/lib/python2.7/site-packages/cinder/image/glance.py - line 434, to look like:

output[attr] = getattr(image, attr, None)

to see if this resolves your issue.

View solution in original post

16 REPLIES 16

rcallawa
10,278 Views

If the copy offload operation succeeds, you should see an info-level log message in Cinder logs stating "Copied image %(img)s to volume %(vol)s using copy offload workflow"

If the copy offload operation fails, you should see a debug level message in Cinder logs stating "Copy offload either not configured or unsupported." or an exception "Copy offload workflow unsuccessful".

takeshik
10,278 Views

I cloud not find these messages in any the cinder log.

By the way, Please tell me some example operation to see the copy offload operation.

Regards,

Takeshi.K

rcallawa
10,278 Views

If you use the cinder command line tool to issue the following command (replacing 01234567890 with the id of an image in your glance repository)

cinder create --image-id 01234567890 20

This will trigger the creation of a new Cinder volume whose source content comes from Glance.

Be sure that Glance is configured to use an NFS export from a different FlexVol volume on the same cluster where the Cinder backend is defined.

takeshik
10,278 Views

Thank you.

I tried a command.

[root@tkstack glance(keystone_admin)]# cinder create --image-id 17a544b2-1bf8-4b1c-954a-c3a405b51fbd 20

ERROR: The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-02de21a7-02ab-48f6-a670-4097910ea322)

There may be  some problems..

rcallawa
10,278 Views

Without logs from Cinder, I can't help much more.

Please provide relevant logs from Cinder, and the output from these two commands:

glance image-list

cinder service-list

takeshik
10,281 Views

Thank you.

Exactly,

When I will retry , I would like to provide some logs and outputs from commands.

takeshik
10,278 Views

When I run this command , volume.log and schedule.log were not output anything. api.log was output.

cinder create --image-id 17a544b2-1bf8-4b1c-954a-c3a405b51fbd 20

2014-08-02 20:51:20.492 7776 INFO eventlet.wsgi.server [-] (7776) accepted ('10.130.205.95', 57096)

2014-08-02 20:51:20.496 7776 INFO urllib3.connectionpool [-] Starting new HTTP connection (1): 10.130.205.95

2014-08-02 20:51:20.551 7776 INFO cinder.api.openstack.wsgi [req-60e69d94-25b0-40dc-b0c2-0547084a790f 13d3305af9e94e888b451e3775dd09ac 51907191e78543afa754de6b69624205 - - -] POST http://10.130.205.95:8776/v1/51907191e78543afa754de6b69624205/volumes

2014-08-02 20:51:20.552 7776 AUDIT cinder.api.v1.volumes [req-60e69d94-25b0-40dc-b0c2-0547084a790f 13d3305af9e94e888b451e3775dd09ac 51907191e78543afa754de6b69624205 - - -] Create volume of 20 GB

2014-08-02 20:51:20.720 7776 ERROR cinder.api.middleware.fault [req-60e69d94-25b0-40dc-b0c2-0547084a790f 13d3305af9e94e888b451e3775dd09ac 51907191e78543afa754de6b69624205 - - -] Caught error: deleted

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault Traceback (most recent call last):

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/cinder/api/middleware/fault.py", line 75, in __call__

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     return req.get_response(self.application)

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1296, in send

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     application, catch_exc_info=False)

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1260, in call_application

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     app_iter = application(self.environ, start_response)

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 144, in __call__

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     return resp(environ, start_response)

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/keystoneclient/middleware/auth_token.py", line 679, in __call__

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     return self.app(env, start_response)

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 144, in __call__

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     return resp(environ, start_response)

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 144, in __call__

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     return resp(environ, start_response)

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/routes/middleware.py", line 131, in __call__

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     response = self.app(environ, start_response)

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 144, in __call__

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     return resp(environ, start_response)

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 130, in __call__

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     resp = self.call_func(req, *args, **self.kwargs)

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 195, in call_func

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     return self.func(req, *args, **kwargs)

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/cinder/api/openstack/wsgi.py", line 895, in __call__

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     content_type, body, accept)

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/cinder/api/openstack/wsgi.py", line 943, in _process_stack

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     action_result = self.dispatch(meth, request, action_args)

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/cinder/api/openstack/wsgi.py", line 1019, in dispatch

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     return method(req=request, **action_args)

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/cinder/api/v1/volumes.py", line 432, in create

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     **kwargs)

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/cinder/volume/api.py", line 189, in create

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     flow_engine.run()

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/taskflow/utils/lock_utils.py", line 51, in wrapper

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     return f(*args, **kwargs)

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 118, in run

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     self._run()

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 128, in _run

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     self._revert(misc.Failure())

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/engine.py", line 81, in _revert

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     misc.Failure.reraise_if_any(failures.values())

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/taskflow/utils/misc.py", line 487, in reraise_if_any

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     failures[0].reraise()

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/taskflow/utils/misc.py", line 494, in reraise

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     six.reraise(*self._exc_info)

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 36, in _execute_task

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     result = task.execute(**arguments)

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/api/create_volume.py", line 341, in execute

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     self._check_image_metadata(context, image_id, size)

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/cinder/volume/flows/api/create_volume.py", line 180, in _check_image_metadata

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     image_meta = self.image_service.show(context, image_id)

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 228, in show

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     base_image_meta = self._translate_from_glance(image)

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 336, in _translate_from_glance

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     image_meta = _extract_attributes(image)

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/cinder/image/glance.py", line 434, in _extract_attributes

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     output[attr] = getattr(image, attr)

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault   File "/usr/lib/python2.7/site-packages/warlock/model.py", line 69, in __getattr__

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault     raise AttributeError(key)

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault AttributeError: deleted

2014-08-02 20:51:20.720 7776 TRACE cinder.api.middleware.fault

2014-08-02 20:51:20.723 7776 INFO cinder.api.middleware.fault [req-60e69d94-25b0-40dc-b0c2-0547084a790f 13d3305af9e94e888b451e3775dd09ac 51907191e78543afa754de6b69624205 - - -] http://10.130.205.95:8776/v1/51907191e78543afa754de6b69624205/volumes returned with HTTP 500

2014-08-02 20:51:20.726 7776 INFO eventlet.wsgi.server [req-60e69d94-25b0-40dc-b0c2-0547084a790f 13d3305af9e94e888b451e3775dd09ac 51907191e78543afa754de6b69624205 - - -] 10.130.205.95 - - [02/Aug/2014 20:51:20] "POST /v1/51907191e78543afa754de6b69624205/volumes HTTP/1.1" 500 401 0.233147

Two command

[root@tkstack ~(keystone_admin)]# glance image-list

+--------------------------------------+-------------------+-------------+------------------+-----------+--------+

| ID                                   | Name              | Disk Format | Container Format | Size      | Status |

+--------------------------------------+-------------------+-------------+------------------+-----------+--------+

| 309e930a-7eba-483c-9bc8-d8880b392d45 | CentOS-6.5_x86_64 | qcow2       | bare             | 344457216 | active |

| 17a544b2-1bf8-4b1c-954a-c3a405b51fbd | cirros            | qcow2       | bare             | 13147648  | active |

| 156e8961-aedc-4879-8b42-903236f2e8d8 | Fedora 20 x86_64  | qcow2       | bare             | 210829312 | active |

+--------------------------------------+-------------------+-------------+------------------+-----------+--------+

[root@tkstack ~(keystone_admin)]# cinder service-list

+------------------+-------------------------------+------+---------+-------+----------------------------+-----------------+

|      Binary      |              Host             | Zone |  Status | State |         Updated_at         | Disabled Reason |

+------------------+-------------------------------+------+---------+-------+----------------------------+-----------------+

|  cinder-backup   |       tkstack.sample.net      | nova | enabled |   up  | 2014-08-02T11:50:04.000000 |       None      |

| cinder-scheduler |       tkstack.sample.net      | nova | enabled |   up  | 2014-08-02T11:50:07.000000 |       None      |

|  cinder-volume   |       tkstack.sample.net      | nova | enabled |  down | 2014-07-28T07:00:45.000000 |       None      |

|  cinder-volume   | tkstack.sample.net@cdot-iscsi | nova | enabled |   up  | 2014-08-02T11:50:08.000000 |       None      |

|  cinder-volume   |  tkstack.sample.net@cdot-nfs  | nova | enabled |   up  | 2014-08-02T11:50:08.000000 |       None      |

|  cinder-volume   |  tkstack.sample.net@lvmdriver | nova | enabled |  down | 2014-07-31T15:19:05.000000 |       None      |

+------------------+-------------------------------+------+---------+-------+----------------------------+-----------------+

[root@tkstack ~(keystone_admin)]#

takeshik
10,278 Views

Additional Information.

When I edited from "glance_api_version=2" to "glance_api_version=1" (or comment out this parameter)  in cinder.conf,

this command succeeded.

[root@tkstack cinder(keystone_admin)]# cinder create --image-id 17a544b2-1bf8-4b1c-954a-c3a405b51fbd 20

+---------------------+--------------------------------------+

|       Property      |                Value                 |

+---------------------+--------------------------------------+

|     attachments     |                  []                  |

|  availability_zone  |                 nova                 |

|       bootable      |                false                 |

|      created_at     |      2014-08-02T12:02:29.168455      |

| display_description |                 None                 |

|     display_name    |                 None                 |

|      encrypted      |                False                 |

|          id         | 0d8e77c5-75ed-4725-9aed-3e5609a0fe87 |

|       image_id      | 17a544b2-1bf8-4b1c-954a-c3a405b51fbd |

|       metadata      |                  {}                  |

|         size        |                  20                  |

|     snapshot_id     |                 None                 |

|     source_volid    |                 None                 |

|        status       |               creating               |

|     volume_type     |                 None                 |

+---------------------+--------------------------------------+

[root@tkstack cinder(keystone_admin)]#

Also , I could find "Copy Offload" by netapp driver in volume.log.

2014-08-02 21:02:31.382 15478 INFO cinder.volume.drivers.netapp.nfs [req-c1cdec7c-fb37-4bb8-9845-ed8781fefde2 13d3305af9e94e888b451e3775dd09ac 51907191e78543afa754de6b69624205 - - -] Copied image to volume volume-0d8e77c5-75ed-4725-9aed-3e5609a0fe87 using regular download.

2014-08-02 21:02:31.382 15478 INFO cinder.volume.drivers.netapp.nfs [req-c1cdec7c-fb37-4bb8-9845-ed8781fefde2 13d3305af9e94e888b451e3775dd09ac 51907191e78543afa754de6b69624205 - - -] Registering image in cache img-cache-17a544b2-1bf8-4b1c-954a-c3a405b51fbd

2014-08-02 21:02:31.384 15478 INFO cinder.volume.drivers.netapp.nfs [req-c1cdec7c-fb37-4bb8-9845-ed8781fefde2 13d3305af9e94e888b451e3775dd09ac 51907191e78543afa754de6b69624205 - - -] Cloning from cache to destination img-cache-17a544b2-1bf8-4b1c-954a-c3a405b51fbd

Is this correct?

If this configuration is correct , Should I use "1" for an api version of glance in cinder.conf ?

Thanks,

rcallawa
10,278 Views

Yes, 2 is the correct value for api version of Glance.

Please see the upstream documentation for all the settings/configuration that is required for copy offload:

http://docs.openstack.org/icehouse/config-reference/content/ontap-cluster-nfs.html#d6e2969

takeshik
8,670 Views

I confused.

I have already read below document.

http://docs.openstack.org/icehouse/config-reference/content/ontap-cluster-nfs.html#d6e2969

When a glance_api_version = 2 , the command did not succeed.

When a glance_api_version = 1 , the command succeed.

Should I use "version 2" to use NetApp unified driver for copy offload?

However I seems it works fine with "version 1" too.

rcallawa
8,670 Views

You have hit a bug in Cinder:

https://bugs.launchpad.net/cinder/+bug/1308058

This appears to be fixed in the first service release of Icehouse, 2014.1.1 - which distribution of OpenStack are you using?

takeshik
8,670 Views

Thank you

I use RDO Icehouse release.

https://github.com/redhat-openstack/rdo-release

[root@tkstack glance(keystone_admin)]# rpm -qi openstack-cinder-2014.1-2.fc21.noarch

Name        : openstack-cinder

Version     : 2014.1

Release     : 2.fc21

Architecture: noarch

Install Date: 2014年07月23日 16時36分06秒

Group       : Applications/System

Size        : 134169

License     : ASL 2.0

Signature   : RSA/SHA1, 2014年04月23日 23時08分35秒, Key ID e50be6ab0e4fbd28

Source RPM  : openstack-cinder-2014.1-2.fc21.src.rpm

Build Date  : 2014年04月22日 04時48分42秒

Build Host  : arm02-builder11.arm.fedoraproject.org

Relocations : (not relocatable)

Packager    : Fedora Project

Vendor      : Fedora Project

URL         : http://www.openstack.org/software/openstack-storage/

Summary     : OpenStack Volume service

Description :

OpenStack Volume (codename Cinder) provides services to manage and

access block storage volumes for use by Virtual Machine instances.

[root@tkstack glance(keystone_admin)]#

rcallawa
8,745 Views

It doesn't appear that Red Hat has created a new RPM for RDO on Fedora 21 yet:

https://repos.fedorapeople.org/repos/openstack/openstack-icehouse/fedora-20/

You could manually update the file /usr/lib/python2.7/site-packages/cinder/image/glance.py - line 434, to look like:

output[attr] = getattr(image, attr, None)

to see if this resolves your issue.

takeshik
8,672 Views

Thanks,

After I edited a glancy.py ,  I tried again.

It was works fine.

By the way , What should I check in cDOT about this copy offload?

For example, clone status , snapshot show or lun show or  any messages.

rcallawa
8,673 Views

You can go into diag mode and look at the value of the copy_reqs counter - it should be increasing by 1 every time a copy offload operation is successful.

login: admin

Password:

rcallawacluster-1::> set diag

Warning: These diagnostic commands are for use by NetApp personnel only.

Do you want to continue? {y|n}: y

rcallawacluster-1::*> statistics start -object copy_manager

Statistics collection is being started for Sample-id: sample_12

rcallawacluster-1::*> statistics show -object copy_manager -instance rcallawa-nfs-vserver -counter copy_reqs

Object: copy_manager

Instance: rcallawa-nfs-vserver

Start-time: 8/2/2014 14:43:56

End-time: 8/2/2014 14:44:34

Cluster: rcallawacluster-1

Number of Constituents: 2 (complete_aggregation)

    Counter                                                     Value

    -------------------------------- --------------------------------

    copy_reqs                                                       0

takeshik
6,401 Views

Thank you

I will try it again .

Public