Unscheduled Random Flex Clones

[kambadwal]

we are getting random Flex clones that are happing to a volume.  The volume has  282 GB Free and the total space is 3 TB

we are using Nseries 6060 and running 8.0.1

we get the vclone busy in our snapshots and can not delete.  we have to delete the Lun for the sd_cl and then delete the flex volume.

does anyone have any suggestions what could be causing this?

[SAN04-A: api_mpool_01:notice]: Multicreation of snapshot named {cc962dd9-3e07-4a0e-828e-d351dfd8c1e5} successful. It took 764 milli seconds from start to finish in ZAPI.

[SAN04-%3A+lun.offline%3Awarning%5D: LUN /vol/vol8_SAN04_A/{55fa623d-df27-4f13-8abc-967645f29010}.aux has been taken offline

[SAN04-%3A+lun.destroy%3Ainfo%5D: LUN /vol/vol8_SAN04_A/{55fa623d-df27-4f13-8abc-967645f29010}.aux destroyed

[SAN04-%3A+wafl.volume.clone.created%3Ainfo%5D: Volume clone sdw_cl_vol8_SAN04_A_0 of volume vol8_SAN04_A was created successfully.

[SAN04-%3A+lun.newLocation.offline%3Awarning%5D: LUN /vol/sdw_cl_vol8_SAN04_A_0/WEBSS02_FC_LUN1 has been taken offline to prevent map conflicts after a copy or move operation.

[SAN04-%3A+lun.newLocation.offline%3Awarning%5D: LUN /vol/sdw_cl_vol8_SAN04_A_0/FC_LUN3 has been taken offline to prevent map conflicts after a copy or move operation.

[SAN04-%3A+lun.newLocation.offline%3Awarning%5D: LUN /vol/sdw_cl_vol8_SAN04_A_0/FC_LUN2 has been taken offline to prevent map conflicts after a copy or move operation.

[SAN04-%3A+lun.newLocation.offline%3Awarning%5D: LUN /vol/sdw_cl_vol8_SAN04_A_0/SQL/LUN_DRIVE_X has been taken offline to prevent map conflicts after a copy or move operation.

[SAN04-%3A+lun.newLocation.offline%3Awarning%5D: LUN /vol/sdw_cl_vol8_SAN04_A_0/SQL/LUN_DRIVE_Y has been taken offline to prevent map conflicts after a copy or move operation.

[SAN04-%3A+lun.map%3Ainfo%5D: LUN /vol/sdw_cl_vol8_SAN04_A_0/FC_LUN3 was mapped to initiator group

[SAN04-%3A+lun.offline%3Awarning%5D: LUN /vol/vol8_SAN04_A/{b4b69669-e8bf-4b6f-9bfd-4b23f2b69c90}.aux has been taken offline

[SAN04-%3A+lun.destroy%3Ainfo%5D: LUN /vol/vol8_SAN04_A/{b4b69669-e8bf-4b6f-9bfd-4b23f2b69c90}.aux destroyed

[SAN04-%3A+lun.offline%3Awarning%5D: LUN /vol/vol8_SAN04_A/{75c34bf7-b92d-4da5-b343-598042c466f2}.aux has been taken offline

[SAN04-%3A+lun.destroy%3Ainfo%5D: LUN /vol/vol8_SAN04_A/{75c34bf7-b92d-4da5-b343-598042c466f2}.aux destroyed

[SAN04-%3A+lun.offline%3Awarning%5D: LUN /vol/vol8_SAN04_A/{48e4bedc-c772-427c-ba3d-5b837db6b686}.aux has been taken offline

[pascalduk]

Looks like this is from Snapmanager for exchange. If you  have a flexclone license installed, then SME will use it to create clones of the LUNs and run the verification. After the verification the flexclone volume should be destroyed automatically.

Are you sure the verification was not still active when you manually delete the clone volumes? If you have a large Exchange environment the verification can take a long time.

[thomas_glodde]

Hi,

what SnapDrive version are you using?

Regards,

Thomas

[avbohemen]

Snapshots named after a GUID like {cc962dd9-3e07-4a0e-828e-d351dfd8c1e5}, usually indicate hardware vss snapshots used for Open File Backups. Because SnapDrive is a VSS hardware provider, a backup agent tells it to create a snapshot, then backs up the data in that snapshot, then deletes the snapshot after the backup is finished. Or so it should be, I have seen various cases in which the snapshot (and thus the vol clone) remained on the storage system long after the backup was finished.

As a workaround, you could disable open file backups to see if the flexclone is still created. To make sure the flexclone is removed after the backup, it is recommended to upgrade to the latest version of SnapDrive and to patch windows with every available storage, vss, mpio and other hotfixes.

You are correct that the only way to remove these snapshots is by deleting the lun in the flexclone, then the flexclone volume, and then the snapshot. But you should check first if no backups are running for that server.

[thomas_glodde]

SnapDrive 6.3.x has a bug regarding FlexClone. They implemented features to properly snapshot flexcloned luns but they have a bug that flexcloned luns cannot be properly removed with snapdrive, thus remaining in the system. better stick with snapdrive 6.2.x.

[avbohemen]

The bug you mentioned is solved in SnapDrive 6.3P1 and 6.3P2 (P2 is cummulative to P1 and fixes a bug with lun clones). See:

http://now.netapp.com/NOW/download/software/snapdrive_win/6.3P1/

http://now.netapp.com/NOW/download/software/snapdrive_win/6.3P2/

So far, I have not seen any problems with 6.3P2.

[matthewb]

I apologize for replying to an old post, but I needed to respond to the above post.  

I think you may be thinking of bug 445705 - which is simply that Snapdrive was unable to take a snapshot of a flexclone'd volume (A volume that had already been cloned, then a snapshot of the snapshot was attempted, which is not happening in this case). 

The GUID snapshot is created as part of a VSS operation.  When a VSS snapshot request is made by a requestor (the backup utility, whether it be SME or Netbackup or other is considered the requestor), there are two ways this request can be made.   It can specify which provider to use, or it can just call for a snapshot.  When a requestor does not specify which provider to use, Microsoft will default to using any Hardware provider installed in the system - in this case, 'Data ONTAP VSS Hardware Provider'.  We commonly see this with a 3rd party VSS-aware backup utility. 

When the snapshot is taken on the filer, some versions of Snapdrive use a filer API call "snapshot-multicreate-validate" (pre-snapdrive 6.4, post Snapdrive 5.0).  This API call creates a snapshot on the selected volumes with the GUID that was assigned to the VSS session, and the "verify" part of the Snapshot effectively mounts the GUID-named snapshot back to the host to verify it completed successfully.  This is all normal processes.  Since our Hardware Provider was designed to be used with SME, some 3rd party utilities do not handle this gracefully, and a clone can be left behind randomly. 

Unless the Hardware provider needs to be used for a specific reason, the 3rd party VSS utility should be configured to select the Microsoft VSS provider.