Subscribe

repeating non-conformant dataset after datamotion + thin prov

Hi,

maybe someone else has this issue?

Two of my testlab datasets show "nonconformant" every couple of minutes and generate alarms. These datasets use a "thin/nofat san" provisioning policy (no guarantee, no space reserve).

The NMC "Conform Now" says it needs to disable the lun space reserv, and a manual conformance run solves it, dataset goes "conformant", but at next monitoring interval, the same thing happens again - nonconformant and wanting to turn the lun reservation off...

I get these errors in DFM 40D15 conformance log:

Oct 18 22:24:44 [dfmserverSmiley Very HappyEBUG]: [532:0x90c]: Conformance checker started scanning at Mon Oct 18 22:24:43 2010
Oct 18 22:24:44 [dfmserverSmiley Very HappyEBUG]: [532:0x90c]: Finished scanning dataset UCN Testlab dmotionVMDS with protection policy: UNPROTECTED, unresolvable tasks: 0, resolvable tasks: 0, resolvable-need confirm tasks: 1
Oct 18 22:41:09 [dfmserver:ERROR]: [532:0xb04]: VolLUNSpaceReserveValidate::handle_error(): Volume dmotionSAN:/dmotionSAN (115905) has failed lun-space-reservation validation.  Error Code: 141 Reason: This volume's lun-space-reservation option value does not match the policy option values Suggestion: Set lun-space-reservation option as per policy option values.
Oct 18 22:41:09 [dfmserverSmiley Very HappyEBUG]: [532:0xb04]: Conformance checker started scanning at Mon Oct 18 22:41:09 2010
Oct 18 22:41:09 [dfmserverSmiley Very HappyEBUG]: [532:0xb04]: Finished scanning dataset UCN Testlab dmotionSAN with protection policy: UNPROTECTED, unresolvable tasks: 0, resolvable tasks: 0, resolvable-need confirm tasks: 1
Oct 18 22:41:16 [dfmserver:ERROR]: [532:0xd80]: VolLUNSpaceReserveValidate::handle_error(): Volume dmotionVMDS:/dmotionVMDS (117344) has failed lun-space-reservation validation.  Error Code: 141 Reason: This volume's lun-space-reservation option value does not match the policy option values Suggestion: Set lun-space-reservation option as per policy option values.
Oct 18 22:41:16 [dfmserverSmiley Very HappyEBUG]: [532:0xd80]: Conformance checker started scanning at Mon Oct 18 22:41:15 2010
Oct 18 22:41:16 [dfmserverSmiley Very HappyEBUG]: [532:0xd80]: Finished scanning dataset UCN Testlab dmotionVMDS with protection policy: UNPROTECTED, unresolvable tasks: 0, resolvable tasks: 0, resolvable-need confirm tasks: 1
Oct 18 22:57:37 [dfmserver:ERROR]: [532:0xf0c]: VolLUNSpaceReserveValidate::handle_error(): Volume dmotionSAN:/dmotionSAN (115905) has failed lun-space-reservation validation.  Error Code: 141 Reason: This volume's lun-space-reservation option value does not match the policy option values Suggestion: Set lun-space-reservation option as per policy option values.
Oct 18 22:57:37 [dfmserverSmiley Very HappyEBUG]: [532:0xf0c]: Conformance checker started scanning at Mon Oct 18 22:57:37 2010
Oct 18 22:57:37 [dfmserverSmiley Very HappyEBUG]: [532:0xf0c]: Finished scanning dataset UCN Testlab dmotionSAN with protection policy: UNPROTECTED, unresolvable tasks: 0, resolvable tasks: 0, resolvable-need confirm tasks: 1
Oct 18 22:57:44 [dfmserver:ERROR]: [532:0xb04]: VolLUNSpaceReserveValidate::handle_error(): Volume dmotionVMDS:/dmotionVMDS (117344) has failed lun-space-reservation validation.  Error Code: 141 Reason: This volume's lun-space-reservation option value does not match the policy option values Suggestion: Set lun-space-reservation option as per policy option values.
Oct 18 22:57:45 [dfmserverSmiley Very HappyEBUG]: [532:0xb04]: Conformance checker started scanning at Mon Oct 18 22:57:44 2010
Oct 18 22:57:45 [dfmserverSmiley Very HappyEBUG]: [532:0xb04]: Finished scanning dataset UCN Testlab dmotionVMDS with protection policy: UNPROTECTED, unresolvable tasks: 0, resolvable tasks: 0, resolvable-need confirm tasks: 1
 

 

The filer LUN within this vFilers's volume is definitely NOT space reserved, and has never been:

dmotionSAN@filer9> vol status -v dmotionSAN
         Volume State           Status            Options
     dmotionSAN online          raid_dp, flex     nosnap=on, nosnapdir=off,
                                redirect          minra=off, no_atime_update=on,
                                                  nvfail=off,
                                                  ignore_inconsistent=off,
                                                  snapmirrored=off,
                                                  create_ucode=on,
                                                  convert_ucode=on,
                                                  maxdirsize=18350,
                                                  schedsnapname=ordinal,
                                                  fs_size_fixed=off,
                                                  compression=off, guarantee=none,
                                                  svo_enable=off, svo_checksum=off,
                                                  svo_allow_rman=off,
                                                  svo_reject_errors=off, no_i2p=on,
                                                  fractional_reserve=100,
                                                  extent=off,
                                                  try_first=volume_grow,
                                                  read_realloc=off,
                                                  snapshot_clone_dependency=on
                Containing aggregate: 'aggr0'

                Plex /aggr0/plex0: online, normal, active
                    RAID group /aggr0/plex0/rg0: normal

        Snapshot autodelete settings for dmotionSAN:
                                        state=on
                                                commitment=destroy
                                        trigger=volume
                                        target_free_space=5%
                                        delete_order=oldest_first
                                        defer_delete=prefix
                                        prefix=dfpm
                                        destroy_list=lun_clone,vol_clone,cifs_share
        Volume autosize settings:
                                        state=off


dmotionSAN@filer9> lun show -v
        /vol/dmotionSAN/o10g_lun0/o10g_lun0   40.0g (42953867264)   (r/w, online, mapped)
                Serial#: P3I4dZZTXv1j
                Share: none
                Space Reservation: disabled
                Multiprotocol Type: linux
                Maps: dfpm_dmotionSAN=0

It is just like it was created by ProvMgr.

The CLI on DFM server shows:

C:\>dfpm dataset conform -D 115778
Dataset dry run results
----------------------------------
Do:  Disable  space reservation for LUN dmotionSAN:/dmotionSAN/o10g_lun0/o10g_lun0
Effect:  Disable space reservation of the LUN.

I also found:

C:\> dfpm dataset get 115778

Allow custom volume settings on provisioned volumes:                                                       No
Enable periodic write guarantee checks on SAN datasets:                                                       No

But however I try to set this option,  it does not do anything:

C:\>dfpm dataset set 115778 isEnableWGChecks=Never
Left settings unchanged for 115778.

C:\>dfpm dataset set 115778 isEnableWGChecks=No
Left settings unchanged for 115778.

C:\>dfpm dataset set 115778 isEnableWGChecks=Yes
Left settings unchanged for 115778.

C:\>dfpm dataset set 115778 isEnableWGChecks=On
Left settings unchanged for 115778.

These 2 datasets toggle to nonconformant all the time since I performed datamotion on them in the past (successfully and cleand up).

They have been fine before the first datamotion (nondisruptive migration) operation AFAICT.

Ideas, anyone?

Re: repeating non-conformant dataset after datamotion + thin prov

Can you get the output of the following command for the provisioning policy you are using?

dfpm policy get <provisioning policy name or id>

Also what was the ONTAP version when you didnt have problem ?

I am asuming you are currently running 7.3.3

Regards

adai

Re: repeating non-conformant dataset after datamotion + thin prov

Hi adai,

C:\>dfpm policy get 115705
Name:                                     LAB SAN/NoFat non-SnapDrive
Description:                              Testlab SAN LUNs - no SnapDrive available, no dedupe
Type:                                     san
Disk Failure Reliability:                 any
Sub System Failure Reliability:           Disabled
Controller Failure Reliability:           Disabled
Resource Label:
Dataset Member Deduplication Option:      Disabled
Dataset Member Deduplication Schedule:    auto
Thin Provision Storage Space:             Enabled
Guarantee Space For LUN Writes:           Disabled
Thin Provisioning Configuration:          none
Storage Container To Provision:           lun
Dataset Member Nearly Full Threshold (%): 85
Dataset Member Full Threshold (%):        91
Custom Provisioning Script Path:

I saw it in the past with DOT 7.3.3Px already, the actual problem now is with 7.3.4P2, this does not seem to make a difference. These two datasets/vfilers have been created under 7.3.4P1, DFM 4.0D15.

When i change the policy to "more-thick" provisiong (space reservation on, guarantee on, frac_reserve 0, autosize on), the error goes away permanently after one conformance run.

When I switch back to "do not guarantee" (non-space-reserved, non-guaranteed)  the error hits me again and again...

This does not happen when dataset/vfiler dataset has never been datamotion'ed before. At least this is what I observe. I can't find any obvious cause for this... :-/

Re: repeating non-conformant dataset after datamotion + thin prov

@m.schuren

I'm trying your scenario but I'm not seeing it. My dfm server is  4.1 but I don't think there has been any change in conformance handling from 4.0 to 4.1. My filers are 7.3.5X1.

I have a san policy:

C:\Documents and Settings\Administrator>dfpm policy get san1
Name:                                     San1
Description:
Type:                                     san
Disk Failure Reliability:                 any
Sub System Failure Reliability:           Disabled
Controller Failure Reliability:           Disabled
Resource Label:
Dataset Member Deduplication Option:      Disabled
Dataset Member Deduplication Schedule:    auto
Thin Provision Storage Space:             Enabled
Guarantee Space For LUN Writes:           Disabled
Thin Provisioning Configuration:          none
Storage Container To Provision:           lun
Dataset Member Nearly Full Threshold (%): 80
Dataset Member Full Threshold (%):        90
Custom Provisioning Script Path:

C:\Documents and Settings\Administrator>

can you give the out put of the following command one after the other.

1. dfpm dataset conform <your_dataset_name>

2. dfm dataset conform -D <your_dataset_name>

I'll try with DFM 4.0D145 tomorrow.

warm regards,

Abhishek

If this post resolved your issue, help others by selecting ACCEPT AS SOLUTION or adding a KUDO.

Re: repeating non-conformant dataset after datamotion + thin prov

I think I know this issue, there has been a change in the default space reservation of ONTAP from 7.3.3 or 7.3.4 or latter.

Will investigate more and let you know.

Regards

adia

Re: repeating non-conformant dataset after datamotion + thin prov

Hi sinhaa,

C:\>dfpm dataset conform 117293
Started conformance check for dataset 117293.
Check the status of the dataset with Management Console
or by viewing the conformance.log file.

C:\>dfpm dataset conform -D 117293
Dataset dry run results
----------------------------------
Do:  Disable  space reservation for LUN dmotionVMDS:/dmotionVMDS/dmotionVMDS01/dmotionVMDS01
Effect:  Disable space reservation of the LUN.


C:\>dfpm dataset conform -a 117293
Started conformance check for dataset 117293.
Check the status of the dataset with Management Console
or by viewing the conformance.log file.

C:\>dfpm dataset conform -D 117293
Dataset dry run results
----------------------------------
No dry run results available.

...wait 10 minutes or so...

C:\>dfpm dataset conform -D 117293
Dataset dry run results
----------------------------------
Do:  Disable  space reservation for LUN dmotionVMDS:/dmotionVMDS/dmotionVMDS01/dmotionVMDS01
Effect:  Disable space reservation of the LUN.

Re: repeating non-conformant dataset after datamotion + thin prov

Hi adai,

you are probably right. I never tested this provisioning policy with DOT < 7.3.3 , I think.

What can I do to solve it (besides switching to thick-provisioned luns)?

Re: repeating non-conformant dataset after datamotion + thin prov

Hello m.schuren,

      We tried to investigate your issue and its seems that this is a bug. Dataset doesn't readilty become non-conformant, even after a forceful conformance run, but after some time it does.

Nov 03 14:47:25 [dfmserver:ERROR]: [17884:0xf19dbba0]: VolLUNSpaceReserveValidate::handle_error(): Volume November_2010_1:/November_2010_2 (7490) has failed lun-space-reservation validation.  Error Code: 141 Reason: This volume's lun-space-reservation option value does not match the policy option values Suggestion: Set lun-space-reservation option as per policy option values.

I'll raise a burt for the same and fwd you the burt id.

warm regards,

Abhishek

If this post resolved your issue, help others by selecting ACCEPT AS SOLUTION or adding a KUDO.

Re: repeating non-conformant dataset after datamotion + thin prov

Th burt is the following:

460374

thanks.

If this post resolved your issue, help others by selecting ACCEPT AS SOLUTION or adding a KUDO.

Re: repeating non-conformant dataset after datamotion + thin prov

Hi sinhaa,

thanks for taking the efforts.

I will track this burt ID.

Were you able to reproduce the issue?