Active IQ Unified Manager Discussions

Snap Mirror Compression and OnCommand 5

rdenyer001
4,841 Views

Does anyone know if its possible to enable Snap Mirror Compression  with  OnCommand 5 ?

I have found a document TR-3790  "SnapMirror Network Compression"   which explains how to use compression with DFM 3.8 but it relies  on  the /etc/snapmirror.conf  file  and the compression=enable  option.

Since I have upgraded to OnComand 5.0  my  /etc/snapmirror.conf files seem to have been purged of any  snapmirror relationships that are setup and managed via OnCommand.

8 REPLIES 8

smoot
4,841 Views

I'm not aware of any changes in OC5 that would cause it to purge snapmirror.conf, but perhaps something changed that I'm not aware of.

If you enable SnapMirror compression by editing snapmirror.conf, those changes can be erased if you use the hostPreferredAddr[12] options. This is a known problem and should have been mentioned in the TR.

adaikkap
4,841 Views

Hi Richard,

Can you get the output of the following command for your source and destination filers of snapmirror ?

dfm host get -q <filerid/name>

Regards

adai

rdenyer001
4,841 Views

Here  is the output from dfm host get -q <filerid/name>

Destination

C:\Windows\system32>dfm host get -q HOST1
host=HOST1.dc.local
hostLogin=XXXX
hostPassword=********
hostLoginProtocol=ssh
hostPreferredAddr1=
hostPreferredAddr2=
hostPrimaryAddress=XX.X.X.XX
hostRLMAddress=
hostConsoleAddress=
cpuTooBusyThreshold=95
cpuBusyThresholdInterval=00:15:00
hostPingMethod=all
hostAdminTransport=http
hostNdmpLogin=XXXX
hostNdmpPassword=********
hostAdminPort=80
perfAdvisorTransport=httpOnly
prefsnmpVersion=1
useHostsEquiv=No
maxActiveDataTransfers=160
perfDataExportEnabled=No
defaultvFilerInterface=
processHostIP=
autoClientStatEnabled=No
hostClockSkewedThreshold=1 minute
hostClockNearlySkewedThreshold=30 seconds

Source
C:\Windows\system32>dfm host get -q HOST2
host=HOST2.dc.local
hostLogin=XXXX
hostPassword=********
hostLoginProtocol=ssh
hostPreferredAddr1=
hostPreferredAddr2=
hostPrimaryAddress=XX.X.X.XX
hostRLMAddress=XX.X.X.XX
hostConsoleAddress=
cpuTooBusyThreshold=95
cpuBusyThresholdInterval=00:15:00
hostPingMethod=all
hostAdminTransport=http
hostNdmpLogin=XXXX
hostNdmpPassword=********
hostAdminPort=80
perfAdvisorTransport=httpOnly
prefsnmpVersion=1
useHostsEquiv=No
maxActiveDataTransfers=160
perfDataExportEnabled=No
defaultvFilerInterface=
processHostIP=
autoClientStatEnabled=No
hostClockSkewedThreshold=1 minute
hostClockNearlySkewedThreshold=30 seconds

adaikkap
4,841 Views

Hi Richard,

             I created a compression enabled relationship on the filer and imported the same into a dataset in DFM 4.0.2D1.

C:\>dfm version

dfbm.exe         4.0.2.7809 (4.0.2D1)

dfdrm.exe        4.0.2.7809 (4.0.2D1)

dfpm.exe         4.0.2.7809 (4.0.2D1)

dfm.exe          4.0.2.7809 (4.0.2D1)

dfmcheck.exe     4.0.2.7809 (4.0.2D1)

dfmconfig.exe    4.0.2.7809 (4.0.2D1)

dfmconsole.exe   4.0.2.7809 (4.0.2D1)

dfmmonitor.exe   4.0.2.7809 (4.0.2D1)

dfmperf.exe      4.0.2.7809 (4.0.2D1)

dfmscheduler.exe 4.0.2.7809 (4.0.2D1)

dfmserver.exe    4.0.2.7809 (4.0.2D1)

dfmwatchdog.exe  4.0.2.7809 (4.0.2D1)

eventd.exe       4.0.2.7809 (4.0.2D1)

grapher.exe      4.0.2.7809 (4.0.2D1)

C:\>dfm controller list

ID   Type                     Host Name                           Host Address                              ProductId    Deleted

---- ------------------------ ----------------------------------- ----------------------------------------- ------------ --------

  90 Active/Active Controller fas-sim-1                           192.168.98.10                             0099931871   No

  91 Active/Active Controller fas-sim-2                           192.168.98.11                             0099931872   No

C:\>dfm host get -q 90

host=fas-sim-1

hostLogin=

hostPassword=********

hostLoginProtocol=rsh

hostPreferredAddr1=

hostPreferredAddr2=

hostPrimaryAddress=192.168.98.10

hostRLMAddress=

hostConsoleAddress=

cpuTooBusyThreshold=95

cpuBusyThresholdInterval=00:15:00

hostPingMethod=echo_snmp

hostAdminTransport=http

hostNdmpLogin=

hostNdmpPassword=********

hostAdminPort=80

perfAdvisorTransport=httpOnly

prefsnmpVersion=1

useHostsEquiv=No

maxActiveDataTransfers=

perfDataExportEnabled=No

defaultvFilerInterface=

processHostIP=

autoClientStatEnabled=No

C:\>dfm host get -q 91

host=fas-sim-2

hostLogin=

hostPassword=********

hostLoginProtocol=rsh

hostPreferredAddr1=

hostPreferredAddr2=

hostPrimaryAddress=192.168.98.11

hostRLMAddress=

hostConsoleAddress=

cpuTooBusyThreshold=95

cpuBusyThresholdInterval=00:15:00

hostPingMethod=echo_snmp

hostAdminTransport=http

hostNdmpLogin=

hostNdmpPassword=********

hostAdminPort=80

perfAdvisorTransport=httpOnly

prefsnmpVersion=1

useHostsEquiv=No

maxActiveDataTransfers=

perfDataExportEnabled=No

defaultvFilerInterface=

processHostIP=

autoClientStatEnabled=No

C:\>

Ran Multiple updates and found the conf entry to be intact.

Now upgraded my dfm server to 5.0 and ran multiple update and still find my conf entry to be intact in snapmirror.conf.

C:\>dfpm dataset list -x compression

Id:                              392

Name:                            compression

Protection Policy:               Mirror

Application Policy:

Description:

Owner:

Contact:

Volume Qtree Name Prefix:

Snapshot Name Format:

Primary Volume Name Format:

Secondary Volume Name Format:

Secondary Qtree Name Format:

DR Capable:                      No

Requires Non Disruptive Restore: No

Node details:

   Node Name:           Primary data

   Resource Pools:

   Provisioning Policy:

   Time Zone:

   DR Capable:          No

   vFiler:

   Node Name:           Mirror

   Resource Pools:

   Provisioning Policy:

   Time Zone:

   DR Capable:          No

   vFiler:

C:\>dfpm dataset list -m compression

Id         Node Name            Dataset Id Dataset Name         Member Type                                        Name

---------- -------------------- ---------- -------------------- -------------------------------------------------- -------------------------------------------------------

       383 Primary data                392 compression          volume                                             fas-sim-1:/comp_vsm_src

       386 Mirror                      392 compression          volume                                             fas-sim-2:/comp_vsm_dst

C:\>dfpm dataset list -R compression

Id         Name                        Protection Policy           Provisioning Policy Relationship Id State        Status  Hours Source                       Destination

---------- --------------------------- --------------------------- ------------------- --------------- ------------ ------- ----- ---------------------------- ----------------------------

       392 compression                 Mirror                                                      389 snapmirrored idle    0.0   fas-sim-1:/comp_vsm_src      fas-sim-2:/comp_vsm_dst

C:\>dfpm backup list compression

Backup Id Backup Version        Retention Type Retention Duration (in seconds)   Node Name            Description                         Properties(Name=Value)

--------- --------------------- -------------- --------------------------------- -------------------- ----------------------------------- -------------------------

       12 22 Dec 2011 00:42:08  unlimited                                        Mirror

       10 22 Dec 2011 00:42:08  unlimited                                        Primary data

       11 22 Dec 2011 00:41:54  monthly                                          Mirror

        9 22 Dec 2011 00:41:54  monthly                                          Primary data

        8 22 Dec 2011 00:19:35  unlimited                                        Mirror

        7 22 Dec 2011 00:19:21  weekly         3024000                           Mirror

        5 22 Dec 2011 00:19:21  weekly         3024000                           Primary data

        3 22 Dec 2011 00:16:51  daily          604800                            Mirror               test

        1 22 Dec 2011 00:16:51  daily          604800                            Primary data         test

C:\>dfm run cmd fas-sim-2 " rdfile /etc/snapmirror.conf"

Job: 1

Host: fas-sim-2

Command:  rdfile /etc/snapmirror.conf

Status: done

Submitted: Thu Dec 22 00:45:28 2011

Attempts: 1

Login Protocol: rsh

Use RLM Port: No

Started: Thu Dec 22 00:45:28 2011

Completed: Thu Dec 22 00:45:29 2011

Stdout: #Regenerated by registry Tue Sep 27 08:58:29 GMT 2011

Stdout: fas-sim-1:/vol/qsm_src/qsm_qt1 fas-sim-2:/vol/qsm_dst/qsm_qt1 restart=never 0-59/3 * * *

Stdout: fas-sim-1:/vol/qsm_src/qsm_qt2 fas-sim-2:/vol/qsm_dst/qsm_qt2 restart=never 0-59/4 * * *

Stdout: fas-sim-1:/vol/qsm_src/- fas-sim-2:/vol/qsm_dst/qsm_dst_non_qtree_data - 0-59/3 * * *

Stdout: fas-sim-1:vsm_src fas-sim-2:vsm_dst restart=never * 0-23 * *

Stdout: comp_con=multi(192.168.98.12,192.168.98.13)

Stdout: comp_con:comp_vsm_src fas-sim-2:comp_vsm_dst compression=enable - - - - -

C:\>

C:\>dfm version

dfbm.exe         5.0.0.7636 (5.0)

dfdrm.exe        5.0.0.7636 (5.0)

dfpm.exe         5.0.0.7636 (5.0)

dfm.exe          5.0.0.7636 (5.0)

dfmcheck.exe     5.0.0.7636 (5.0)

dfmconfig.exe    5.0.0.7636 (5.0)

dfmconsole.exe   5.0.0.7636 (5.0)

dfmmonitor.exe   5.0.0.7636 (5.0)

dfmperf.exe      5.0.0.7636 (5.0)

dfmscheduler.exe 5.0.0.7636 (5.0)

dfmserver.exe    5.0.0.7636 (5.0)

dfmwatchdog.exe  5.0.0.7636 (5.0)

eventd.exe       5.0.0.7636 (5.0)

grapher.exe      5.0.0.7636 (5.0)

C:\>

Can you provide the steps to reproduce  ?

Regards

adai

fas-sim-2> rdfile /etc/snapmirror.conf

#Regenerated by registry Tue Sep 27 08:58:29 GMT 2011

fas-sim-1:/vol/qsm_src/qsm_qt1 fas-sim-2:/vol/qsm_dst/qsm_qt1 restart=never 0-59/3 * * *

fas-sim-1:/vol/qsm_src/qsm_qt2 fas-sim-2:/vol/qsm_dst/qsm_qt2 restart=never 0-59/4 * * *

fas-sim-1:/vol/qsm_src/- fas-sim-2:/vol/qsm_dst/qsm_dst_non_qtree_data - 0-59/3 * * *

fas-sim-1:vsm_src fas-sim-2:vsm_dst restart=never * 0-23 * *

comp_con=multi(192.168.98.12,192.168.98.13)

comp_con:comp_vsm_src fas-sim-2:comp_vsm_dst compression=enable - - - - -

fas-sim-2>

rdenyer001
4,841 Views

Thanks Adai,

Most of my Snapmirror relationships have been created directly from the "NetApp Management Console"  and dont have  entries in the snapmirror.conf file.

So I guess there are two options 1. wait until the NetApp management tools catch-up with ALL the features availible on a NetApp system or  2. Edit my snapmirror.conf  files  to add the compression option

Regards,

Richard

adaikkap
4,841 Views

Hi Richard,

     I think there is some misunderstanding. Let me try to clarify some.

  1. PM does not support creation of compression enabled snapmirror relationships.But it interoperates
  2. PM does not make a snapmirror.conf entry for all relationship its creates, except for those that are created using preferred interface and uses connections.

Case 1:Where PM creates the relationships:

================================

So if you create a normal QSM or VSM relationship using PM it neither makes a  snapmirror.conf entry nor schedules in the same as below.

fas-sim-1:/vol/qsm_src/- fas-sim-2:/vol/qsm_dst/qsm_dst_non_qtree_data - 0-59/3 * * *

Instead the schedule are driven by DFM server scheduler service which does the scheduling as per the policy you use in PM.

If you have the option hostPreferredAddr1/2 set in DFM for the source and destination filer of Snapmirror.And create a VSM or QSM relationship using PM then PM creates a connection and make these entries in snapmirror.conf

fas-sim-1_fas-sim-2_1208568606=multi(192.168.98.12,192.168.98.13)

fas-sim-1_fas-sim-2_1208568606:comp_vsm_src fas-sim-2:comp_vsm_dst compression=enable - - - - -

(Note if 1 is set on primary then 1 in secondary is used. Its the respective interface and not 1 in source and 2 in destination is used)

Even in this case the schedule are driven by DFM server scheduler service and not ontap.This applies to both QSM and VSM.

So PM is not capable of creating a compression enabled QSM or VSM relationship.

The same applies to SV also where we don't create the snapvault snap schedules. In order to have the SV traffic in a preferred interface the following options in filer should be set.

ndpmd_preferred.interface.

Case2:Where relationships are imported.

First and foremost rule:

PM does not delete any snapmirror.conf or snapvault schedule when the corresponding relationships are imported into a Dataset.This enables PM to interoperate with Compression enabled

snapmirror.

So to have a compression enabled relationship in PM have the relationship created in the filer and the corresponding conf entries.Import them into a data and have the schedule in PM

fas-sim-1_fas-sim-2_1208568606=multi(192.168.98.12,192.168.98.13)

fas-sim-1_fas-sim-2_1208568606:comp_vsm_src fas-sim-2:comp_vsm_dst compression=enable - - - - -

It is advised to remove the schedule in the snapmirror.conf to remove the redundant scheduling. Make sure that hostPreferredAddr1/2 are not set on the source and destination filer of the compression enabled snapmirror filers in DFM.

When a relationship is imported say like below which has snapmirror.conf entry as follows

fas-sim-1:/vol/qsm_src/- fas-sim-2:/vol/qsm_dst/qsm_dst_non_qtree_data - 0-59/3 * * *

and hostPreferredAddr1/2 are correspondingly set on the source and destination filer then PM makes an entry in the snapmirror.conf as follows

fas-sim-1_fas-sim-2_1208568606=multi(192.168.187.12,192.168.187.13)

fas-sim-1_fas-sim-2_1208568606:/vol/qsm_src/- fas-sim-2:/vol/qsm_dst/qsm_dst_non_qtree_data - - - - -

Still the old entry remains.

The reason why hostPreferredAddr1/2 should not be set for compression enabled snapmirror is because before every transfer we rewrite the snapmirror.conf with the above two entry in this process compression options is deleted as there is no support for it in NDMP or api.

Hope this clarifies.

Regards

adai

localizedinsanity
4,841 Views

I had originally asked....how would you go about retrofitting existing PM created relationships to enable compression?  I subsequently worked that out....here are the steps...

* add dfm host set <hostname> hostPreferredAddr1=x.x.x. to each source node's configuration within DFM

* manually edit the /etc/snapmirror.conf file to populate it using the output of "snapmirror status" for source/destination pairings

     * remember to add the "compression=enable" in the options field and to keep the schedule - - - -

     * remember to configure your connection string settings

     * I use a text editor to fully create the file structure, then paste it into the CLI using "wrfile /etc/snapmirror.conf" & ctrl-c

Once you do those steps, Protection Manager will use compression the next time it updates.  This process assumes you have pre-existing datasets with policies defined and mirrors baselined/updating.  If you need to configure new relationships, Protection Manager will populate the snapmirror.conf file (thanks to the hostPreferredAddr1 setting) and all you need to do is edit the configuration file to change the source node name to the connection string and add the compression option. 

NOTE - an initialize process will NOT be compressed because Protection Manager cannot enable compression as pointed out in previous comments.  Unless you create the relationship using System Manager without initializing, make the changes to snapmirror.conf, manually initialize, then import as outlined in Adai's post, you cannot initialize using compression.  I hope this helps the next person to ask the question!

Thanks to Adai for getting me on the right track with his posting about preferred addresses.

adaikkap
4,841 Views

Hi,

      All you have to do is the following.

For all your Volume SnapMirror relationships pls make an entry in your snapmirror.conf file as follows.

comp_con=multi(192.168.98.12,192.168.98.13)

comp_con:comp_vsm_src fas-sim-2:comp_vsm_dst compression=enable - - - - -

Import those relationships to PM Mirror dataset.

Make sure you dont have the option hostPerferredAddr1 set on your filer otherwise on first update from PM the compression options will be erased from snapmirror.conf.

Read this public report for the same which affects compression option

http://support.netapp.com/NOW/cgi-bin/bol?Type=Detail&Display=381872

Regards

adai

Public