All Data Backup and Recovery Discussions posts

Snapcenter Agent Comms when running on VM guests

bobalon — Thu, 24 Jun 2021 15:25:27 GMT

Hopefully a simple questions for the SC Gurus out there

We have SC and the SC Vsphere plugin (both v4.4) and plan to start backing up MS SQL dBs on the Vm guests using SC. normally when we do this for physical SQL servers the agents on the SQL server need to be able to communicate direct to the relevant Storage SVM to orchestrate the snapshots/snapmirrors etc.

As I understand it when the Server is a VM the snapshot is orchestrated via the Vsphere plug-in host and not the SC agent on the host so is the comms path from the Agent to the SVM hosting the VMDK files/VMFS volumes still required?

I assume the storage discovery is undertaken via the Vssphere plug-in and not direct from the Agents on the hosts?

Any confirmation or pointers to documentation that explains the steps SC uses for VM guest backup would be good and save a lot of trial and error/additional work

thank you

Re: SnapDiff and Snaplock Enterprise

Ontapforrum — Thu, 24 Jun 2021 10:04:58 GMT

Looks like there is no documentation around the two. Yes, under the hood both are flexvol and snapdiff by default supports flexvol. However, the question is - Does Snaplock (WORM Filesystem attribute) allows for changes to the volume/snapshots via 3rd party Backup application using exposed snapdiff API? This is b'cos the way SnapDIff differencing engine works depends upon how it is integrated with 3rd party Backup software, what APIs are exposed, but it general it does the standard stuff such as:

Backup Software using SnapDIff API to:
1) Create a base-snap - 1st full backup
2) Then create a diff-snap - Differential/Incremental
3) Next run- delete the 1st base-snap and use the last diff-snap as 'base' and create a new diff-snaps etc. At any time it will keep only most recent 2 snapshot set.

Even though you can tell certain backup software to use 'already present' snapshots for differencing purpose on read-only file-system but this is something CommVault support will be able to answer. Is the SVM under which the Snaplock vol exits is enabled for snapdiff ? I suggest try to configure/backup the volume and see how far it goes, it will give some clues.

SnapDiff and Snaplock Enterprise

luizdatri — Tue, 22 Jun 2021 13:59:18 GMT

Is it possible to use SnapDiff with SnapLock Enterprise? Under the hood, both are FlexVol based, but there is no documentation that links both features. The customer uses Commvault as backup solution.

Re: SnapCenter 4.4 - Issue with Cloning, Custom Plug-in for MySQL Server

stanton — Thu, 17 Jun 2021 11:10:37 GMT

This looks like a deep enough problem that a support case should be opened as it is going to require NetApp to look deeper into the data than can safely be gathered in a forum setting. Please open a case with NetApp support.

Re: SnapCenter 4.4 - Issue with Cloning, Custom Plug-in for MySQL Server

RobinR1 — Thu, 17 Jun 2021 10:40:23 GMT

What UDEV rules should be set up? We don't have any custom udev rules in place. And during the clone process we see the filesystems get successfully discovered and (with devicemapper and multipath) mapped to /dev/dm-0, /dev/dm-1, etc..

The filesystems on those devices do have a uuid that can be viewed with the blkid command, if that is the uuid you mean?

The devices are also found in /dev/mapper/<id> where <id> seems to be some unique id, different from the blkid

After discovery takes place and those lun's are mapped to multipath devices, the cloning process fails with the above-mentioned error and the lun's are removed again.. (without actually removing the DM devices on the OS first so multipath/dm start complaining about all the paths)

Re: SnapCenter Powershell Command iSCSI

mrahul — Tue, 15 Jun 2021 13:52:07 GMT

For establishing and terminating iscsi sessions you can use SnapCenter cmdlets "Connect-SdIscsiTarget" and "Disconnect-SdIscsiTarget" respectively.

'New-SdStorage' Provisions dedicated disks, shared disks and clustered shared volumes on a LUN

'Connect-SdStorage' cmd connects a dedicated disk, shared disk, or a clustered shared volume based on a LUN.

Re: SnapCenter Powershell Command iSCSI

NEPLUSULTRA — Tue, 15 Jun 2021 12:24:08 GMT

i had a look at this one https://library.netapp.com/ecm/ecm_download_file/ECMP1189424 but i don't see powershell commands to establish an iscsi session.

Re: SnapCenter Powershell Command iSCSI

mrahul — Tue, 15 Jun 2021 11:47:09 GMT

SnapCenter leverages the same commandlets via the snapdrive PS modules which gets installed as part 'SnapCenter Plug-in for Windows' Installation. Access your windows host and you can use the cmdlets.

SnapCenter Powershell Command iSCSI

NEPLUSULTRA — Tue, 15 Jun 2021 10:42:44 GMT

Are there powershell command that i can run on a host to

1. establish an iSCSI session

2. connect the lun

In snapdrive i can run the sdcli command. But i can find the same commands in snapcenter

https://library.netapp.com/ecm/ecm_download_file/ECMLP2874310

Re: Snapcenter Plugin 4.3.1 Backups are fine. Restore errors with "No backups available at this

hmoubara — Tue, 15 Jun 2021 08:33:16 GMT

Hello

While following the kb mentioned, did you find multiple entry for the webclient under the below directory:

Virtual Appliance:

/etc/vmware/vsphere-ui/vc-packages/vsphere-client-serenity/
/etc/vmware/vsphere-client/vc-packages/vsphere-client-serenity/
Windows:

C:\ProgramData\VMware\vCenterServer\cfg\vsphere-ui\vc-packages\vsphere-client-serenity\
C:\ProgramData\VMware\vCenterServer\cfg\vsphere-client\vc-packages\vsphere-client-serenity\

if you do remove both and re-attempt the registry, make sure to refollow the steps before and in between.

Thanks

Re: SnapCenter SnapVault snapshot retention

hmoubara — Tue, 15 Jun 2021 05:14:40 GMT

Hello,

The snapshot created prior to the retention settings set for the snapvault update will not be affected. You will have to manually remove those snapshot manually, and eventually the retention set will take affect on the newly created snapshot based on your policy set.

Thanks

SnapCenter SnapVault snapshot retention

chinchillaking — Mon, 14 Jun 2021 03:39:25 GMT

Hello,

setting as below, but SnapVault did not auto delete old snapshot refer retention count

- primary storage vol1 with LUN did not setup snapshot schedule copy

- SnapCenter create Policy hourly and "Total Snapshot copies to keep" 3, Secondary policy label Select Customer Label "hourly", select "Update SnapVault after creating a local Snapshot copy"
- SnapCenter create Windows host Resource Group, select "Use custom name format for Snapshot copy" $Policy
- secondary storage create Protection policy name svtest with Asyn Vault, SnapMirror label "hourly" and Destination Retention Count 3
- seconday storage create volume relationship, Relationship Type Vault, Vault Policy select svtest, Schedule None
- SnapCenter primary snapshot backup and snapshot transfer to secondary successful

but secondary storage did not keep retention to 3, it did not auto delete old backup snapshot, is normal? How can setup auto delete the old backup snapshot?

Best regards,

Chung

Re: Snapcenter Plugin for vmware vsphere backup jobs failing

PatrickR — Sun, 13 Jun 2021 03:55:24 GMT

Thanks!

Re: Invitation to participate in SnapCenter 4.5 Beta/Preview/Early Access Program

darraghos — Fri, 11 Jun 2021 13:58:38 GMT

Is this still available? The forms seems to be down.

Snapcentre and Oracle Standby Databases

darraghos — Fri, 11 Jun 2021 13:56:49 GMT

We are trying to create a backup of an Oracle Dataguard Standby database with Snapcentre 4.4. When we create the offline schedule with 'Shutdown' mode when we run it it throws this message:

Activity 'Discovering Oracle Database Files' failed with error: Discovery failed for object MYDB with error PL-SCO-20006: Discovery of database files failed for database MYDB on host myhost with error PL-SCO-20035: OFFLINESHUTDOWN DATA backup is not supported for Data Guard standby database.. Activity 'Finalizing Oracle Database Backup' failed with error: PostBackupActivity failedPL-SCO-20032: postBackup operation failed with error: PL-SCO-20035: OFFLINESHUTDOWN DATA backup is not supported for Data Guard standby database.

The guide for Snapcentre 4.4 states:

Oracle Data Guard standby

You can create only offline (mount or shutdown) backups of Data Guard standby databases. Archive log-only backup or full backup is not supported.

So why would this not work? I did see this KB and tried the steps but same error.

Re: SnapCenter Role With Minimum Privileges

TMADOCTHOMAS — Thu, 10 Jun 2021 15:35:43 GMT

A colleague of mine found an article describing how to lock down the account at the SVM level. Here is it in case anyone searches this topic:

https://docs.netapp.com/ocsc-42/index.jsp?topic=%2Fcom.netapp.doc.ocsc-isg%2FGUID-D39A6ACE-D69D-4470-ABE4-A63A7803E880.html

Re: Snapcenter Plugin for vmware vsphere backup jobs failing

Sig — Thu, 10 Jun 2021 00:18:14 GMT

You won't need to configure a snapshot policy via ONTAP, SCVI will do that for you.

You will need to configure a snapmirror relationship via ONTAP.

You will need to configure a snapmirror policy via ONTAP, SCVI will transfer as per your SnapMirror policy.

Snapcenter Plugin for vmware vsphere backup jobs failing

PatrickR — Wed, 09 Jun 2021 23:47:33 GMT

I have a question about the configuration of snapcenter plugin for vmware vsphere; If I configure the policy options 'Update snapvault after backup' and the 'snapmirror label' Am I still required to configure the snapshot policy and snapmirror policy via Ontap?

Re: Snapdrive - SDCLI and HA Config

darraghos — Wed, 09 Jun 2021 10:17:58 GMT

So if running the SDCLI command on a server that has say the Oracle plugin how does it talk back to the load balcner VIP for HA?

Re: Can you join 2 SnapCenter's into 1

sunnyvasu — Wed, 09 Jun 2021 09:34:19 GMT

Thanks NTAp-AVS for your reply.

I'll ask NetApp support for more help.

To clarify, yes we do have 2 SnapCenters.

As per your details..

SCV Appliance --- Connected to --- SnapCenter Server A

NSM - SQL hosts -- connected to --- SnapCenter Server B

Server A backs up VM's and Server B backs up SQL databases on physical servers.

We want to add Server A (SCV Appliance) to Server B (SQL hosts) and be able to see all jobs from Server B (One SnapCenter).

SCV Appliance has a different website to NSM e.g. https://<FQDN>:8080 for SCV and https://<FQDN>:8146 for NSM.

So it looks like we should just log into Server B ( https://<FQDN>:8146), click hosts, Manage Hosts and then add vCenter, which will then see the SCV Appliance metadata.

Re: SnapCenter 4.4 vSphere Plug-in. SnapMirror update doesn't initiate when a VM wont quiesce.

dankirkwood — Tue, 08 Jun 2021 06:14:18 GMT

BURT https://mysupport.netapp.com/NOW/cgi-bin/bol?Type=Detail&Display=1363263

No, you can't search for it. And it's not mentioned in the readme or "known issues" for SVC 4.4.

Pretty awful that NetApp don't disclose the bug, haven't patched it, and leave it to unsuspecting customers to run into it and waste their time on it. We upgraded to 4.4 recently and now we have a half-working backup solution.

Apparently it'll be fixed in 4.5, whenever that comes out!

Re: Can you join 2 SnapCenter's into 1

NTAp-AVS — Mon, 07 Jun 2021 13:33:31 GMT

Hello

The question posted is little complicated to answer as we do not know your complete environment.

All the VM backups are contained in the NSM database of the SCV appliance. All Virtual machine backup schedules are on SCV appliance.

All the SQL backups are contained in the NSM database of Snapcenter Server.

We assume that you have following setup

SCV Appliance --- Connected to --- SnapCenter Server A

SQL hosts -- connected to --- SnapCenter Server B

Now you would like to delete the SCV appliance from Snapcenter Server A and add it to Snapcenter Server B

Is that accurate?

If yes , you can do that. The reason being SCV appliance can exist by itself (just for VM backups) without SnapCenter server as the VM backups are no longer stored on Snapcenter server NSM database

(NSM -> Backend Mysql Database)

On the other hand if you have SQL databases on VMS and if you have SQL databases on RDM luns or VMDKS , the SCV appliance should be added to Snapcenter Server . All the Database related backups are stored in the NSM database of Snapcenter server

We recommend calling your account team or NetApp support hotline and provide us with more details to address your question

Thanks

Can you join 2 SnapCenter's into 1

sunnyvasu — Mon, 07 Jun 2021 09:11:49 GMT

We have 2 SnapCenters. 1 was an older version of Snap Manager/VSC, migrated to data broker appliance and then upgraded to 4.4 SnapCenter appliance. This has all our VMware vCenter metadata.

We've built a Physical Windows SnapCenter, which will be upgraded from 4.3 to 4.4 with physical SQL server database backup jobs.

We want to merge two together e.g. move the 4.4 SnapCenter appliance onto the Physical Windows SnapCenter.

We want to be able to manage jobs from 1 SnapCenter. We have created SQL database backup jobs for physical SQL servers, we want to use with VM's with SQL databases.

Is this possible?

Change PreferredInterfaceRole using Netapp Powershell toolkit

ECOIFFE — Fri, 04 Jun 2021 09:49:44 GMT

How can i modify the PreferredInterfaceRole parameter using Netapp Powershell toolkit ?

I tried using Set-NcNdmpVserverAttr with no luck, even using the -template parameter.

PS C:\Users\xxxx> get-module | Select-Object -property Name,Version

Name Version
---- -------
DataONTAP 9.8.0
Microsoft.PowerShell.Management 3.1.0.0
Microsoft.PowerShell.Utility 3.1.0.0
PSReadline 2.0.0

Re: SnapCenter Role With Minimum Privileges

TMADOCTHOMAS — Thu, 03 Jun 2021 15:11:53 GMT

Thanks for responding to my post. We've had SnapCenter running for over a year. When I set it up, my understanding was we needed a local account on any SVM that SnapCenter connected to with the admin role. Now, as a separate project, we are trying to beef up security and give all accounts the least amount of rights it needs.

I did see information on a recent SnapCenter update that indicated you can now simply create a connection to the cluster rather than an SVM.

So here are some questions:

Is there no way to lock down a local SVM account in the same way as described by the articles for a local cluster account? (Or, at the very least, is it not documented anywhere as a procedure)?
If I switch to connecting via cluster vs. SVM in SnapCenter, I could then follow the procedure in the articles. However, in some ways, I'm wondering if this would be less secure because I would be giving a lot of rights to the whole cluster (even though they're limited) vs. admin rights to a few SVMs. Thoughts?
Is it possible to set the account that connects to storage to a domain level service account rather than a local account?

Re: SnapCenter 4.4 - Issue with Cloning, Custom Plug-in for MySQL Server

stanton — Thu, 03 Jun 2021 13:50:43 GMT

Do you have UDEV rules in place for the source filesystem? Is there a UUID for the source?

SnapCenter 4.4 - Issue with Cloning, Custom Plug-in for MySQL Server

TveUA — Thu, 03 Jun 2021 08:43:39 GMT

Hello,

We're backing up a MySQL database on iscsi storage (on Ontap 9.7) via SnapCenter 4.4 using the Custom Plugin for MySQL 2.0.
This seems to work great, but when we try to clone via SnapCenter we get stuck on:

"Mount Filesystem"=>"Building clone for data file systems and associated entities" and following log entry:

Internal error: Attribute (uuid) does not pass the type constraint because: Validation failed for 'Str' with value undef at accessor SCU::Model::Device::uuid (defined at /opt/NetApp/snapcenter/spl/lib/../plugins/scu/scucore/bin/../modules/SCU/Model/Device.pm line 26) line 4
SCU::Model::Device::uuid('SCU::Model::Device=HASH(0x3ce8730)', undef) called at /opt/NetApp/snapcenter/spl/lib/../plugins/scu/scucore/bin/../modules/SCU/Metadata/FSMetadataBuilder.pm line 105

If you look on the host right before it fails you can see that the LUNS are visible via sanlun lun show.
If we take a look at the BuildHostStackCloneInput json file, fsMetadataInfo=>"path"=>deviceInfo=>uuid = null. Wich explains why the error happens. If we, for testing purposes, edit the FSMetadataBuilder.pm file to use "" instead of null the clone works, but we don't want to edit those files ofcourse.

We have no idea why that value is null to begin with, we've tried on both SLES and Oracle Linux, same result.

Does anybody have any ideas?

Re: SnapCenter Role With Minimum Privileges

aladd — Thu, 03 Jun 2021 03:36:00 GMT

If you are providing SnapCenter with the Vserver admin login, it is restricted only to that vserver. if you wish for it to have access to the cluster vserver, then you can use the documentation you have listed to restrict its rights to ONTAPI calls only.

What does your end goal look like for the SnapCenter deployment?

Re: Snapdrive - SDCLI and HA Config

aladd — Thu, 03 Jun 2021 03:30:41 GMT

Hello ,

I'm not sure I fully understand the question. The plugins are remotely installed and register based on host name.

If I understood you correctly you intend on using IP load balancing in front of the SnapCenter server, which should be irrelevant based on IP load balancing.

Please let me know if I am not understanding something in the question.

SnapCenter ps script to list all "suspended hosts" and resources "scheduled on hold"

Spaceball — Wed, 02 Jun 2021 15:21:25 GMT

Hi,
Based on SnapCenter 4.3.0 i'd like to have a ps script

determine all resources (order by plugin-type) where its schedule is actually on hold
and all Hosts which are actually either suspended or down.

I'm new to powershell and do not now how to do this based on the available cmdlet's.

Can someone please help?

SnapCenter Role With Minimum Privileges

TMADOCTHOMAS — Wed, 02 Jun 2021 15:05:58 GMT

Hello,

I need some insight into the best way to lock down the local SnapCenter account on a NetApp cluster. The following articles outline several steps, however the role they outline includes cluster and vserver commands. I don't believe those commands work in an SVM context. The account we have right now resides in SVMs that contain datastores and iSCSI LUNs, all backed up by SnapCenter.

In summary: I want to assign the limited rights recommended below, but can't assign them all if the account is in an SVM <if I understand correctly>. Anyone have an insight into this?

https://docs.netapp.com/ocsc-44/index.jsp?topic=%2Fcom.netapp.doc.ocsc-isg%2FGUID-58C54D8A-71B1-47E2-8BF4-52CCC929526D.html

Re: Snapdrive - SDCLI and HA Config

darraghos — Wed, 02 Jun 2021 12:10:02 GMT

Those documents seem to be for the web interface side of Snapcentre though. What about the plugins? How are they Snapcentre aware i.e. how does SDCLI work when we failover the Snapcentre servers?

Re: Snapcenter Roadmap and Release Dates

darraghos — Wed, 02 Jun 2021 12:07:27 GMT

Thanks! I'll see if I can apply.

Re: Backup MSSQL with snapcenter (migrate to Netapp Storage)

bobby1977 — Wed, 02 Jun 2021 09:29:46 GMT

Hi,

I am facing the same issue.

I have two SQL server running on Vsphere 7. One SQL is working perfectly after adding SCV in Snapcenter. Second SQL server is showing not available for backup

Re: Snapdrive - SDCLI and HA Config

aladd — Wed, 02 Jun 2021 08:15:12 GMT

Configuration for NLB with SnapCenter, please see the following documentation:

https://docs.netapp.com/ocsc-44/index.jsp?topic=%2Fcom.netapp.doc.ocsc-isg%2FGUID-CD383A96-D0B9-436E-83D6-2EB130DC2A11.html&resultof=%22%6c%6f%61%64%22%20%22%62%61%6c%61%6e%63%65%72%22%20%22%62%61%6c%61%6e%63%22%20

https://kb.netapp.com/Advice_and_Troubleshooting/Data_Protection_and_Security/SnapCenter/How_to_configure_NLB_and_ARR_with_SnapCenter

https://docs.netapp.com/ocsc-44/index.jsp?topic=%2Fcom.netapp.doc.ocsc-isg%2FGUID-B4EEE2A8-DAFE-4B0D-863B-3722D2EC0D30.html&resultof=%22%6c%6f%61%64%22%20%22%62%61%6c%61%6e%63%65%72%22%20%22%62%61%6c%61%6e%63%22%20

Re: clone Oracle database on VM guest with Snapcenter 4

Hiep — Tue, 01 Jun 2021 13:18:04 GMT

Thanks for the video link. The video does clarify some questions and includes other links for more information. I'll check it out.

Re: Snapcenter Roadmap and Release Dates

NetApp_SR — Tue, 01 Jun 2021 06:44:51 GMT

Looking at the release dates of previous versions I would expect the release of Snapcenter 4.5 to be the 4th quarter of 2021. We can guess at the features of the new release by looking at the known issues in Snapcenter 4.4. Any problems are normally addressed in the next version. I also see a SnapCenter 4.5 Beta program but I do not know if new participants are still being accepted. This would give you access to see what new features are planned. The release of some features gets delayed due to finding issues in the beta program. If you are looking for a feature that would address some issue you are having let us know. Sometimes there are workarounds for an issue until a permanent resolution is available.

Invitation to participate in SnapCenter 4.5 Beta/Preview/Early Access Program
https://community.netapp.com/t5/Data-Backup-and-Recovery-Discussions/Invitation-to-participate-in-SnapCenter-4-5-Beta-Preview-Early-Access-Program/m-p/165837

Re: NDMP backup fails for .snapshot

kada — Mon, 31 May 2021 13:11:06 GMT

Check this Knowledge base article if that can help :

https://kb.netapp.com/Advice_and_Troubleshooting/Data_Protection_and_Security/NDMP/NDMP_backup_fails_due_to_lack_of_space_on_volume

NDMP backup fails for .snapshot

hend_bs — Mon, 31 May 2021 06:25:15 GMT

Hello,

an NDMP backup keeps on failing only for one path on our NAS appliance with following log:

42952:nsrndmp_save: nasbra01:/nastun/S_AT_TUN/.snapshot NDMP save running on 'server'

84118:nsrndmp_save: Failed to propagate handle 0000000000000000 to E:\Legato\nsr\bin\nsrndmp_2fh.exe child process: The handle is invalid. (Win32 error 0x6)

accept connection: accepted a connection

42958:nsrdsa_save: Performing Immediate save

42617:nsrndmp_save: NDMP Service Log: Session identifier for Backup : 91416

42923:nsrndmp_save: NDMP Service Error: DUMP: path /vol/S_AT_TUN(1)/.snapshot must refer to a specific snapshot name in ".snapshot".

42923:nsrndmp_save: NDMP Service Error: DUMP: Dump failed to initialize.

42923:nsrndmp_save: NDMP Service Error: DUMP: DUMP IS ABORTED

42923:nsrndmp_save: NDMP Service Error: DATA: Operation terminated: EVENT: INTERNAL ERROR (for /nastun/S_AT_TUN/.snapshot)

I didn't find such an error mentioned before in the knwledge data base, can any one help me figure out the cause and if possible a solution ?

Thank you

Re: clone Oracle database on VM guest with Snapcenter 4

vineethnandanam — Sun, 30 May 2021 05:09:36 GMT

https://tv.netapp.com/detail/video/6211803404001/new-snapcenter-4.4-for-oracle-database-backup-restore-and-clone

Please Refer the link. For more Clarity . I think the video will help you to solve the problem.

clone Oracle database on VM guest with Snapcenter 4

Hiep — Thu, 27 May 2021 22:50:21 GMT

Hello,

I am evaluating using Snapcenter 4 for the Oracle db cloning. Oracle is on a VMware linux guest with VMDKs on the datastores hosted on a FAS3000 via FCoE.

For the Snapcenter host I need the Plug-in for Oracle Database. Do I need the Plug-in for VMware vSphere since the Oracle guest is a VM or any other additional plug-in?

cheers,

Hiep

Snapcenter Plugin 4.3.1 Backups are fine. Restore errors with "No backups available at this time."

Steve_Mepham — Thu, 27 May 2021 13:30:36 GMT

We have 2 separate vSphere systems connecting to one AFF-A300 Netapp storage. Both vSphere Vcenter Appliances have an individual Netapp VSC Appliance, and both are creating backups on a scheduled basis.

However, One of the vSphere Snapcenter Plugins is giving the error "No backups available at this time." and will not display the backups to restore from!

I have tried the following, and re registered the SCV plugin. But still get the same error

https://kb.netapp.com/Advice_and_Troubleshooting/Data_Protection_and_Security/SnapCenter/After_upgrade_of_SCV_no_backups_are_available_to_restore

Snapcenter Roadmap and Release Dates

darraghos — Wed, 26 May 2021 12:46:45 GMT

Is there any published feature set for Snapcenter i.e. do we know what might be released with Snapcenter 4.5 and when that might be released?

Re: Unable to Protect Storage VMs

SpindleNinja — Wed, 26 May 2021 12:09:57 GMT

Yeah. the application commands have been deprecated for a few versions now I believe.

The network error is also suspicious, i would open a ticket at this point if no one else has an idea.

Re: Unable to Protect Storage VMs

donnatsun — Tue, 25 May 2021 14:23:59 GMT

I'm not sure if there is and how/why it got there tbh. Looking at the asup - APPDM-APPLICATION.XML it lists (SANDBOX_FS01, Application Template NAS, smartContainer). I created the SVM and the data volume (SANDBOX_FS01) using 9.7 system manager. How do you recommend I proceed from here?

Re: Unable to Protect Storage VMs

SpindleNinja — Tue, 25 May 2021 12:53:59 GMT

Hey Don, Are there any apps defined? Probably not, but worth checking.

The error in the top left also has me wondering what else it could be.

Re: Snapcenter - vSphere plugin - VM consistency exclusion

Tony87 — Tue, 25 May 2021 07:14:31 GMT

Bump

Same question / situation here. How can we exclude consistency for specific VM's on a datastore?

Unable to Protect Storage VMs

donnatsun — Mon, 24 May 2021 23:10:34 GMT

Unable to Protect Storage VMs (Disaster Recovery). (2) FAS8700 clusters, ONTAP 9.7P12, Using system manager on destination to attempt creating a SVM DR Snapmirror relationship. Below is the error message be generated. Do not understand the message..applications other than MAX Data applications.

SnapCenter MongoDB Plugin restore in a VM environment

emre_bcbsa — Thu, 20 May 2021 23:22:15 GMT

Hello,

Anyone tried to restore a MongoDB database in a VMware environment from SnapCenter, using the MongoDB Plugin?

We have a Sharded Replicaset cluster running in Vmware. The snapshots through the MongoDB Plugin are working fine. The issue is doing the restore through the SnapCenter Plugin and we're not sure how this will work with the layers Volume>Snapshot>LUN>Datastore>VM disks.

Or should we explore at doing the restore manually, from the Ontap CLI, unmount the filesystem and snapshot restore?

Anyone else in the same configuration that can share?

Re: Can we get NetApp Snapdrive agents for HP-UX from NetApp..?

NYAFFA200 — Thu, 20 May 2021 15:23:59 GMT

Thanks for the Update..!

Re: Can we get NetApp Snapdrive agents for HP-UX from NetApp..?

stanton — Tue, 18 May 2021 13:12:36 GMT

The NetApp SnapDrive software was deprecated for HP-UX many years ago. As of August 2022 the NetApp SnapDrive software will be completely EOS on all operating systems, so it is not the direction you want to go anymore.

For creating Snapshots and handling other ONTAP activities from HP-UX I would suggest you look into NetApp's Ansible modules (see https://netapp.io/configuration-management-and-automation/) or directly at the new REST API interface for ONTAP that you can script around. While some may argue that it is not as elegant a solution as SnapDrive, it is absolutely more flexible and configurable.

Re: Can we get NetApp Snapdrive agents for HP-UX from NetApp..?

NYAFFA200 — Mon, 17 May 2021 15:26:31 GMT

Please suggest your ideas to get the Snapdrive Agents for HP-UX Servers.

Can we get NetApp Snapdrive agents for HP-UX from NetApp..?

NYAFFA200 — Mon, 17 May 2021 15:25:05 GMT

We are running some workloads on HP-UX Servers in NetApp Infra and we are planning to take snapshots of those NetApp Volumes. We observed, there is no Snapdrive agents available in the download portal. We need to know, whether we can get Snapdrive agents directly from NetApp.

Re: Backup exec support matrix

Fabian1993 — Mon, 17 May 2021 06:16:34 GMT

If your Hardware is on the List they must Support your Enviroment

ll hardware must be present on this HCL for support, unlessa section clearly states that the devices listed are not the only devices supported (such as the Disk Target section)

The NetApp Matrix ist sometimes not updated, maybe some from NetApp can acknowledgethat..

Re: Backup exec support matrix

chinchillaking — Mon, 17 May 2021 02:32:45 GMT

Hi Fabian,

Thanks for your reply, I know Veritas support but not in NetApp support matrix https://mysupport.netapp.com/matrix/.

As discuss with Veritas support before, they did not answer if backup problem occured when use Backup Exec 21 with OnTap and Veriats will take all support ownership.

Best regards,

Chung

Re: Backup exec support matrix

Fabian1993 — Sun, 16 May 2021 13:18:16 GMT

Check the HCL:

https://www.veritas.com/content/support/en_US/doc/BE_21_HCL

max ONTAP 9.7 with Version 21

Re: SnapCenter LDAP(s) settings?

Hans088 — Fri, 14 May 2021 07:15:58 GMT

Unfortunately no, we're now on version 4.4 and snapcenter still uses plaintext passwords for accounts used by MSSQL/MySQL/Exchange agents. Abysmal.

Backup exec support matrix

chinchillaking — Fri, 14 May 2021 02:29:41 GMT

Hello,

NetApp support matrix NDMP support to Backup Exec 20, may we know when will support Backup Exec 21?

Best regards,

Chung

SCC-00161: Application unquiesce for the plug-in [MAXDB] failed with exit code [102] and error :

C_Greiner — Wed, 12 May 2021 13:32:35 GMT

Hello Netapp.Community,

we still got problems with our MaxDB-Backup.

After changing some configurations to backup also the Log Area we got this error:

SCC-00161: Application unquiesce for the plug-in [MAXDB] failed with exit code [102] and error : Operation failed. Reason: The watchdog killed the running operation. Verify that plugins/wrapper/wrapper binary has execute permission set. Increase the WRAPPER_TIMEOUT_IN_MSEC value in agent.properties if operations require more time to complete. SCC-00185: Cleanup for plug-in [MAXDB] failed with exit code [102] and error:

Can someone help us.

Best Regards

Re: NetApp Data Broker 1.1 and SnapCenter for vSphere - Backups not Deleting

gherardini — Tue, 11 May 2021 19:15:54 GMT

I am having this same issue, Data Broker 1.0.1, vCenter 6.7U3m.

My issue started when I added a second Policy for a shorter retention period (35 days vs 42 days) and edited each of our Resource groups to use the new, shorter retention policy. Maybe it will only address its own 35 days and will begin to delete snapshots outside of the retention period?

The only way that I've had this succeed is to suspend the current Resource Groups, create new Resource Groups using the new Policy, and let the new RG reach its retention period and begin to delete its own, outdated snapshots. This requires a manual deletion of the older snapshots from the original RG so that the volume won't fill up. If you simply delete the original RG (instead of suspending) before creating the new one, you will lose ALL your snapshots from that RG (can you fix that?? maybe with a selection to "Do you wish to delete all your current snapshots associated with this RG? --No, I don't). I learned THAT lesson one day....

But currently I still have the issue that if i edit the RG with a new policy, my retention policy is not propagating the the previous policy's snapshots. Still waiting to see if the new Policy will get to deleting its own snapshots w=once they are aged over 36 days...

thx

Snapdrive - SDCLI and HA Config

darraghos — Tue, 11 May 2021 11:20:40 GMT

We are loking to setup our Snapcentre 4.4 P1 servesr behind a load balancer. The load balancer provide s VIP/DNS Alias to access Snapcentre. From the SDCLI perspective, how to we specif this VIP?

Re: SnapCenter plugin migration to new vCenter

aladd — Thu, 06 May 2021 04:56:41 GMT

Hello ,

Depending on if you are moving the plugin server to the new environment. Yes, just a change of the URL to the new vcenter should be sufficient.

If you need to migrate metadata to a new instance, you can use the following.

https://docs.netapp.com/us-en/sc-plugin-vmware-vsphere/scpivs44_migrate_from_snapcenter_backup_metadata_to_the_virtual_appliance.html

Re: Snapcenter Nested Clones of Oracle Databases

darraghos — Wed, 05 May 2021 21:24:25 GMT

Awesome! Thanks! 😀

Re: SCCLI Tool

darraghos — Wed, 05 May 2021 21:22:42 GMT

Looks like it's making web services call but not powershell via WinRM or the like 🙂

Re: SnapCenter for VMWare

jcolonfzenpr — Fri, 30 Apr 2021 00:42:27 GMT

i will post something here just to ignite the conversation:

All your questions can be answered with a simple documentation read.

http://docs.netapp.com/ocsc-44/topic/com.netapp.doc.ocsc-con/GUID-4F08234F-71AD-4441-9E54-3F2CD2914309.html

Hope this help!

SnapCenter for VMWare

FelixZhou — Thu, 29 Apr 2021 15:50:26 GMT

We have SnapCenter for VMware in place will start working on details.

A few questions may make our work easier.

1. if we take snapshots on VMs through the SnapCenter, will the snapshots be stored on NetApp volumes or VMWare datastores?

2. we plan to use VM snapshots to replace NDMP backups, since we have big amount of VMs, is that a good idea to schedule snapshot policies on datastores instead of on VMs? will the snapshot process takes longer if we do so? which way is the better choice?

3. we currently using Veritas NetBackup to back up VMs by using NDMP, sometime the snapshot causes issue on VMs, snapshot take too long or not delete correctly. is that possible we can direct the snapshot from datastore to volume? or any other ways to improve the process?

thanks for sharing !!

Re: SnapCenter plugin modifies Windows hostfile

kombayn — Tue, 27 Apr 2021 07:45:43 GMT

Hello guys. Could someone check on his hosts with SnapCenter plugin installed, if the Windows host file contains SVM entries?

SnapCenter plugin migration to new vCenter

Sanjuro — Mon, 26 Apr 2021 08:50:58 GMT

We're running vCenter 6.7 U3 and are migrating to vCenter 7.0. Both are appliances, fresh install of 7.0, no Windows at all. We're running SnapCenter 4.4 with the databrokers. One databroker per vCenter. Is it just a matter of pointing to the new vCenter in the databroker webpage? <https://ipaddress:8080>

I can't seem to find the instructions for that scenario.

Thanks!

Re: Exchange KB5001779 breaks SnapManager for Exchange

borismekler — Wed, 21 Apr 2021 15:28:23 GMT

wrote:
Of course running Exchange without this security patch is risky and thereford not a good solution - but running without backup is not aceptable at all.

I'd rather use crash-consistent snapshots than expose the org to a sev 9.8 RCE.

Re: Migrate an snapvault destination aggr to a new controller ?

aborzenkov — Wed, 21 Apr 2021 14:37:28 GMT

wrote:
the old filer is cluster mode , 8.3 ontap version , it is v32340 which is a very old system
the new filer is Fas2720 , probably will be received with ontap 9.8 version ,

v32340 is probably a typo, and "v" indicates V-Series which may not even have any shelves that can be relocated to start with.

This really goes far beyond the simple yes/no question that fits forum format, rather this is migration project to design and implement.

Re: Migrate an snapvault destination aggr to a new controller ?

ousturali — Wed, 21 Apr 2021 14:18:24 GMT

the old filer is cluster mode , 8.3 ontap version , it is v32340 which is a very old system

the new filer is Fas2720 , probably will be received with ontap 9.8 version ,

the disk assignment is ok, but i am not really sure about the snapshots of the snapvault destination volumes

we mustnot loose them , because they are the customers long time archival backup data.

anyway thanks a lot for your answers

Re: Migrate an snapvault destination aggr to a new controller ?

aborzenkov — Wed, 21 Apr 2021 09:53:47 GMT

wrote:
For most scenarios, the aggregate, volume, and snapshot structure will remain intact when moving the old shelves to the new controllers.

That is true only for moving aggregates between 7-Mode systems (unless something has changed very recently) and there is no information what OP has. It is quite unlikely that any new system today will be (or even supports) 7-Mode.

Re: Migrate an snapvault destination aggr to a new controller ?

ttran — Wed, 21 Apr 2021 09:33:29 GMT

Hi Ousturali,

For most scenarios, the aggregate, volume, and snapshot structure will remain intact when moving the old shelves to the new controllers. One exception is when you have snapshots that are 32-bit and the new controllers are running an ONTAP version that no longer supports 32-bit snapshots. In addition, 32-bit aggregates will need to be converted to 64-bit aggregates. Data ONTAP 8.3 and newer no longer supports 32-bit data. So how impactful this will be is determined by the current ONTAP version the data is on and what ONTAP version the new controllers are running.

Another thing to keep in mind is if you are also migrating the disks containing the root aggregate over then you will want to perform a formal upgrade of the existing data to match the ONTAP version of the new controllers. Booting a newer version of ONTAP with an older major revision of ONTAP will update the RAID and WAFL versions on every disk but the actual ONTAP kernel isn't properly upgraded resulting in a messy situation to recover to make the data accessible again. If the root disks & aggregate aren't moved over then you will have to recreate the SVM/VSERVER, shares, LIFs, and other configurations before you can access the old data.

Here is a reference document for moving a shelf to another controller:

Physically moving an aggregate composed of disks

Regards,

Re: Exchange KB5001779 breaks SnapManager for Exchange

Hetim — Wed, 21 Apr 2021 09:21:10 GMT

We had exactly the same situation - no SME backups were possible since the security patch was installed.

Deinstallation of KB5001779 was possible (Exchange 2016 CU20) AND a solution for this issue: SME backups now are running fine again!

Of course running Exchange without this security patch is risky and thereford not a good solution - but running without backup is not aceptable at all.

We will migrate to SnapCenter as soon as possible.

Migrate an snapvault destination aggr to a new controller ?

ousturali — Wed, 21 Apr 2021 06:53:38 GMT

Hi to all

The customer has a old filer, with snapvault destination volumes and aggrs inside that.They bought a new controller with new disks.We will ,first configure the new filer for future snapvault destinatinon.Then they want us to connect the old filers shelves to the new controller.

My question is about the snapvault volumes, they have been in place for the last 5 years, and lots of data inside them.

If we connect the old shelves to new filer, and assign the disks to this new filer, do we have the old structure of aggrs and volumes with snapshots?.Or do we loose snapshots or something else,

In my opinion, all the aggr, volume and snapshots must retain ,but i havent tried something like that.

what is your opinion?

regards

Re: Exchange KB5001779 breaks SnapManager for Exchange

Combato — Mon, 19 Apr 2021 13:43:49 GMT

We also have big issues with SnapManager 7.2.0

Can't roll back the patch either for the same reason (security).

Will need a solution pretty soon, otherwise we will go for another vendors product.

Re: Exchange KB5001779 breaks SnapManager for Exchange

DAR1 — Mon, 19 Apr 2021 13:13:48 GMT

Is there currently an issue with that bug page?, I am unable to load it here, it just shows loading...

We have reviewed our environment and we are unable to roll back the latest patch for security reasons.

Re: Exchange KB5001779 breaks SnapManager for Exchange

NTAp-AVS — Mon, 19 Apr 2021 11:57:01 GMT

A Bug has been opened to address this issue . NetApp Engineering is actively working to get a workaround/solution

https://mysupport.netapp.com/site/bugs-online/product/SME/BURT/1393644

Please subscribe to this bug

Until a workaround/ solution is identified, only if you are able to (after assessing the security needs of your systems) , we recommend you to roll back the patch.

Here is the kb that documents

https://kb.netapp.com/Advice_and_Troubleshooting/Data_Protection_and_Security/SnapManager_Suite/SME_unable_to_see_database_after_Exchange_Security_update_KB5001779

Re: Exchange KB5001779 breaks SnapManager for Exchange

BenRi — Mon, 19 Apr 2021 09:29:51 GMT

We're using Snapmgr in 7mode. Snapcenter is not supported so we can 1. uninstall that KB, which is not an option or 2. look at another backup software. Thats not a "solution" netapp...

Re: Exchange KB5001779 breaks SnapManager for Exchange

LW — Mon, 19 Apr 2021 08:37:52 GMT

Here ist the NetApp KB article: SME unable to see database after Exchange Security update KB5001779 - NetApp Knowledge Base
For me it is not an option to uninstall critical security updates. However, Exchange Server 2016 is only supported up to CU11. Any newer CU is not supported and therefore I would not expect an solution by NetApp.

I have migrated to SnapCenter 4.4. (which was already runnig for SQL, VMWare and HANA). This is documented here

http://docs.netapp.com/ocsc-44/index.jsp?topic=%2Fcom.netapp.doc.ocsc-isg%2FGUID-C3F798B8-5C60-4828-B0C5-48A666251066.html

Please note that you can not restore an SnapManager Snapshot with SnapCenter.

Backup and restore works fine with SnapCenter and the Exchange Scurity updates.

Re: Exchange KB5001779 breaks SnapManager for Exchange

borismekler — Mon, 19 Apr 2021 07:59:15 GMT

SnapCenter (at least version 4.4P1) seems fine.

Re: Exchange KB5001779 breaks SnapManager for Exchange

ADVUNIBN1 — Mon, 19 Apr 2021 07:48:47 GMT

Is this only a SnapManager issue or is SnapCenter with this Exchange Patch also affected?

Re: Exchange KB5001779 breaks SnapManager for Exchange

borismekler — Sun, 18 Apr 2021 09:36:50 GMT

For what it's worth, I've been advised by support to uninstall the security update. Since that would open the servers up to severety 9.8 remote code execution vulnerabilities, I'd rather forgo backups for a while, or even look for an alternative solution in general.

Re: Exchange KB5001779 breaks SnapManager for Exchange

Annelie — Sun, 18 Apr 2021 09:29:39 GMT

Yes, we have the exact same error in our logs for SnapManager for Exchagne.

No backups are going after the installation of the kb!

Re: Unable to locate /usr/lib64/libHBAAPI.so library

AlexDawson — Sat, 17 Apr 2021 06:31:00 GMT

Hi there!

From https://library.netapp.com/ecmdocs/ECMP1217221/html/GUID-042DBE6A-090F-4018-A651-F1A8EA58B1AD.html -

To avoid this problem, make sure you have the correct management software package for your HBA and host architecture installed:

For QLogic HBAS, install the QLogic QConvergeConsole CLI package.
For Emulex HBAs, install the Emulex OneCommand Manager core application (CLI) package.

Hope this helps!

Re: Exchange KB5001779 breaks SnapManager for Exchange

LW — Fri, 16 Apr 2021 17:53:42 GMT

Where is the log extract from (wich logfile?)

Normal powerhsell comands (in Exchange Shell) like get-exchangeserver -status -identity run as expected.

In my case: The Windows schedule job runs but actually nothing happens and there is no log created under C:\Program Files\NetApp\SnapManager for Exchange\Report\Backup.

Re: Exchange KB5001779 breaks SnapManager for Exchange

DAR — Fri, 16 Apr 2021 15:03:40 GMT

We are getting an error 0 event with the following message when attempting to run a snapmanager backup.

There are no Storage Groups/Databases that are configured to be backed up by SnapManager

Re: Exchange KB5001779 breaks SnapManager for Exchange

borismekler — Fri, 16 Apr 2021 14:43:52 GMT

Negative, I am not seeing these errors. get-exchangeserver -status -identity (servername) returns normal output.

Re: Exchange KB5001779 breaks SnapManager for Exchange

NTAp-AVS — Fri, 16 Apr 2021 14:40:43 GMT

Upon reviewing further

[04/16 12:59:23 754] Getting dash board info...
[04/16 12:59:23 754] Calling PowerShell to get current exchange server...
[12:59:23.754] [PowerShell Cmdlet]: get-exchangeserver -status -identity <Exchange server name>
[04/16 12:59:23 754] [PowerShell Cmdlet]: get-exchangeserver -status -identity <Exchange server name>
[12:59:23.911] [Remote PowerShell Cmdlet Error]:The syntax is not supported by this runspace. This can occur if the runspace is in no-language mode.
[04/16 12:59:23 912] [Remote PowerShell Cmdlet Error]:The syntax is not supported by this runspace. This can occur if the runspace is in no-language mode.

======================================

Please verify if you are seeing the same errors in your environment , it is an indication that Exchange Powershell cmdlets are failing.

Re: Exchange KB5001779 breaks SnapManager for Exchange

NTAp-AVS — Fri, 16 Apr 2021 13:26:20 GMT

This issue is new and it will need to be investigated by NetApp engineering, once we have a solution from them this forum will be updated with a BUG number or a KB article

Thanks

Re: Exchange KB5001779 breaks SnapManager for Exchange

DAR — Fri, 16 Apr 2021 13:11:53 GMT

It appears that we have all logged calls with Netapp for this issue, our case number is: 2008734252

As this is clearly a problem affecting multiple customers is it possible for any updates with the progress regarding a solution to be added to this thread?

Re: Exchange KB5001779 breaks SnapManager for Exchange

harness — Fri, 16 Apr 2021 13:00:40 GMT

As yet we do not have a workaround for this issue. Please open a support ticket in order to get this issue addressed.

Re: Exchange KB5001779 breaks SnapManager for Exchange

borismekler — Fri, 16 Apr 2021 12:21:53 GMT

Case number 2008734372 here.

Since this appears to be an easily reproducible issue, I'm hoping for a speedy resolution.

Re: Exchange KB5001779 breaks SnapManager for Exchange

LW — Fri, 16 Apr 2021 12:15:33 GMT

Same issue here. After updating the security Updates (April 2021) SnapManager breakes and right now I have now workaround.

Will also create an support ticket. Can you please share your Case IDs?

Re: Exchange KB5001779 breaks SnapManager for Exchange

DAR — Fri, 16 Apr 2021 11:18:07 GMT

We are also experiencing the same issues following the installation of this patch, We have logged this with Netapp and are currently awaiting a response.

I am not aware of any work around at this time and from similar issues posted to Microsoft here :

https://techcommunity.microsoft.com/t5/exchange-team-blog/released-april-2021-exchange-server-security-updates/ba-p/2254617/page/3#comments

It does not appear that Microsoft have a workaround at this time.

I will try to post here if and when we get any more info or a fix.

Exchange KB5001779 breaks SnapManager for Exchange

borismekler — Fri, 16 Apr 2021 10:26:57 GMT

I have a few older servers still using SnapManager rather than SnapCenter, and the critical security update that Microsoft released a few days ago (KB5001779) completely breaks SnapManager compatibility - backup jobs no longer run, and console reports "Unable to determine Exchange virtual server group status before SnapManager operation" straight upon opening. This is happening both in Exchange 2013 and Exchange 2019, same error. I have opened a case with support, but while I'm waiting for a response, has anyone else run into this, and is there a workaround?

Unable to locate /usr/lib64/libHBAAPI.so library

srikanthmutyala — Wed, 14 Apr 2021 18:14:26 GMT

servername:/software/Netapp\ # sanlun fcp show adapter
Unable to locate /usr/lib64/libHBAAPI.so library
Make sure the package installing the library is installed & loaded

This is on Oracle Linux 7.9. Let mem know what needs to be done to fix this issue.

TIA

SnapCenter Remote Offline Verification of MS SQL AOAG database members

bobalon — Mon, 12 Apr 2021 15:40:29 GMT

Hi there

Looking for some help with the Verification of MS SQL AOAG database members. I have a simple AOAG setup across 2 SQL Servers for DB1.

I want to offload CheckDB to an offline server on a remote site so the assigned policy Schedule snapmirrors the data and log volumes etc. to the second site and the remote serve is selected as the verification server.

What I am seeing is that the Database from the Primary node always fails verification as an NTFS storage snapshot cannot be created, the other database from the secondary node is completed ok on the same server. I have proven this by failing over the AOAG and re-running the Snapcenter Job., with the primary always failing.

The fix seems to be to specify the Tablock option on the verification tab of the SnapCenter Policy. This allows both the Primary and Secondary Database files to complete Checkdb on the remote server successfully.

I have also found that if I clone the primary DB and mount on a remote server that I can execute CheckDB from SQL manually with no issues.

Therefore, can anyone confirm if this issue with checking a Primary AOAG Database is to be expected and that Tablock should always be specified for an AOAG database when verify remotely?

Or suggest if I am missing something in my setup. Note file/folder permissions have all been checked and verified, and SQL servers are same version etc

Finally does anyone know what snapcenter skips (other than the NTFS snapshot) when the Tablock option is selected?

thank you

Invitation to participate in SnapCenter 4.5 Beta/Preview/Early Access Program

Deb — Mon, 12 Apr 2021 09:27:49 GMT

Thank you for being our valued partner and participation in our previous EAP programs.

We are pleased to announce SnapCenter 4.5 Beta/Early Access Program(EAP), launching on April 17, 2021.

Expression of interest to participate, can be submitted with this form.

We will be sharing the details of SC 4.5 payload with the participants over email.

EAP or community page for SC 4.5: https://community.netapp.com/t5/SnapCenter-4-5-EAP-Beta/ct-p/snap_45_eap

*Signed NDA is a pre-requsite to get access to the preview build.

Regards

SnapCenter Product Team

Snapcenter datastore jobs are slow

GREGD — Wed, 07 Apr 2021 13:10:11 GMT

I have snapcenter 4.3. 2 ,Plug-in for vspere virtual appliance 4.2 ,vcenter 6.7

I see today that the job where 4 datastore were backuped need 30 minutes that is finish.

is this normal? snapshot of datastore is finish in few second when i look at netapp volumes.

I was thinking the job is finish in max 10 minutes. but i check the logs all job are so slow.