Options
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
ndmpcopy from multiple sources to same destination folder fails
2021-07-20
12:08 AM
3,632 Views
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
We have 3 NetApp storages that face the same problem when we use them as ndmpcopy destinations. In summary the issue looks like this:
When we use ndmpcopy to dump simultaneously different files from multiple sources to the same folder on a specific volume all of them but one (the one that started 1st) are failing with the following error:
Ndmpcopy: Starting copy [ 2586 ] ...
Ndmpcopy: 10.250.1.190: Notify: Connection established
Ndmpcopy: 10.13.4.65: Notify: Connection established
Ndmpcopy: 10.250.1.190: Connect: Authentication successful
Ndmpcopy: 10.13.4.65: Connect: Authentication successful
Ndmpcopy: 10.13.4.65: Log: Session identifier: 55018
Ndmpcopy: 10.13.4.65: Log: Session identifier for Restore : 55018
Ndmpcopy: 10.250.1.190: Log: DUMP: creating "/vol/NFS_Prod1Envs_EntApp_LC3/../snapshot_for_backup.29777" snapshot.
Ndmpcopy: 10.250.1.190: Log: DUMP: Using Partial Volume Dump of selected subtrees
Ndmpcopy: 10.250.1.190: Log: DUMP: Date of this level 0 dump: Mon Jul 19 09:19:47 2021.
Ndmpcopy: 10.13.4.65: Log: RESTORE: Error: Another restore is in progress on the same destination.
Ndmpcopy: 10.250.1.190: Log: DUMP: Date of last level 0 dump: the epoch.
Ndmpcopy: 10.250.1.190: Log: DUMP: Dumping /vol/NFS_Prod1Envs_EntApp_LC3/QTree/new/test-flat.file2 to NDMP connection
Ndmpcopy: 10.250.1.190: Log: DUMP: mapping (Pass I)[regular files]
Ndmpcopy: 10.250.1.190: Log: DUMP: Reading file names from NDMP.
Ndmpcopy: 10.250.1.190: Log: DUMP: mapping (Pass II)[directories]
Ndmpcopy: 10.250.1.190: Log: DUMP: estimated 3234535 KB.
Ndmpcopy: 10.250.1.190: Log: DUMP: dumping (Pass III) [directories]
Ndmpcopy: 10.250.1.190: Log: DUMP: dumping (Pass IV) [regular files]
Ndmpcopy: 10.13.4.65: Log: RESTORE: RESTORE IS ABORTED
Ndmpcopy: 10.250.1.190: Log: DUMP: Network communication error
Ndmpcopy: 10.13.4.65: Log: DATA: Operation terminated: EVENT: INTERNAL ERROR (for NFS_Prod1Envs_SSD_LC1)
Ndmpcopy: 10.13.4.65: Connection halted: HALT: Internal error!
Ndmpcopy: 10.250.1.190: Log: DUMP: DUMP IS ABORTED
Ndmpcopy: 10.250.1.190: Log: DUMP: Deleting "/vol/NFS_Prod1Envs_EntApp_LC3/../snapshot_for_backup.29777" snapshot.
Ndmpcopy: Notify: Transfer failed
Ndmpcopy: Done
What baffles us is the fact that we have several other older NetApp storages on which this issue doesn’t occur and the ndmpcopy of multiple files, from multiple sources to the same destination folder works without any issue.
Few technical details:
- ONTAP versions of problematic NetApps: 9.7P12, 9.7, 8.2.5
- ONTAP versions of few none-problematic NetApps: 8.1.3P3
- QTree is used on all the volumes
- Ndmp version 4 is used on all
- We use SVM-mode ndmp on the CDOT NetApps (tests with node-scope showed same results)
We will appreciate it if you share any knowledge or ideas with us on how this issue can be resolved. Will be happy to provide any technical details or configuration specs if that can help.
Thank you!
Solved! See The Solution
1 ACCEPTED SOLUTION
eladgotfrid1 has accepted the solution
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello @eladgotfrid1 ,
The functionality to perform multiple ndmpcopy restores to the same location was intentionally disabled starting with ONTAP 8.2+, which is why you haven't seen the issue with the systems running ONTAP 8.1.
Regards,
Team NetApp
Team NetApp
4 REPLIES 4
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Clarification:
9.7P12, 9.7, 8.2.5 are the NetApps that host the destination folder
Source NetApps are any others including the 9.7P12, 9.7, 8.2.5
eladgotfrid1 has accepted the solution
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello @eladgotfrid1 ,
The functionality to perform multiple ndmpcopy restores to the same location was intentionally disabled starting with ONTAP 8.2+, which is why you haven't seen the issue with the systems running ONTAP 8.1.
Regards,
Team NetApp
Team NetApp
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thank you.
Can this behavior be overridden somehow?
Can you point me to the doc/kb mentioning this?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
The functionality was disabled in Data ONTAP 8.2 and later due to the following bug:
https://mysupport.netapp.com/NOW/cgi-bin/bol?Type=Detail&Display=611765
I am not currently aware of any plans to reintroduce the ability to perform multiple concurrent restores to the same volume in future versions of ONTAP.
Regards,
Team NetApp
