Help Migrating Data from Filer to Win Server

[expat_macgeek]

All,

I'm hoping someone out there can help me out with this.  Let me start by saying that I know nothing about NetApp filers.  Nada.  I have been tasked with migrating data off of an organizations Filer and onto a Windows 2003 file server so that the NetApp can be reconfigured.  Our organization does not support NetApp, and this group decided to buy one without any foresight into how it should be used correctly, backed up, or even operated at a basic level.  The guy who purchased / implemented it has left the organization, so nobody here is much help with this task.

Here's what I know:

- Filer is a NetApp F800 series.

- Configured as CIFS, and user data is being redirected to shares on the Filer.

- They have no way to backup the data on the Filer.

- The Filer needs to be reconfigured to present as volumes attached to a server in order to be protected by our current backup solution.

Is there a way to move that data off of the Filer using the Filer itself / DataONTAP.  I need to move all of that data to a shared folder on a Win2k3 file server and redirect users to the new location.  Redirecting the users isn't a problem, but getting it off the filer all in one shot is the problem.

I don't mean to sound like an idiot here, but this is just not my area.  I know that this isn't a lot of information on the setup we've got, but I don't know jack about it, but I can provide more if needed.  Just tell me where to look.  Nobody knows anything about their own piece of hardware here, so I'm really flying blind.  Any assistance would be greatly appreciated.

Thanks in advance,

Chris

[pascalduk]

You can use robocopy of course.

[benjstarratt · Accepted solution]

I too will suggest robocopy.  The newest version from Microsoft (included with Windows 7) is multithreaded which is great for massive amounts of data.  I would suggest a process something like this:

Environment: NetApp called toaster with multiple shares.  Server called microwave.

Document share permissions on toaster by looking at them in Windows Explorer

Build a baseline

Create multiple shares on microwave with the same names as the ones on toaster

Run robocopy \\toaster\share1 \\microwave\share1  /copyall /mir /r:0 /w:0 /log:robolog.txt in a command prompt

Repeat for each additional share.

Review robolog.txt to see how many files were missed (especially the very bottom) and troubleshoot

Get an outage window

Run robocopy \\toaster\share1 \\microwave\share1  /copyall /mir /r:0 /w:0 /log:robolog.txt in a command prompt

Repeat for each additional share.  This will only copy the changed data which will reduce the outage duration.

Repeat for each additional share

Review robolog.txt to see how many files were missed (especially the very bottom) and troubleshoot

Shutdown toaster

Rename microwave to toaster

There are a lot of different tweaks depending on your tolerance for downtime, need for speed and number of shares, but this should get the data off the NetApp.  Play with robocopy a bit to get the hang of it.

Final step: Keep toaster in the lab and find out why people adore them.

[expat_macgeek]

In response to pborkstrom and benjstarratt:  Thanks for the replies - I appreciate you taking the time.

I probably should have put this in my original post.  While I didn't try robocopy, I did try the initial move using Xcopy.  Robocopy is definitely more robust, that I understand, but the problem that I ran into with Xcopy will likely be the same with Robocopy.  Just a bit more info - there is 600GB of data on the NetApp, and the amount of downtime that I have is little to none.  (I know - it's ridiculous, but that's the situation.)  What makes it even more difficult is that the data isn't coming off in the typical NTFS folder order, which I think has to do with the way that CIFS stores data on a volume.  If I do this using Robocopy, I will likely have to write a script for each users folder in the share in order to migrate groups of users.

When I initially ran Xcopy to copy the data and retain the source permissions, the process would fail when it hit file names that were too long, citing insufficient memory to continue.  Unfortunately, given the number of users and the sensitive nature of their personal files, contacting each user to shorten file names whenever this happens isn't going to be possible if I ever want this process to get completed.

When you also factor in that the NetApp is very slow to access (which is another issue altogether), using Robocopy could take quite a while to complete.  The Filer has a GigE connection to a C4506 core where the target server is also connected, and both the source and target are on the same VLAN.

So given that information, would your suggestion of using Robocopy still apply?  I honestly can't think of another way, but I appreciate the suggestions.

Thanks!

[benjstarratt]

Absolutely.  Robocopy is a good tool for this task.  Since the second (or third, fourth or fifth) pass will be iterative, there will be little data actually moved.  Do a couple trial runs and check how long the incremental runs take.  I have used robocopy for exactly the tasks you are contemplating on much larger volumes of data.  The only issue is when there are many millions of tiny files (less than 4k).  Then a raw copy tool (ghost/netbackup image) is a better solution though not useful in this case.  Multithreading can speed it up a lot.  Run the robocopy directly from the file server and give it at least twice as many threads as cores.  Usually, you can get the switch over to be less than an hour including renaming the new server.  It is possible to be even quicker--but then things start getting complicated.

It is possible to programmatically recreate share permissions using command line tools if you have more than you can do manually.

Finally, I've never played with the Microsoft File Services Migration Tool, but I am aware of its existence: http://www.microsoft.com/windowsserver2008/en/us/fsmt.aspx

NetApp emulates a Microsoft CIFS server very closely (much, much better than Samba for instance).  It might be possible to use the file service migration tool to help with the migration.  If you try it, post the results!

Good luck!

[jain]

Robocopy is the way to go.

If you can spend few $$$, I would suggest Securecopy which handles local groups very well, multithreads and also migrates shares making it much easier to plan and implement.

[expat_macgeek]

Thanks benjstarratt - Robocopy is working out well.  I haven't run into the any of the problems that occurred with Xcopy, and despite the fact that I haven't redirected the users to the new home location, it's at least working as a temporary backup solution.  Until I can cut them over, I've schedule it to run nightly with the /mir switch, so the destination accurately reflects the source.

Much appreciated

V/r - Chris

[pborkstrom]

My 2 cents...

I just learned Netapp Filers this year.  Being a Windows admin w/absolutely no unix/storage experience, I've come quite far.  Netapp filers are very powerful devices.  Windows shares, once set up, and very low maintenance.  The disk array (Raid_DP) is robust and can survive disk failures.  Due to the filers having multiple disks, performance is stellar.  I can understand the need to fully understand your platform, but if you have the opportunity to learn about the filers - doit.

[steveb]

Chris,

How about DON"T migrate the data off the filer's native CIFS shares - these offer many benfits over shares provisioned via LUNS to a traditional Windows filer server (performance, availaibility, integrated snapshot backup, flexible volumes, capacity efficiency etc), see http://www.netapp.com/us/solutions/applications/windows-file-services/

Serving files is NetApp's beach head into many accounts.

An alternative approach is to integrate the filer into the backup environment - what do you have deployed in the way of backup software applications and backup targets (disk pools, VTL, tapelibary)?

You can either mount Snapshots from the backup server over the LAN and backup files over CIFS (slow and cheap) or do NDMP dumps, typically over FC direct to a library or disk pool.

The system engineer from your supplier should be able to help you with this, some documentation is here http://www.netapp.com/us/library/white-papers/wp-3066.html

- Steve

[expat_macgeek]

@Steve,

Circumstances demand that this be done.  NetApp is not a supported storage solution in the organization that I work in, and CIFS shares cannot be protected by our currently approved backup solution.  The reason for moving the data to a temporary location is so that the Filer can be reconfigured to present as volumes attached to a Windows file server so that it can be snapped by our current backup solution - FilesX.  Right now, it's just twisting in the wind.  The person who purchased / installed the filer gave no thought to backing it up, and has left the organization.

While I appreciate the suggestions, and I agree on the value of the NetApp Filer, our organization has an approved hardware baseline that has to be maintained for exactly this reason.

[pascalduk]

Your main problem is that the F800 is an extremely old and relative slow filer and no longer supported

[expat_macgeek]

No kidding - these people had no idea what they were getting themselves into.  When they purchased the F800 is was just about to be EOL'd.