topic Re: Volumes left behind after verification completes in ONTAP Discussions

Volumes left behind after verification completes

BrendonHiggins — Thu, 05 Jun 2025 07:33:32 GMT

We have two MS Exchange 2003 clusters, V01 & V02. Each cluster has two nodes and runs Windows 2003. Ent. Ed, with SnapDrive 4.2.1 and SME 3.2. Looking to upgrade soon but not this week...

We use a 2^nd server to verify the backups as the server load is high on the Exchange boxes.

The problem is once the verification has finished the flexclone volumes are sometimes left behind. It is normally just the transaction log volume or the database volume. The problem tends to happen on V01 more frequently and we have completed a bare metal re-install of the cluster nodes to try and resolve the issue. – No success.

There are no events on the Filer logs (FAS3070 cluster) and I have not been able to work out a pattern as to when it happens. The work around is to check the filer each morning for the flexclone volumes and offline / delete them.

Does anyone have any ideas to a possible solution. I have heard ESEUTIL throttling can cause this issue but testing has show the problem happens regardless.

Thanks for taking the time to read this and think about the issue. Any help / ideas welcome.

Re: Volumes left behind after verification completes

sflynn — Fri, 19 Sep 2008 18:06:02 GMT

Hi Brendon...

I'm not sure why SME isn't cleaning up these volumes after the verification is complete. It should delete the works and leave everything clean.

Is there any info in the backup reports for a particular verification job that gives any clue as to what's happening? Or even the event logs on the verification server...those would be helpful to look at as well.

I'll follow up with a few folks to see if this is a known issue or not. I'll post what I find probably next week.

Thanks...

Shannon

Re: Volumes left behind after verification completes

BrendonHiggins — Mon, 22 Sep 2008 09:14:24 GMT

Luck would have it that the issue happened over the weekend.

SnapDrive_gbdc01exmbf01db_clone_of_exchsnap__gbdc01exmbv01_09212008_190000__weekly_snapshot_0

The SME log is standard expect for this:

Operation completed successfully in 45.735 seconds.

Dismounting LUN C:\Program Files\NetApp\SnapManager for Exchange\SnapMgrMountPoint\MPDisk001 of Snapshot ...

The virtual disk may not be connected, because its mount point cannot be found.

(SnapDrive Error Code: 0xc0040221)

Re-trying to force dismounting LUN...

SnapManager will pause 70 seconds after force dismount, please wait...

SnapDrive failed to dismount the snapshot.

Error Code: 0xc0040221

The virtual disk may not be connected, because its mount point cannot be found.

Mounting Snapshot for LUN E of Computer GBDC01EXMBN02

Mount point directory

Snapshot will be mounted on subdirectory

This Snapshot is mounted as the drive .

Mount Snapshot succeeded.

RUNNING TRANSACTION LOG INTEGRITY VERIFICATION

Transaction log directory is located at:

C:\Program Files\NetApp\SnapManager for Exchange\SnapMgrMountPoint\MPDisk001\EXCHSRVR\MDBDATA\SG1\

Start running ESEUTIL on "C:\Program Files\NetApp\SnapManager for Exchange\SnapMgrMountPoint\MPDisk001\EXCHSRVR\MDBDATA\SG1\"...

Command: ["C:\Program Files\Exchsrvr\bin\eseutil.exe" /ml E00]

Checking...

Microsoft(R) Exchange Server Database Utilities

Version 6.5

Initiating FILE DUMP mode...

************

Nothing on the Windows server log

*************

Console Messages

Sun Sep 21 20:04:54 BST : LUN /vol/SnapDrive_gbdc01exmbf01db_clone_of_exchsnap__gbdc01exmbv01_09212008_190000__weekly_snapshot_0/sg1db.lun unmapped from initiator group viaRPC.iqn.1991-05.com.microsoft:gbdc01exver01.dla.com

Sun Sep 21 20:05:04 BST : LUN /vol/SnapDrive_gbdc01exmbf01db_clone_of_exchsnap__gbdc01exmbv01_09212008_190000__weekly_snapshot_0/sg1db.lun has been taken offline

Sun Sep 21 20:05:08 BST : Volume 'SnapDrive_gbdc01exmbf01db_clone_of_exchsnap__gbdc01exmbv01_09212008_190000__weekly_snapshot_0' has been set temporarily offline

Re: Volumes left behind after verification completes

BrendonHiggins — Mon, 17 Nov 2008 15:00:20 GMT

Think I have found the solution.

I was getting alot of Plug & Play errors in the System Log Event, Event ID 257, see Microsoft Article with Hotfix:

http://support.microsoft.com/kb/924390

Will try this weekend and report back.

Re: Volumes left behind after verification completes

ianaforbes — Thu, 04 Dec 2008 16:15:15 GMT

Did you ever find a resolution for this? It looks to be a bug because there are many people running into the sdame issue.

Re: Volumes left behind after verification completes

BrendonHiggins — Thu, 04 Dec 2008 17:18:07 GMT

Long story short, no. Hotfix was not the solution and filer CPU @ 100% was not the cause. Still looking for the answer.

Bren

Re: Volumes left behind after verification completes

sflynn — Thu, 04 Dec 2008 20:04:17 GMT

Hey Ian/Brendon...

Was there a case opened with NGS for this issue? I'm going to bring this up with the engineers and see what we can figure out. But a case number would be good for them to get more info on what's happening.

Thanks...

Shannon

Re: Volumes left behind after verification completes

BrendonHiggins — Thu, 04 Dec 2008 20:45:53 GMT

I have 3 different case numbers from the last 12 months. Out of the office this week but should be able to dig them out next week.

Re: Volumes left behind after verification completes

cdlovejoy — Tue, 16 Dec 2008 23:48:55 GMT

Is there any resolution on this yet? Also, I was wondering if it was a 64bit issue. We are experiencing the same issue and it's quite frustrating.

Thanks,

Carl.

Re: Volumes left behind after verification completes

sflynn — Tue, 16 Dec 2008 23:56:06 GMT

64bit shouldn't have anything to do with volumes on the storage being left behind.

As soon as I get a case number that I can reference, I can do more investigating and see what I can figure out.

Thanks...

Shannon

Re: Volumes left behind after verification completes

BrendonHiggins — Wed, 17 Dec 2008 11:54:52 GMT

Sorry been off playing with my new VMWare toys.

NetApp LOG # 2494469

NETAPP LOG # 2687346

NETAPP LOG # 2705972

NETAPP LOG # 2962044

I can not find the original ticket but the above give a good insight into what has been going on.

Re: Volumes left behind after verification completes

cdlovejoy — Wed, 17 Dec 2008 17:18:58 GMT

I just opened a case 2 days ago so I'll let the tech assigned work on this before I go public. I don't want to think I do not have faith in the engineers. I was just curious if anyone had seen this prior and if so what the thoughts were. The reason I wondered about 64 bit is we have an identical server other than it's 32 bit but all OS/SQL/Netapp products are the same version level and we are not having issues with our 32 bit machine. As soon as I hear from the tech assigned to my case I'll post the solution/response.

Thanks,

Carl.

Re: Volumes left behind after verification completes

BrendonHiggins — Wed, 24 Dec 2008 14:11:39 GMT

Just applied this hotfix. Will report back on the results.

http://now.netapp.com/NOW/cgi-bin/bol?Type=Detail&Display=223992

Merry Christmas all

Re: Volumes left behind after verification completes

cdlovejoy — Wed, 24 Dec 2008 16:28:15 GMT

I'm sorry to report that I have already installed that hotfix. It did nothing for this issue on my box.

Thanks,

Carl.

Merry Christmas

Re: Volumes left behind after verification completes

stuartwatkins — Fri, 09 Jan 2009 08:31:51 GMT

Anti virus? Do you have anti virus that is chewing on this and causing any problems? I have heard of this before

Worth a shot, remove any AV see if it makes a difference?

Re: Volumes left behind after verification completes

BrendonHiggins — Fri, 09 Jan 2009 08:36:22 GMT

I can not believe that you would post my solution to my thread as your own work. You forum troll and points whor. But you a quite correct it was the AV which was casing the problem and you did make the coffee this morning so have some points.

Bren

Re: Volumes left behind after verification completes

ianaforbes — Fri, 09 Jan 2009 12:04:49 GMT

If antivirus is the issue was removing antivirus the solution? What if we need A/V on the server? Ia there a workaround?

Re: Volumes left behind after verification completes

BrendonHiggins — Fri, 09 Jan 2009 14:11:16 GMT

That is just what our risk team said. "There MUST be av on the server..."

Answer - Only run white list apps on the server.

http://www.mcafee.com/uk/enterprise/products/host_intrusion_prevention/host_intrusion_prevention_server.html

Will post back if we have any problems but early results are good.

Re: Volumes left behind after verification completes

cdlovejoy — Fri, 09 Jan 2009 14:20:25 GMT

I'm glad you solved your problem, however, our issue still exists and we have no AV software installed. In fact, the only thing on the box itself is the OS and SQL 2005. No other software is present. We have system hardened the box but only via known OS configuration changes. This problem has been escalated and I have been working with Netapp for 2 days on this. We worked on it quite a bit yesterday and I'm hoping a solution is on the horizon.

Thanks,

Carl.

Re: Volumes left behind after verification completes

BrendonHiggins — Fri, 09 Jan 2009 14:27:57 GMT

Have you tried to create a 2nd verification server on a vinila install? We have five different sql verification and none of them give us problems, it was only ever SME. So SMSQL should work. When the verification failes does snapdrive show the internal SCSI HDD as a veritual disk? See pic.