Network and Storage Protocols
Network and Storage Protocols
I am running Data Ontap v8.0.1 7 Mode on an IBM nSeries 6040 A20. I have a CIFS share created and can see this from numerous systems within my environment. However, on several Server 2008 R2 installations I am unable to connect to the shares (using either a run command or mapping the drive). I can ping the relevant controller without issue - either by name or IP address. I have turned off Windows Firewall in case this was causing an issue. Can anyone help me troubleshoot this issue as I am running out of ideas as to what is causing the problem. Any questions please let me know
Jon
you probably need to look at the security settings on the 2008r2 server. I have many '08r2 servers connecting to CIFS shares on DOT 7.3.3.
I'd start checking, under Group Policy\computer config\Policies\Windows Settings\Security Settins\Local Policies
The first couple I'd look at would be FIPS and LM authentication levels.
Hi,
You could start off by gathering more info by looking in /etc/messages for error messages. You could also turn on more verbose cifs error messages first by turning
this option on
options cifs.trace_login on
Then try to map share and see if you get more info. If you got more info post it here. Read up on support here:
http://now.netapp.com/NOW/knowledge/docs/olio/guides/winCIFS.shtml
Have you enabled smb2? What type of domain is this? <cifs domaininfo> will give us this. remove any private info please.
Cheers,
Eric
PS: you could also log a case with tech support and they their view on this.
Message was edited by: eric.barlier
Hi Eric (and everyone else who answered),
I took a look in the /etc/messages and even with verbose logging on spotted nothing noteworthy. We have then enabled SMB2 (which we may leave on anyway) but this made no difference. Our Domain is a Windows 2000 AD (soon to be upgraded).
In response to an earlier post I looked at the FIPS setting and this was fine. Is there anything specific with LM that I should be looking for?
Thanks again for all your help and apologies if the response is basic, am new to the world of NetApp. Unfortunately as I am running an nSeries I did not have access to the link you sent. Wish NetApp and IBM would sort this out as the simulator would be good too.
Jon
It's probably the SMB signing requirement on 2008.
options cifs.signing.enable on
Hi we had this very same problem with a customer this week and its was solved by turning on SMB as stated above. The same customer had standard win 2008 servers with no issue but the R2 servers failed. They also had SnapDrive issues when using RPC to connect to the filer till SMB was enabled then all was well
Good Luck
Mick
I came across this because we've been having a somewhat related issue, I actually posted in February with specific errors whereby we were getting ACCESS_DENIED and user/group mismatches (the filer was unable to lookup the GUID of domain users, therefore they weren't able to access CIFS shares), but received no response. Our cifs "testdc" command is failing to communicate with the domain controllers when we have our Cisco WAN acceleration enabled and the NetApp smb signing defaults (whereby SMB signing is off). After two months of back and forth Cisco came back saying it is because SMB signing is enabled and we should disable it across the board. This of course is contrary to what Microsoft does by default on domain controllers so the lack of documentation anywhere is a bit disconcerting... It's a bit of a catch 22.
To the original poster, I would ask if the 2008 servers you're referring to also happen to be domain controllers? MSFT makes a distinction in the policy for domain controllers vs non. By default on a Windows 2003 or 2008 domain controller, not only is SMB signing enabled, it is required of a 2003/2008 SMB *server* that's a DC (Microsoft KB: http://support.microsoft.com/kb/887429 has a list of defaults though isn't current with 2008 info). Specifically, the group policy object "Microsoft Network Server: Digitally sign communications (always)" is defined as enabled whereas in a regular 2003/2008 file server it is disabled. the relevance that this is the SMB server setting (not "client") is important though. If he is just trying to map a share from a server, technically the server is the "client" and the NetApp server is the "server" so none of this SMB policy matters, right? I wonder if the communications failure is actually happening in the negotiations process prior to it even getting to the point of the NetApp acting as the server, but more so a failure before that with the AD aspect, whereby the NetApp is failing as a client communicating with the DC as a server.
Now the NetApp side of signing is a bit convuluted... By default, older versions than DataOnTap 7.1 had signing capability allowed, but by default, on NetApp file servers running DataOnTap 7.1. or newer, SMB is not even signing capable, ever, without explicitly changing the setting. Therefore, with the changes after DataOnTap 7.1, you actually HAVE to enable SMB signing (not the default) to communicate with domain controllers or you will apparently experience the access_denied / GUID mapping issues randomly. Cisco's recommendation apparently is to disable signing on the domain controllers and elsewhere and override concept of Microsoft's default security. NetApp, as far as I can tell feels the same way, but who knows since they sort of tucked this change away in some release notes and other than a few tidbits in a KB on the signing options there's not much information on it. NetApp's documentation of the SMB signing options could be better from what we've found, http://now.netapp.com/NOW/knowledge/docs/ontap/rel733/html/ontap/filesag/GUID-0C291FD0-68D3-4DAE-A493-9958EA4C70DC.html is probably the only relevant article to the matter but it doesn't go into great detail. There really should be a "best practices" somewhere for such an apparently important topic that's relevant and requires the attention of anyone running Active Directory. Take from NetApp's KB on SMB signing:
It is not possible to configure the storage system to require SMB signing communications from clients, which is the equivalent of the Microsoft Network server policy "Digitally sign communications (always)." For performance reasons, SMB signing is disabled by default on the storage system.
... if NetApp says you can't require SMB signing from clients, what is the "options cifs.smb2.signing.required" option going to do if it is set to "on"? Are you saying that options cifs.smb2.signing.required on does not really mean the NetApp server is going to require SMB signing? Or is this setting referring to when the SMB2 session is acting as a client, not a server?
Cisco's recommendation to turn off SMB signing is a bit puzzling in that they're recommending to ignore known security issues in favor of performance, but it is worth noting the performance issues. If you do have SMB signing enabled there's no ability for WAN optimization. Repeat: SMB signed traffic is not accelerated on Cisco's appliance or any others. So if you go with the defaults from Microsoft, essentially your Cisco appliance is not being used for DC traffic (and if your DC also happens to be a file server, your file server traffic isn't being optimized). On top of that hit you're also taking a 15% hardware performance hit on both the server and client side (256 bit encryption overhead), regardless of if you're accelerating your traffic or not. By default Windows clients are set to have "Microsoft network client: digitally sign communications (if server agrees)" enabled so sign they will. So if it's enabled on NetApp then any Win Vista/7 clients will always be communicating with SMB signing on (and thus the performance hit, and no acceleration if you do that).
NetApp seems to be missing a piece of the puzzle in the signing world. A MSFT Windows server has the default configuration "Microsoft network server: Digitally sign communications (always)" disabled and "microsoft network client: digitally sign communications (if server agrees)" enabled, but in NetApp's world they're consolidating the "server" and "client" options into one, thereby requiring it for all communications*. For Microsoft servers that is not the case because the server has the ability to act as a signing client but doesn't force signing when it itself is the SMB server.
The real problem here is the signing settings for NetApp seem to be on or off at the client/server level, there is no in between. My guess as to why the options were consolidated is in general the NetApp device is never acting as an SMB client, only an SMB server. Except in the case of domain traffic this isn't true? Nobody at NetApp has been really able to help explain what exactly the "Cifs testdc" command line is actually doing as far as network traffic. Is it initiating an SMB client session? Who knows. It is indeed odd though that I can get it to fail by turning on acceleration but the command works fine with no acceleration enabled. Cisco can't really explain this, and NetApp's lack of any documentation on what the "testdc" command line does doesn't help.
If a domain controller is requiring it as an SMB server and you want your NetApp filer to talk with it, it seems like you have no other choice than to enable it across the board. By "enable it" I mean allow it in the first place, the NetApp setting isn't forcing anything, but if the Windows clients will then always be using signing.
Maybe it's time to develop an SMB3 whereby Cisco, Microsoft, NetApp, whomever, can all work together and make everything compatible somehow, or at least have a little more granularity, ehh?
Another possible consideration to have with this whole debate is whether or not you have SMB2 enabled. It doesn't directly matter to the signing vs not argument, but by enabling SMB2 you do get a few more signing options and thus a few more variables, namely the above mentioned smb2.signing.required option. By default SMB2 is not enabled on NetApp. Windows Vista / 7 clients use SMB2 preferred by default, so if NetApp has it on they will be using SMB2. (Unrelated: We've recently started enabling this because we've been having file lock issues on Win7 clients that this seems to resolve by enabling. Users were getting messages that files were already open when they weren't--SMB2 improves upon the client's ability to keep files open over a network connection that might experience a hiccup now and then. When you enable SMB2 options, your Win Vista/7 clients will use SMB2 instead and the file handle / locks seem to work better for these clients).
So the long and short of it... What really is NetApp's recommended practice? It would be nice if someone official could make an official statement of best practice. It would be even nicer if there were an option to enable signing when the the client or server requires it, but not otherwise (Windows 7 machiens don't require it for instance, by default, but if NetApp has it on they will be using Signing). Or perhaps more another way of stating that would be it would be if there were an option to enable signing when the netapp storage is acting as an SMB client but disable signing if it's acting as a server. In this scenario communications to DCs wouldn't be affected and storage SMB server traffic wouldn't be signed.
And last, but not least, please make sure to take note of the yet unfixed DataOnTap bug ID 413029 before enabling/disabling SMB signing. I was on the phone with support and they assured me I could enable SMB signing without a restart, but as it turns out you can't because there's a bug in all verisons of DataOnTap whereby it will prevent access from clients until you restart CIFS. In my case this happened to involve all clients on Windows XP (SMB2 clients-->Win7 users were fine) had to reboot or alternatively I would've had to restart CIFS killing all sessions. http://now.netapp.com/NOW/cgi-bin/bol?Type=Detail&Display=413029
Are you saying that options cifs.smb2.signing.required on does not really mean the NetApp server is going to require SMB signing? Or is this setting referring to when the SMB2 session is acting as a client, not a server?
According to manual page and tr-3740 it does force signing at least when NetApp is server. Quoting tr-3740:
SMB 2.0 signing is never disabled as seen in CIFS/SMB; the possible configurations are either “required” or “not required.”
It is still not clear what happens when NetApp acts as client; the only documented option related to this is cifs.smb2.client.enable.
It would be even nicer if there were an option to enable signing when the the client or server requires it, but not otherwise
As far as I can tell, this is what happens with SMB2 by default ... again at least on server side.
aborzenkov wrote:
Are you saying that options cifs.smb2.signing.required on does not really mean the NetApp server is going to require SMB signing? Or is this setting referring to when the SMB2 session is acting as a client, not a server?
According to manual page and tr-3740 it does force signing at least when NetApp is server. Quoting tr-3740:
SMB 2.0 signing is never disabled as seen in CIFS/SMB; the possible configurations are either “required” or “not required.”
It is still not clear what happens when NetApp acts as client; the only documented option related to this is cifs.smb2.client.enable.
It would be even nicer if there were an option to enable signing when the the client or server requires it, but not otherwise
As far as I can tell, this is what happens with SMB2 by default ... again at least on server side.
so are we in agreement their documentation is inconsistant(?)
regarding what happens with SMB2 by default... SMB2 is by default disabled, and by default once enabled signing is still set to not required. But it's interesting you point that out... It doesn't seem to be clear in their documentaiton, but if we're to believe the TR-3740 technical report you reference, the SMB2 configuration just outright ignores the options.cifs.signing enabled flag?
I believe you've cracked an important piece of the missing puzzle here.. If SMB2 enables signing irregardless of the cifs.signing.enable flag then this would explain why we have cifs.signing.enabled OFF but the domain controllers (which require signing) are still able to communicate with the filer.
Is there any way to determine which SMB2 sessions are using signing, which aren't? I can see which connections are using SMB vs SMB2 by doing cifs sessions -p SMB or cifs sessions -p SMB2 but no details on signing there. Is it hidden away anywhere? Also that TR explains that the SMB2 client option is used for connecting to domain controllers, but what actions specifically can trigger a connection for DC via SMB/CIFS? I'm listing out my sessions w/ -p and obviously don't ever see connections to any DCs. I can run the testdc command and that might trigger it, but you can't really run testdc and list out cifs -p at the same time because the console is tied up, and accessing the 'sessions' from computer management and hitting refresh isn't showing anything there either while testdc is running.
Thanks for the TR reference, that's a pretty helpful document in general.