Re: CIFS Performance

DATACENTER_NEO · ‎2013-06-03

Hello,

i have a CIFS on netapp 2240, and this throughput is only ~40MB/s on 10Gb network, when my client is windows, and when my client is linux the throughput is ~120MB/s.

If i configure a share on windows 2012 my throughput is ~130MB/s with clienta windows and linux.

My problem is betwen client windows and netapp CIFS. I configure many options on netapp but the performance is always the same.

my options configurations is:

cifs.LMCompatibilityLevel 1

cifs.W2K_password_change off

cifs.W2K_password_change_interval 4w

cifs.W2K_password_change_within 3600h

cifs.audit.account_mgmt_events.enable on

cifs.audit.autosave.file.extension

cifs.audit.autosave.file.limit 0

cifs.audit.autosave.onsize.enable off

cifs.audit.autosave.onsize.threshold 75%

cifs.audit.autosave.ontime.enable off

cifs.audit.autosave.ontime.interval 1d

cifs.audit.enable off

cifs.audit.file_access_events.enable on

cifs.audit.liveview.allowed_users

cifs.audit.liveview.enable off

cifs.audit.logon_events.enable on

cifs.audit.logsize 31457280

cifs.audit.nfs.enable off

cifs.audit.nfs.filter.filename

cifs.audit.saveas /etc/log/adtlog.evt

cifs.bypass_traverse_checking on

cifs.client.dup-detection ip-address

cifs.comment

cifs.enable_share_browsing on

cifs.gpo.enable off

cifs.gpo.trace.enable off

cifs.grant_implicit_exe_perms off

cifs.guest_account

cifs.home_dir_namestyle

cifs.home_dirs_public_for_admin on

cifs.idle_timeout 28800

cifs.ipv6.enable off

cifs.max_mpx 1124

cifs.ms_snapshot_mode xp

cifs.neg_buf_size 65535

cifs.netbios_aliases

cifs.netbios_over_tcp.enable on

cifs.nfs_root_ignore_acl off

cifs.oplocks.enable on

cifs.oplocks.opendelta 8

cifs.per_client_stats.enable off

cifs.perfmon.allowed_users

cifs.perm_check_ro_del_ok off

cifs.perm_check_use_gid on

cifs.preserve_unix_security off

cifs.restrict_anonymous 0

cifs.restrict_anonymous.enable off

cifs.save_case on

cifs.scopeid

cifs.search_domains

cifs.show_dotfiles on

cifs.show_snapshot off

cifs.shutdown_msg_level 2

cifs.sidcache.enable on

cifs.sidcache.lifetime 1440

cifs.signing.enable off

cifs.smb2.enable on

cifs.smb2.signing.required on

cifs.smb2_1.branch_cache.enable on

cifs.smb2_1.branch_cache.hash_time_out 3600 (value might be overwritten in takeover)

cifs.snapshot_file_folding.enable off

cifs.symlinks.cycleguard on

cifs.symlinks.enable on

cifs.tcp_window_size 2096560

cifs.trace_dc_connection off

cifs.trace_login off

cifs.universal_nested_groups.enable on

cifs.widelink.ttl 10m

can i do more something to emproove this (client side or server side)?

Thanks,

rvalappi · ‎2013-06-03

there are two ways. .

1. get a netapp performance engineer to review our perfstat

2. find cifs performance

a. get the cifs stat output

b. check for cifs domaininfo

c. round trip to your dc

d. roud trip to your vscan or fpolicy server

3. what is the hostname/SN number of your system. is it sending ASUP ?

DATACENTER_NEO · ‎2013-06-04

Rajesh,

2A

Neonetapp1> cifs stat
                        reject   202171 0%
                         mkdir        4 0%
                         rmdir 2063990 0%
                          open        0 0%
                        create        0 0%
                         close 156788069 7%
                       X&close        0 0%
                         flush 4714718 0%
                       X&flush        0 0%
                        delete   967422 0%
                        rename 1078304 0%
                      NTRename        0 0%
                        getatr       74 0%
                        setatr        1 0%
                          read        0 0%
                        X&read        0 0%
                         write   107491 0%
                       X&write        0 0%
                          lock        0 0%
                        unlock        0 0%
                         mknew        0 0%
                        chkpth       30 0%
                          exit        0 0%
                         lseek        0 0%
                      lockread        0 0%
                    X&lockread        0 0%
                   writeunlock        0 0%
                      readbraw        0 0%
                     writebraw        0 0%
                        writec        0 0%
                     gettattre        0 0%
                     settattre        0 0%
                      lockingX 9433682 0%
                           IPC 1623831 0%
                         open2        0 0%
                   find_first2 120710911 5%
                    find_next2 2007396 0%
                 query_fs_info 59659578 3%
               query_path_info 714300930 31%
                 set_path_info      199 0%
               query_file_info 240282407 11%
                 set_file_info 70010663 3%
                   create_dir2        0 0%
                  Dfs_referral   441337 0%
                    Dfs_report        0 0%
                          echo 2822447 0%
                    writeclose        0 0%
                         openX       99 0%
                         readX 277912817 12%
                        writeX 397157289 17%
                     findclose        0 0%
                          tcon        0 0%
                          tdis 4011547 0%
                       negprot    58745 0%
                         login 1383756 0%
                        logout   211983 0%
                         tconX 4109959 0%
                       dskattr        0 0%
                        search        0 0%
                        fclose    60456 0%
                     NTCreateX 178153608 8%
                 NTTransCreate       23 0%
                  NTTransIoctl 26058437 1%
                 NTTransNotify 2275931 0%
                 NTTransSetSec 1903767 0%
               NTTransQuerySec 4062159 0%
              NTNamedPipeMulti        0 0%
                   NTCancel CN 1297937 0%
                NTCancel Other      275 0%
                      SMB2Echo        0 0%
                   SMB2Negprot        0 0%
              SMB2TreeConnnect        0 0%
            SMB2TreeDisconnect        0 0%
                     SMB2Login        0 0%
                    SMB2Create        0 0%
                      SMB2Read        0 0%
                     SMB2Write        0 0%
                      SMB2Lock        0 0%
                    SMB2Unlock        0 0%
                SMB2OplkBrkAck        0 0%
                    SMB2ChgNfy        0 0%
                     SMB2CLose        0 0%
                     SMB2Flush        0 0%
                    SMB2Logout        0 0%
                    SMB2Cancel        0 0%
                 SMB2IPCCreate        0 0%
                   SMB2IPCRead        0 0%
                  SMB2IPCWrite        0 0%
                  SMB2QueryDir        0 0%
        SMB2QueryFileBasicInfo        0 0%
         SMB2QueryFileStndInfo        0 0%
          SMB2QueryFileIntInfo        0 0%
           SMB2QueryFileEAInfo        0 0%
          SMB2QueryFileFEAInfo        0 0%
         SMB2QueryFileModeInfo        0 0%
          SMB2QueryAltNameInfo        0 0%
       SMB2QueryFileStreamInfo        0 0%
          SMB2QueryNetOpenInfo        0 0%
          SMB2QueryAttrTagInfo        0 0%
           SMB2QueryAccessInfo        0 0%
      SMB2QueryFileUnsupported        0 0%
          SMB2QueryFileInvalid        0 0%
            SMB2QueryFSVolInfo        0 0%
           SMB2QueryFSSizeInfo        0 0%
            SMB2QueryFSDevInfo        0 0%
           SMB2QueryFSAttrInfo        0 0%
         SMB2QueryFSFullSzInfo        0 0%
          SMB2QueryFSObjIdInfo        0 0%
            SMB2QueryFSInvalid        0 0%
         SMB2QuerySecurityInfo        0 0%
              SMB2SetBasicInfo        0 0%
             SMB2SetRenameInfo        0 0%
           SMB2SetFileLinkInfo        0 0%
           SMB2SetFileDispInfo        0 0%
             SMB2SetFullEAInfo        0 0%
               SMB2SetModeInfo        0 0%
              SMB2SetAllocInfo        0 0%
                SMB2SetEOFInfo        0 0%
            SMB2SetUnsupported        0 0%
            SMB2SetInfoInvalid        0 0%
           SMB2SetSecurityInfo        0 0%
       SMB2FsctlPipeTransceive        0 0%
             SMB2FsctlPipePeek        0 0%
        SMB2FsctlEnumSnapshots        0 0%
         SMB2FsctlDfsReferrals        0 0%
            SMB2FsctlSetSparse        0 0%
          SMB2FsctlSecureShare        0 0%
      SMB2FsctlFileUnsupported        0 0%
       SMB2FsctlIpcUnsupported        0 0%
                   cancel lock        0
                     wait lock        0
                 copy to align   452396
                  alignedSmall   639906
                  alignedLarge     9701
               alignedSmallRel        0
               alignedLargeRel        0
                 FidHashAllocs   235182
                 TidHashAllocs      943
                 UidHashAllocs       85
                      mbufWait        0
                       nbtWait        0
                      pBlkWait        0
              BackToBackCPWait        0
                       cwaWait        0
             short msg prevent     4746
                   multipleVCs   190162
                   SMB signing        0
              mapped null user        0
                    PDCupcalls        0
                     nosupport        0
                read pipe busy        0
               write pipe busy        0
               trans pipe busy        0
              read pipe broken        0
             write pipe broken        0
             trans pipe broken        0
               queued writeraw        0
                nbt disconnect    49359
                smb disconnect    20582
                dup disconnect      298
            OpLkBkXorBatchToL2 1063853
          OpLkBkXorBatchToNone        5
                OpLkBkL2ToNone   214639
              OpLkBkNoBreakAck       18
            OpLkBkNoBreakAck95        0
            OpLkBkNoBreakAckNT       18
              OpLkBkIgnoredAck     1056
          OpLkBkWaiterTimedOut        0
            OpLkBkDelayedBreak    26181
           SharingErrorRetries   748212
                  FoldAttempts        0
                   FoldRenames        0
            FoldRenameFailures        0
                 FoldOverflows        0
                FoldDuplicates        0
               FoldWAFLTooBusy        0
               NoAllocCredStat        0
             RetryRPCcollision        0
                  TconCloseTID        0
               GetNTAPExtAttrs        0
               SetNTAPExtAttrs        0
                    SearchBusy        0
                ChgNfyNoMemory        0
                ChgNfyNewWatch   913039
               ChgNfyLastWatch   912976
             UsedMIDTblCreated        0
           UnusedMIDTblCreated        0
             InvalidMIDRejects        0
          SMB2InvalidSignature        0
     SMB2DurableCreateReceived        0
    SMB2DurableCreateSucceeded        0
    SMB2DurableReclaimReceived        0
   SMB2DurableReclaimSucceeded        0
    SMB2DurableHandlePreserved        0
       SMB2DurableHandlePurged        0
      SMB2DurableHandleExpired        0
               SMB2FileDirInfo        0
           SMB2FileFullDirInfo        0
         SMB2FileIdFullDirInfo        0
           SMB2FileBothDirInfo        0
         SMB2FileIdBothDirInfo        0
             SMB2FileNamesInfo        0
        SMB2FileDirUnsupported        0
                 SMB2QueryInfo        0
                   SMB2SetInfo        0
                     SMB2Ioctl        0
        SMB2RelatedCompRequest        0
      SMB2UnRelatedCompRequest        0
               SMB2FileRequest        0
               SMB2PipeRequest        0
                 SMB2nosupport        0
Max Multiplex = 49, Max pBlk Exhaust = 0, Max pBlk Reserve Exhaust = 0
Max FIDs = 3177, Max FIDs on one tree = 2550
Max Searches on one tree = 9, Max Core Searches on one tree = 0
Max sessions = 32
Max trees = 413
Max shares = 20
Max session UIDs = 27, Max session TIDs = 161
Max locks = 3331
Max credentials = 85
Max group SIDs per credential = 20
Max pBlks = 896 Current pBlks = 896 Num Logons = 0
Max reserved pBlks = 32 Current reserved pBlks = 32
Max gAuthQueue depth         = 3
Max gSMBBlockingQueue depth = 10
Max gSMBTimerQueue depth     = 4
Max gSMBAlfQueue depth       = 2
Max gSMBRPCWorkerQueue depth = 10
Max gOffloadQueue depth      = 2
Local groups: builtins = 6, user-defined = 1, SIDs = 5
RPC group count = 20, RPC group active count = 0
Max Watched Directories = 157, Current Watched Directories = 108
Max Pending ChangeNotify Requests = 157, Current Pending ChangeNotify Requests = 108
Max Pending DeleteOnClose Requests = 2688, Current Pending DeleteOnClose Requests = 0

2B-

Neonetapp1> cifs domaininfo
NetBios Domain:           NEOIDC
Windows 2003 Domain Name: neoidc.uolhostidc
Type:                     Windows 2003
Filer AD Site:            Default-First-Site-Name

Current Connected DCs:    \\NEO-SRV-AD001
Total DC addresses found: 4
Preferred Addresses:
                          172.30.70.138                    BDC
Favored Addresses:
                          200.98.238.139 NEO-SRV-AD002    PDC
                          200.98.238.138                   PDC
Other Addresses:
                          172.30.70.139                    PDC

Connected AD LDAP Server: \\NEO-SRV-AD001.neoidc.uolhostidc
Preferred Addresses:
172.30.70.138

Favored Addresses:
                          200.98.238.139
                           neo-srv-ad002.neoidc.uolhostidc
                          200.98.238.138
                           neo-srv-ad001.neoidc.uolhostidc
Other Addresses:
                          None

2C-

Neonetapp1> ping -s 172.30.70.138
64 bytes from 172.30.70.138 (172.30.70.138): icmp_seq=0 ttl=128 time=0.400 ms
64 bytes from 172.30.70.138 (172.30.70.138): icmp_seq=1 ttl=128 time=0.431 ms
64 bytes from 172.30.70.138 (172.30.70.138): icmp_seq=2 ttl=128 time=0.850 ms
64 bytes from 172.30.70.138 (172.30.70.138): icmp_seq=3 ttl=128 time=0.363 ms
64 bytes from 172.30.70.138 (172.30.70.138): icmp_seq=4 ttl=128 time=0.326 ms
64 bytes from 172.30.70.138 (172.30.70.138): icmp_seq=5 ttl=128 time=0.320 ms

--- 172.30.70.138 ping statistics ---
6 packets transmitted, 6 packets received, 0% packet loss
round-trip min/avg/max = 0.320/0.448/0.850 ms
===============================================================================
Neonetapp1> ping -s 200.98.238.139
64 bytes from 200.98.238.139 (200.98.238.139): icmp_seq=0 ttl=127 time=0.381 ms
64 bytes from 200.98.238.139 (200.98.238.139): icmp_seq=1 ttl=127 time=0.389 ms
64 bytes from 200.98.238.139 (200.98.238.139): icmp_seq=2 ttl=127 time=0.412 ms
64 bytes from 200.98.238.139 (200.98.238.139): icmp_seq=3 ttl=127 time=0.488 ms
64 bytes from 200.98.238.139 (200.98.238.139): icmp_seq=4 ttl=127 time=0.430 ms

--- 200.98.238.139 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 0.381/0.420/0.488 ms
===============================================================================
Neonetapp1> ping -s 200.98.238.138
64 bytes from 200.98.238.138 (200.98.238.138): icmp_seq=0 ttl=127 time=0.805 ms
64 bytes from 200.98.238.138 (200.98.238.138): icmp_seq=1 ttl=127 time=0.876 ms
64 bytes from 200.98.238.138 (200.98.238.138): icmp_seq=2 ttl=127 time=1.559 ms
64 bytes from 200.98.238.138 (200.98.238.138): icmp_seq=3 ttl=127 time=0.400 ms
64 bytes from 200.98.238.138 (200.98.238.138): icmp_seq=4 ttl=127 time=0.878 ms
64 bytes from 200.98.238.138 (200.98.238.138): icmp_seq=5 ttl=127 time=1.795 ms

--- 200.98.238.138 ping statistics ---
6 packets transmitted, 6 packets received, 0% packet loss
round-trip min/avg/max = 0.400/1.052/1.795 ms

2D-

Neonetapp1> vscan

Virus scanning is disabled.

No vscan servers are connected.

List of extensions to scan:
Extensions-to-scan list is empty.

List of extensions not to scan:
Extensions-not-to-scan list is empty.

Number of files scanned: 0
Number of scan failures: 0
Number of throttled requests: 0

3-

Neonetapp1> sysconfig -a

NetApp Release 7.3.4: Thu May 27 17:52:48 PDT 2010

System ID: 0135077937 (Neonetapp1); partner ID: 0135077939 (Neonetapp2)

System Serial Number: 600000000755 (Neonetapp1)

The network interface is 1Gb and not 10Gb and my appliance is FAS 2020 and not 2240.

Thanks

shaunjurr · ‎2013-06-04

Hi,

The CIFS/SMB protocoll is not very performant. If you have newer clients than win2k3 and XP, SMB2 will give you much better performance. I see, however, that you aren't using it. Enable it on the filer CLI with 'options cifs.smb2.enable on' . Do take some time to read the documentation as well. Many of your questions will be answered there with much better authority than you will get on Communities.

You might also want to try updating to a much newer 7.3.x release to take advantage of some performance enhancements in ONTap.

S.

DATACENTER_NEO · ‎2013-06-04

Ok,

but on other netapp i have the same problem, on ontap is 8.x, interface is 10Gb (FAS2240) and smb 2 enable.

can i sent the same informations for your analysis ?

Thanks

DATACENTER_NEO · ‎2013-06-04

In this case i have this configurations:

Options

=========================================================

Netapp01> options cifs

cifs.LMCompatibilityLevel 1

cifs.W2K_password_change off

cifs.W2K_password_change_interval 4w

cifs.W2K_password_change_within 3600h

cifs.audit.account_mgmt_events.enable on

cifs.audit.autosave.file.extension

cifs.audit.autosave.file.limit 0

cifs.audit.autosave.onsize.enable off

cifs.audit.autosave.onsize.threshold 75%

cifs.audit.autosave.ontime.enable off

cifs.audit.autosave.ontime.interval 1d

cifs.audit.enable off

cifs.audit.file_access_events.enable on

cifs.audit.liveview.allowed_users

cifs.audit.liveview.enable off

cifs.audit.logon_events.enable on

cifs.audit.logsize 31457280

cifs.audit.nfs.enable off

cifs.audit.nfs.filter.filename

cifs.audit.saveas /etc/log/adtlog.evt

cifs.bypass_traverse_checking on

cifs.client.dup-detection ip-address

cifs.comment

cifs.enable_share_browsing on

cifs.gpo.enable off

cifs.gpo.trace.enable off

cifs.grant_implicit_exe_perms off

cifs.guest_account

cifs.home_dir_namestyle

cifs.home_dirs_public_for_admin on

cifs.idle_timeout 28800

cifs.ipv6.enable off

cifs.max_mpx 1124

cifs.ms_snapshot_mode xp

cifs.neg_buf_size 65535

cifs.netbios_aliases

cifs.netbios_over_tcp.enable on

cifs.nfs_root_ignore_acl off

cifs.oplocks.enable on

cifs.oplocks.opendelta 8

cifs.per_client_stats.enable off

cifs.perfmon.allowed_users

cifs.perm_check_ro_del_ok off

cifs.perm_check_use_gid on

cifs.preserve_unix_security off

cifs.restrict_anonymous 0

cifs.restrict_anonymous.enable off

cifs.save_case on

cifs.scopeid

cifs.search_domains

cifs.show_dotfiles on

cifs.show_snapshot off

cifs.shutdown_msg_level 2

cifs.sidcache.enable on

cifs.sidcache.lifetime 1440

cifs.signing.enable off

cifs.smb2.enable on

cifs.smb2.signing.required on

cifs.smb2_1.branch_cache.enable on

cifs.smb2_1.branch_cache.hash_time_out 3600 (value might be overwritten in takeover)

cifs.snapshot_file_folding.enable off

cifs.symlinks.cycleguard on

cifs.symlinks.enable on

cifs.tcp_window_size 2096560

cifs.trace_dc_connection off

cifs.trace_login off

cifs.universal_nested_groups.enable on

cifs.widelink.ttl 10m

cifs.wins_servers 192.168.200.6,192.168.200.5

=========================================================

Netapp01> cifs stat

reject 0 0%

mkdir 0 0%

rmdir 0 0%

open 0 0%

create 0 0%

close 436 0%

X&close 0 0%

flush 0 0%

X&flush 0 0%

delete 53 0%

rename 5 0%

NTRename 0 0%

getatr 0 0%

setatr 0 0%

read 0 0%

X&read 0 0%

write 0 0%

X&write 0 0%

lock 0 0%

unlock 0 0%

mknew 0 0%

chkpth 0 0%

exit 0 0%

lseek 0 0%

lockread 0 0%

X&lockread 0 0%

writeunlock 0 0%

readbraw 0 0%

writebraw 0 0%

writec 0 0%

gettattre 0 0%

settattre 0 0%

lockingX 72 0%

IPC 67 0%

open2 0 0%

find_first2 1280 0%

find_next2 27 0%

query_fs_info 91 0%

query_path_info 2303 0%

set_path_info 2 0%

query_file_info 141 0%

set_file_info 170 0%

create_dir2 0 0%

Dfs_referral 50 0%

Dfs_report 0 0%

echo 19378 0%

writeclose 0 0%

openX 0 0%

readX 101608 1%

writeX 5086492 42%

findclose 0 0%

tcon 0 0%

tdis 2260 0%

negprot 690 0%

login 215 0%

logout 43 0%

tconX 64 0%

dskattr 0 0%

search 0 0%

fclose 0 0%

NTCreateX 1881 0%

NTTransCreate 0 0%

NTTransIoctl 156 0%

NTTransNotify 310 0%

NTTransSetSec 0 0%

NTTransQuerySec 0 0%

NTNamedPipeMulti 0 0%

NTCancel CN 14 0%

NTCancel Other 0 0%

SMB2Echo 0 0%

SMB2Negprot 290 0%

SMB2TreeConnnect 2371 0%

SMB2TreeDisconnect 2213 0%

SMB2Login 989 0%

SMB2Create 310890 3%

SMB2Read 2798078 23%

SMB2Write 2740381 23%

SMB2Lock 720 0%

SMB2Unlock 710 0%

SMB2OplkBrkAck 1766 0%

SMB2ChgNfy 6220 0%

SMB2CLose 275385 2%

SMB2Flush 138 0%

SMB2Logout 846 0%

SMB2Cancel 2689 0%

SMB2IPCCreate 2040 0%

SMB2IPCRead 2112 0%

SMB2IPCWrite 2112 0%

SMB2QueryDir 94857 1%

SMB2QueryFileBasicInfo 0 0%

SMB2QueryFileStndInfo 0 0%

SMB2QueryFileIntInfo 613 0%

SMB2QueryFileEAInfo 126267 1%

SMB2QueryFileFEAInfo 0 0%

SMB2QueryFileModeInfo 0 0%

SMB2QueryAltNameInfo 0 0%

SMB2QueryFileStreamInfo 125656 1%

SMB2QueryNetOpenInfo 18024 0%

SMB2QueryAttrTagInfo 0 0%

SMB2QueryAccessInfo 0 0%

SMB2QueryFileUnsupported 0 0%

SMB2QueryFileInvalid 0 0%

SMB2QueryFSVolInfo 55514 0%

SMB2QueryFSSizeInfo 391 0%

SMB2QueryFSDevInfo 0 0%

SMB2QueryFSAttrInfo 55516 0%

SMB2QueryFSFullSzInfo 920 0%

SMB2QueryFSObjIdInfo 0 0%

SMB2QueryFSInvalid 56 0%

SMB2QuerySecurityInfo 130384 1%

SMB2SetBasicInfo 67037 1%

SMB2SetRenameInfo 85 0%

SMB2SetFileLinkInfo 0 0%

SMB2SetFileDispInfo 569 0%

SMB2SetFullEAInfo 0 0%

SMB2SetModeInfo 0 0%

SMB2SetAllocInfo 24397 0%

SMB2SetEOFInfo 36632 0%

SMB2SetUnsupported 5 0%

SMB2SetInfoInvalid 0 0%

SMB2SetSecurityInfo 2634 0%

SMB2FsctlPipeTransceive 2214 0%

SMB2FsctlPipePeek 0 0%

SMB2FsctlEnumSnapshots 31 0%

SMB2FsctlDfsReferrals 1072 0%

SMB2FsctlSetSparse 0 0%

SMB2FsctlSecureShare 0 0%

SMB2FsctlFileUnsupported 2078 0%

SMB2FsctlIpcUnsupported 1370 0%

cancel lock 0

wait lock 0

copy to align 5

alignedSmall 194139

alignedLarge 305982

alignedSmallRel 0

alignedLargeRel 0

FidHashAllocs 63

TidHashAllocs 0

UidHashAllocs 0

mbufWait 0

nbtWait 0

pBlkWait 0

BackToBackCPWait 0

cwaWait 0

short msg prevent 320

multipleVCs 16

SMB signing 909

mapped null user 0

PDCupcalls 0

nosupport 0

read pipe busy 0

write pipe busy 0

trans pipe busy 0

read pipe broken 0

write pipe broken 0

trans pipe broken 0

queued writeraw 0

nbt disconnect 955

smb disconnect 13

dup disconnect 9

OpLkBkXorBatchToL2 1766

OpLkBkXorBatchToNone 0

OpLkBkL2ToNone 315

OpLkBkNoBreakAck 0

OpLkBkNoBreakAck95 0

OpLkBkNoBreakAckNT 0

OpLkBkIgnoredAck 72

OpLkBkWaiterTimedOut 0

OpLkBkDelayedBreak 209

SharingErrorRetries 6

FoldAttempts 0

FoldRenames 0

FoldRenameFailures 0

FoldOverflows 0

FoldDuplicates 0

FoldWAFLTooBusy 0

NoAllocCredStat 0

RetryRPCcollision 0

TconCloseTID 0

GetNTAPExtAttrs 0

SetNTAPExtAttrs 0

SearchBusy 0

ChgNfyNoMemory 0

ChgNfyNewWatch 1678

ChgNfyLastWatch 1676

UsedMIDTblCreated 14

UnusedMIDTblCreated 6

InvalidMIDRejects 0

SMB2InvalidSignature 10

SMB2DurableCreateReceived 245363

SMB2DurableCreateSucceeded 112391

SMB2DurableReclaimReceived 1

SMB2DurableReclaimSucceeded 0

SMB2DurableHandlePreserved 11

SMB2DurableHandlePurged 1

SMB2DurableHandleExpired 10

SMB2FileDirInfo 0

SMB2FileFullDirInfo 0

SMB2FileIdFullDirInfo 0

SMB2FileBothDirInfo 20

SMB2FileIdBothDirInfo 47425

SMB2FileNamesInfo 14

SMB2FileDirUnsupported 0

SMB2QueryInfo 513341

SMB2SetInfo 131359

SMB2Ioctl 6765

SMB2RelatedCompRequest 234328

SMB2UnRelatedCompRequest 0

SMB2FileRequest 6874826

SMB2PipeRequest 14651

SMB2_1_LeaseBreaks 0

SMB2_1_LeaseUpgrades 0

SMB2_1_LeaseBreakExcuses 0

SMB2_1_LeaseBreakAckTimeouts 0

SMB2_1_HandleLeaseBreaks 0

SMB2_1_LeaseBreaksToNone 0

SMB2_1_LeaseBreakAcksIgnored 0

SMB2nosupport 325

Max Multiplex = 1023, Max pBlk Exhaust = 62874, Max pBlk Reserve Exhaust = 0

Max FIDs = 1038, Max FIDs on one tree = 1036

Max Searches on one tree = 1, Max Core Searches on one tree = 0

Max sessions = 10

Max trees = 11

Max shares = 7

Max session UIDs = 2, Max session TIDs = 4

Max locks = 1040

Max credentials = 12

Max group SIDs per credential = 12

Max pBlks = 1024 Current pBlks = 1024 Num Logons = 0

Max reserved pBlks = 32 Current reserved pBlks = 32

Max gAuthQueue depth = 5

Max gSMBBlockingQueue depth = 3

Max gSMBTimerQueue depth = 5

Max gSMBAlfQueue depth = 2

Max gSMBRPCWorkerQueue depth = 1

Max gOffloadQueue depth = 2

Local groups: builtins = 6, user-defined = 1, SIDs = 6

RPC group count = 10, RPC group active count = 0

Max Watched Directories = 8, Current Watched Directories = 7

Max Pending ChangeNotify Requests = 8, Current Pending ChangeNotify Requests = 6

Max Pending DeleteOnClose Requests = 3072, Current Pending DeleteOnClose Requests = 0

=========================================================

Netapp01> cifs domaininfo

NetBios Domain: domainxx

Windows 2003 Domain Name: domainxx.local

Type: Windows 2003

Filer AD Site: Default-First-Site-Name

Current Connected DCs: \\AD02 and \\AD01

Total DC addresses found: 2

Preferred Addresses:

None

Favored Addresses:

192.168.200.5 AD01 PDC

192.168.200.6 PDC

Other Addresses:

None

Connected AD LDAP Server: \\ad01.domainxx.local

Preferred Addresses:

None

Favored Addresses:

192.168.200.5

ad01.domainxx.local

192.168.200.6

ad02.domainxx.local

Other Addresses:

None

=========================================================

RTT

=========================================================

Netapp01> ping -s 192.168.200.5

64 bytes from 192.168.200.5 (192.168.200.5): icmp_seq=0 ttl=127 time=0.218 ms

64 bytes from 192.168.200.5 (192.168.200.5): icmp_seq=1 ttl=127 time=0.245 ms

64 bytes from 192.168.200.5 (192.168.200.5): icmp_seq=2 ttl=127 time=0.236 ms

64 bytes from 192.168.200.5 (192.168.200.5): icmp_seq=3 ttl=127 time=0.216 ms

--- 192.168.200.5 ping statistics ---

4 packets transmitted, 4 packets received, 0% packet loss

round-trip min/avg/max = 0.216/0.228/0.245 ms

Netapp01> ping -s 192.168.200.6

64 bytes from 192.168.200.6 (192.168.200.6): icmp_seq=0 ttl=127 time=0.189 ms

64 bytes from 192.168.200.6 (192.168.200.6): icmp_seq=1 ttl=127 time=0.192 ms

64 bytes from 192.168.200.6 (192.168.200.6): icmp_seq=2 ttl=127 time=0.231 ms

64 bytes from 192.168.200.6 (192.168.200.6): icmp_seq=3 ttl=127 time=0.219 ms

--- 192.168.200.6 ping statistics ---

4 packets transmitted, 4 packets received, 0% packet loss

round-trip min/avg/max = 0.189/0.207/0.231 ms

=========================================================

on my tests i have this values:

Windows 2008 -> NetApp = ~35MB/s

Windows 2012 -> NetApp = ~35MB/s

Windows 2008 -> Windows 2012 = ~100MB/s

Windows 2012 -> Windows 2008 = ~90MB/s

Windows 2008 and 2012 are in within a LUN on NetApp.

shaunjurr · ‎2013-06-05

Hi,

It's still all a bit confusing because you are not specific enough about which Netapp you have sent results for. The first results here show that you aren't using smb2. The new cifs stat output looks more like you are using smb2.

Try to limit your posts to a minimum of relevant information and keep the tested combinations well labeled. Also, watch the output of 'sysstat -x 1' while you are doing your tests. Use a new empty volume without any dedupe/compression. Try using a single identical file for all tests or a single config in a program like "iometer".

The 35MB/s you post is pretty typical "smb1" speed.

S.

DATACENTER_NEO · ‎2013-06-07

Ok,

On first situation (CIFS on FAS2020) i enabled the SMB2 and my throughput is 30MB/s yet.

i need change more some cifs options?

DATACENTER_NEO · ‎2013-06-07

Result of command "sysstat -x 1" while test file tranfer.

=====================================================================================================

CPU NFS CIFS HTTP Total Net kB/s Disk kB/s Tape kB/s Cache Cache CP CP Disk FCP iSCSI FCP kB/s iSCSI kB/s

in out read write read write age hit time ty util in out in out

80% 0 628 0 639 40152 817 7229 42777 0 0 2s 100% 63% F 57% 0 11 0 0 551 0

79% 0 663 0 674 41174 832 6080 39415 0 0 2s 99% 62% Fn 56% 0 11 0 0 530 0

85% 0 511 0 512 34320 675 7304 60600 0 0 2s 100% 80% Ff 64% 0 1 0 0 4 0

82% 0 559 0 559 38017 747 6776 58788 0 0 2s 100% 77% F 63% 0 0 0 0 0 0

69% 0 531 0 565 31861 638 8672 34392 0 0 2s 99% 54% Ff 48% 0 34 0 0 430 0

81% 0 582 0 582 37582 746 10078 47976 0 0 2s 100% 77% Ff 74% 0 0 0 0 0 0

71% 0 514 0 528 32011 655 7656 40840 0 0 2s 99% 63% F 55% 0 14 0 0 77 0

79% 0 634 0 640 41903 825 7904 40496 0 0 2s 99% 60% F 53% 0 6 0 0 25 0

83% 0 496 0 502 33071 654 13626 43006 0 0 2s 99% 77% Fn 70% 0 6 0 0 14 0

84% 0 535 0 539 35759 707 6300 64116 0 0 2s 100% 74% Ff 69% 0 4 0 0 13 0

80% 0 593 0 669 40450 976 10868 36008 0 0 2s 99% 86% Ff 62% 0 76 0 0 0 167

85% 0 581 0 583 34780 704 9660 59272 0 0 2s 99% 79% F 68% 0 2 0 0 1 0

76% 0 650 0 650 43945 867 2796 38580 0 0 2 99% 41% Fn 37% 0 0 0 0 0 0

77% 0 687 0 696 31356 1119 8731 42877 0 0 2s 99% 62% Fn 56% 0 9 0 0 20 393

83% 0 505 0 505 34410 676 11796 49886 0 0 2s 100% 75% Ff 69% 0 0 0 0 0 0

81% 0 564 0 644 33076 945 8616 60060 0 0 2s 100% 74% Fs 68% 0 80 0 0 4 167

80% 0 594 0 596 39813 808 6028 49400 0 0 2s 99% 64% F 59% 0 2 0 0 1 0

79% 0 586 0 591 39905 1200 8528 40588 0 0 2s 99% 65% F 60% 0 5 0 0 0 401

79% 0 630 0 639 38961 773 6890 39313 0 0 2s 100% 67% Fn 60% 0 9 0 0 53 0

84% 0 557 0 557 33615 672 9772 48912 0 0 2s 99% 86% Fn 76% 0 0 0 0 0 0

CPU NFS CIFS HTTP Total Net kB/s Disk kB/s Tape kB/s Cache Cache CP CP Disk FCP iSCSI FCP kB/s iSCSI kB/s

in out read write read write age hit time ty util in out in out

80% 0 520 0 520 34861 686 10992 48168 0 0 2s 99% 75% Fs 69% 0 0 0 0 0 0

82% 0 515 0 520 29489 594 11616 61648 0 0 2s 99% 79% Ff 73% 0 5 0 0 5 0

30% 0 384 0 384 26170 512 188 5120 0 0 2s 99% 9% : 7% 0 0 0 0 0 0

63% 0 435 0 436 27851 552 5744 40544 0 0 2s 99% 95% F 61% 0 1 0 0 4 0

81% 0 545 0 548 37140 732 7460 42364 0 0 2s 100% 90% 2n 61% 0 3 0 0 12 0

85% 0 503 0 521 32365 644 12940 61436 0 0 2s 100% 77% Ff 67% 0 18 0 0 688 0

79% 0 570 0 594 35560 809 7968 54220 0 0 2s 99% 65% Ff 53% 0 24 0 0 170 33

77% 0 583 0 583 39806 780 5858 45902 0 0 2s 99% 58% F 52% 0 0 0 0 0 0

60% 0 361 0 365 23982 473 6932 39744 0 0 2s 100% 54% F 48% 0 4 0 0 16 0

2% 0 0 0 30 424 10 16 24 0 0 2s 99% 0% - 2% 0 30 0 0 406 0

2% 0 3 0 44 416 488 688 0 0 0 2s 96% 0% - 2% 0 41 0 0 394 459

2% 0 2 0 28 749 84 96 8 0 0 2s 88% 0% - 4% 0 26 0 0 718 66

Neonetapp1>

=====================================================================================================

Shazad_Khan · ‎2015-12-16

Did you ever get this sorted out?

I have the same setting and the same slow results

kurtleta69 · ‎2013-11-12

CIFS signing causes a huge performance issue for us. I see cifs.signing off but it is on for SMB2 in your post. When signing is on in our environment we get 70-80MBs over 10GBe. Turn it off and we get 500-900MBs depending on the client capability.

danpancamo · ‎2014-02-06

I see Max pBlk Exhaust = 62874 in your output...

You appear to be suffering from same "pBlk Exhaust" issue we are having on our 6290. pBlk Exhaust had taken down our 8.1.3P1 6190 Windows Home Directory down once, and our 3160 3 times now over 2 years.

Causes include vcan and slow AD, however after 4 P1 cases, we have yet to find the root cause.

We continue to suspect an AD communication issue, since we do not have vcan setup. However no direct proof that anything is wrong with the AD servers.

CIFS Performance

Get ready to power on