ONTAP Discussions

CIFS Performance

DATACENTER_NEO
25,428 Views

Hello,

i have a CIFS on netapp 2240, and this throughput is only ~40MB/s on 10Gb network, when my client is windows, and when my client is linux the throughput is ~120MB/s.

If i configure a share on windows 2012 my throughput is ~130MB/s with clienta windows and linux.

My problem is betwen client windows and netapp CIFS. I configure many options on netapp but the performance is always the same.

my options configurations is:

cifs.LMCompatibilityLevel    1         

cifs.W2K_password_change     off       

cifs.W2K_password_change_interval 4w        

cifs.W2K_password_change_within 3600h     

cifs.audit.account_mgmt_events.enable on        

cifs.audit.autosave.file.extension           

cifs.audit.autosave.file.limit 0         

cifs.audit.autosave.onsize.enable off       

cifs.audit.autosave.onsize.threshold 75%       

cifs.audit.autosave.ontime.enable off       

cifs.audit.autosave.ontime.interval 1d        

cifs.audit.enable            off       

cifs.audit.file_access_events.enable on        

cifs.audit.liveview.allowed_users           

cifs.audit.liveview.enable   off       

cifs.audit.logon_events.enable on        

cifs.audit.logsize           31457280  

cifs.audit.nfs.enable        off       

cifs.audit.nfs.filter.filename           

cifs.audit.saveas            /etc/log/adtlog.evt

cifs.bypass_traverse_checking on        

cifs.client.dup-detection    ip-address

cifs.comment                           

cifs.enable_share_browsing   on        

cifs.gpo.enable              off       

cifs.gpo.trace.enable        off       

cifs.grant_implicit_exe_perms off       

cifs.guest_account                     

cifs.home_dir_namestyle                

cifs.home_dirs_public_for_admin on        

cifs.idle_timeout            28800     

cifs.ipv6.enable             off       

cifs.max_mpx                 1124      

cifs.ms_snapshot_mode        xp        

cifs.neg_buf_size            65535     

cifs.netbios_aliases                   

cifs.netbios_over_tcp.enable on        

cifs.nfs_root_ignore_acl     off       

cifs.oplocks.enable          on        

cifs.oplocks.opendelta       8         

cifs.per_client_stats.enable off       

cifs.perfmon.allowed_users             

cifs.perm_check_ro_del_ok    off       

cifs.perm_check_use_gid      on        

cifs.preserve_unix_security  off       

cifs.restrict_anonymous      0         

cifs.restrict_anonymous.enable off       

cifs.save_case               on        

cifs.scopeid                           

cifs.search_domains                    

cifs.show_dotfiles           on        

cifs.show_snapshot           off       

cifs.shutdown_msg_level      2         

cifs.sidcache.enable         on        

cifs.sidcache.lifetime       1440      

cifs.signing.enable          off       

cifs.smb2.enable             on        

cifs.smb2.signing.required   on        

cifs.smb2_1.branch_cache.enable on        

cifs.smb2_1.branch_cache.hash_time_out 3600       (value might be overwritten in takeover)

cifs.snapshot_file_folding.enable off       

cifs.symlinks.cycleguard     on        

cifs.symlinks.enable         on        

cifs.tcp_window_size         2096560   

cifs.trace_dc_connection     off       

cifs.trace_login             off       

cifs.universal_nested_groups.enable on        

cifs.widelink.ttl            10m       

can i do more something to emproove this (client side or server side)?

Thanks,

11 REPLIES 11

rvalappi
25,392 Views

there are two ways. .

1.  get a netapp performance engineer to review our perfstat

2. find cifs performance

     a. get the cifs stat output

     b. check for cifs domaininfo

     c. round trip to your dc

     d. roud trip to your vscan or fpolicy server

3. what is the hostname/SN number of your system.  is it sending ASUP  ?

DATACENTER_NEO
25,392 Views

Rajesh,

2A

Neonetapp1> cifs stat
                        reject   202171  0%
                         mkdir        4  0%
                         rmdir  2063990  0%
                          open        0  0%
                        create        0  0%
                         close 156788069  7%
                       X&close        0  0%
                         flush  4714718  0%
                       X&flush        0  0%
                        delete   967422  0%
                        rename  1078304  0%
                      NTRename        0  0%
                        getatr       74  0%
                        setatr        1  0%
                          read        0  0%
                        X&read        0  0%
                         write   107491  0%
                       X&write        0  0%
                          lock        0  0%
                        unlock        0  0%
                         mknew        0  0%
                        chkpth       30  0%
                          exit        0  0%
                         lseek        0  0%
                      lockread        0  0%
                    X&lockread        0  0%
                   writeunlock        0  0%
                      readbraw        0  0%
                     writebraw        0  0%
                        writec        0  0%
                     gettattre        0  0%
                     settattre        0  0%
                      lockingX  9433682  0%
                           IPC  1623831  0%
                         open2        0  0%
                   find_first2 120710911  5%
                    find_next2  2007396  0%
                 query_fs_info 59659578  3%
               query_path_info 714300930 31%
                 set_path_info      199  0%
               query_file_info 240282407 11%
                 set_file_info 70010663  3%
                   create_dir2        0  0%
                  Dfs_referral   441337  0%
                    Dfs_report        0  0%
                          echo  2822447  0%
                    writeclose        0  0%
                         openX       99  0%
                         readX 277912817 12%
                        writeX 397157289 17%
                     findclose        0  0%
                          tcon        0  0%
                          tdis  4011547  0%
                       negprot    58745  0%
                         login  1383756  0%
                        logout   211983  0%
                         tconX  4109959  0%
                       dskattr        0  0%
                        search        0  0%
                        fclose    60456  0%
                     NTCreateX 178153608  8%
                 NTTransCreate       23  0%
                  NTTransIoctl 26058437  1%
                 NTTransNotify  2275931  0%
                 NTTransSetSec  1903767  0%
               NTTransQuerySec  4062159  0%
              NTNamedPipeMulti        0  0%
                   NTCancel CN  1297937  0%
                NTCancel Other      275  0%
                      SMB2Echo        0  0%
                   SMB2Negprot        0  0%
              SMB2TreeConnnect        0  0%
            SMB2TreeDisconnect        0  0%
                     SMB2Login        0  0%
                    SMB2Create        0  0%
                      SMB2Read        0  0%
                     SMB2Write        0  0%
                      SMB2Lock        0  0%
                    SMB2Unlock        0  0%
                SMB2OplkBrkAck        0  0%
                    SMB2ChgNfy        0  0%
                     SMB2CLose        0  0%
                     SMB2Flush        0  0%
                    SMB2Logout        0  0%
                    SMB2Cancel        0  0%
                 SMB2IPCCreate        0  0%
                   SMB2IPCRead        0  0%
                  SMB2IPCWrite        0  0%
                  SMB2QueryDir        0  0%
        SMB2QueryFileBasicInfo        0  0%
         SMB2QueryFileStndInfo        0  0%
          SMB2QueryFileIntInfo        0  0%
           SMB2QueryFileEAInfo        0  0%
          SMB2QueryFileFEAInfo        0  0%
         SMB2QueryFileModeInfo        0  0%
          SMB2QueryAltNameInfo        0  0%
       SMB2QueryFileStreamInfo        0  0%
          SMB2QueryNetOpenInfo        0  0%
          SMB2QueryAttrTagInfo        0  0%
           SMB2QueryAccessInfo        0  0%
      SMB2QueryFileUnsupported        0  0%
          SMB2QueryFileInvalid        0  0%
            SMB2QueryFSVolInfo        0  0%
           SMB2QueryFSSizeInfo        0  0%
            SMB2QueryFSDevInfo        0  0%
           SMB2QueryFSAttrInfo        0  0%
         SMB2QueryFSFullSzInfo        0  0%
          SMB2QueryFSObjIdInfo        0  0%
            SMB2QueryFSInvalid        0  0%
         SMB2QuerySecurityInfo        0  0%
              SMB2SetBasicInfo        0  0%
             SMB2SetRenameInfo        0  0%
           SMB2SetFileLinkInfo        0  0%
           SMB2SetFileDispInfo        0  0%
             SMB2SetFullEAInfo        0  0%
               SMB2SetModeInfo        0  0%
              SMB2SetAllocInfo        0  0%
                SMB2SetEOFInfo        0  0%
            SMB2SetUnsupported        0  0%
            SMB2SetInfoInvalid        0  0%
           SMB2SetSecurityInfo        0  0%
       SMB2FsctlPipeTransceive        0  0%
             SMB2FsctlPipePeek        0  0%
        SMB2FsctlEnumSnapshots        0  0%
         SMB2FsctlDfsReferrals        0  0%
            SMB2FsctlSetSparse        0  0%
          SMB2FsctlSecureShare        0  0%
      SMB2FsctlFileUnsupported        0  0%
       SMB2FsctlIpcUnsupported        0  0%
                   cancel lock        0
                     wait lock        0
                 copy to align   452396
                  alignedSmall   639906
                  alignedLarge     9701
               alignedSmallRel        0
               alignedLargeRel        0
                 FidHashAllocs   235182
                 TidHashAllocs      943
                 UidHashAllocs       85
                      mbufWait        0
                       nbtWait        0
                      pBlkWait        0
              BackToBackCPWait        0
                       cwaWait        0
             short msg prevent     4746
                   multipleVCs   190162
                   SMB signing        0
              mapped null user        0
                    PDCupcalls        0
                     nosupport        0
                read pipe busy        0
               write pipe busy        0
               trans pipe busy        0
              read pipe broken        0
             write pipe broken        0
             trans pipe broken        0
               queued writeraw        0
                nbt disconnect    49359
                smb disconnect    20582
                dup disconnect      298
            OpLkBkXorBatchToL2  1063853
          OpLkBkXorBatchToNone        5
                OpLkBkL2ToNone   214639
              OpLkBkNoBreakAck       18
            OpLkBkNoBreakAck95        0
            OpLkBkNoBreakAckNT       18
              OpLkBkIgnoredAck     1056
          OpLkBkWaiterTimedOut        0
            OpLkBkDelayedBreak    26181
           SharingErrorRetries   748212
                  FoldAttempts        0
                   FoldRenames        0
            FoldRenameFailures        0
                 FoldOverflows        0
                FoldDuplicates        0
               FoldWAFLTooBusy        0
               NoAllocCredStat        0
             RetryRPCcollision        0
                  TconCloseTID        0
               GetNTAPExtAttrs        0
               SetNTAPExtAttrs        0
                    SearchBusy        0
                ChgNfyNoMemory        0
                ChgNfyNewWatch   913039
               ChgNfyLastWatch   912976
             UsedMIDTblCreated        0
           UnusedMIDTblCreated        0
             InvalidMIDRejects        0
          SMB2InvalidSignature        0
     SMB2DurableCreateReceived        0
    SMB2DurableCreateSucceeded        0
    SMB2DurableReclaimReceived        0
   SMB2DurableReclaimSucceeded        0
    SMB2DurableHandlePreserved        0
       SMB2DurableHandlePurged        0
      SMB2DurableHandleExpired        0
               SMB2FileDirInfo        0
           SMB2FileFullDirInfo        0
         SMB2FileIdFullDirInfo        0
           SMB2FileBothDirInfo        0
         SMB2FileIdBothDirInfo        0
             SMB2FileNamesInfo        0
        SMB2FileDirUnsupported        0
                 SMB2QueryInfo        0
                   SMB2SetInfo        0
                     SMB2Ioctl        0
        SMB2RelatedCompRequest        0
      SMB2UnRelatedCompRequest        0
               SMB2FileRequest        0
               SMB2PipeRequest        0
                 SMB2nosupport        0
Max Multiplex = 49, Max pBlk Exhaust = 0, Max pBlk Reserve Exhaust = 0
Max FIDs = 3177, Max FIDs on one tree = 2550
Max Searches on one tree = 9, Max Core Searches on one tree = 0
Max sessions = 32
Max trees = 413
Max shares = 20
Max session UIDs = 27, Max session TIDs = 161
Max locks = 3331
Max credentials = 85
Max group SIDs per credential = 20
Max pBlks = 896 Current pBlks = 896 Num Logons = 0
Max reserved pBlks = 32 Current reserved pBlks = 32
Max gAuthQueue depth         = 3
Max gSMBBlockingQueue depth  = 10
Max gSMBTimerQueue depth     = 4
Max gSMBAlfQueue depth       = 2
Max gSMBRPCWorkerQueue depth = 10
Max gOffloadQueue depth      = 2
Local groups: builtins = 6, user-defined = 1, SIDs = 5
RPC group count = 20, RPC group active count = 0
Max Watched Directories = 157, Current Watched Directories = 108
Max Pending ChangeNotify Requests = 157, Current Pending ChangeNotify Requests = 108
Max Pending DeleteOnClose Requests = 2688, Current Pending DeleteOnClose Requests = 0

2B-

Neonetapp1> cifs domaininfo
NetBios Domain:           NEOIDC
Windows 2003 Domain Name: neoidc.uolhostidc
Type:                     Windows 2003
Filer AD Site:            Default-First-Site-Name

Current Connected DCs:    \\NEO-SRV-AD001
Total DC addresses found: 4
Preferred Addresses:
                          172.30.70.138                    BDC
Favored Addresses:
                          200.98.238.139  NEO-SRV-AD002    PDC
                          200.98.238.138                   PDC
Other Addresses:
                          172.30.70.139                    PDC

Connected AD LDAP Server: \\NEO-SRV-AD001.neoidc.uolhostidc
Preferred Addresses:
                          172.30.70.138

Favored Addresses:
                          200.98.238.139
                           neo-srv-ad002.neoidc.uolhostidc
                          200.98.238.138
                           neo-srv-ad001.neoidc.uolhostidc
Other Addresses:
                          None

2C-

Neonetapp1> ping -s 172.30.70.138
64 bytes from 172.30.70.138 (172.30.70.138): icmp_seq=0 ttl=128 time=0.400 ms
64 bytes from 172.30.70.138 (172.30.70.138): icmp_seq=1 ttl=128 time=0.431 ms
64 bytes from 172.30.70.138 (172.30.70.138): icmp_seq=2 ttl=128 time=0.850 ms
64 bytes from 172.30.70.138 (172.30.70.138): icmp_seq=3 ttl=128 time=0.363 ms
64 bytes from 172.30.70.138 (172.30.70.138): icmp_seq=4 ttl=128 time=0.326 ms
64 bytes from 172.30.70.138 (172.30.70.138): icmp_seq=5 ttl=128 time=0.320 ms

--- 172.30.70.138 ping statistics ---
6 packets transmitted, 6 packets received, 0% packet loss
round-trip min/avg/max = 0.320/0.448/0.850 ms
===============================================================================
Neonetapp1> ping -s 200.98.238.139
64 bytes from 200.98.238.139 (200.98.238.139): icmp_seq=0 ttl=127 time=0.381 ms
64 bytes from 200.98.238.139 (200.98.238.139): icmp_seq=1 ttl=127 time=0.389 ms
64 bytes from 200.98.238.139 (200.98.238.139): icmp_seq=2 ttl=127 time=0.412 ms
64 bytes from 200.98.238.139 (200.98.238.139): icmp_seq=3 ttl=127 time=0.488 ms
64 bytes from 200.98.238.139 (200.98.238.139): icmp_seq=4 ttl=127 time=0.430 ms

--- 200.98.238.139 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 0.381/0.420/0.488 ms
===============================================================================
Neonetapp1> ping -s 200.98.238.138
64 bytes from 200.98.238.138 (200.98.238.138): icmp_seq=0 ttl=127 time=0.805 ms
64 bytes from 200.98.238.138 (200.98.238.138): icmp_seq=1 ttl=127 time=0.876 ms
64 bytes from 200.98.238.138 (200.98.238.138): icmp_seq=2 ttl=127 time=1.559 ms
64 bytes from 200.98.238.138 (200.98.238.138): icmp_seq=3 ttl=127 time=0.400 ms
64 bytes from 200.98.238.138 (200.98.238.138): icmp_seq=4 ttl=127 time=0.878 ms
64 bytes from 200.98.238.138 (200.98.238.138): icmp_seq=5 ttl=127 time=1.795 ms

--- 200.98.238.138 ping statistics ---
6 packets transmitted, 6 packets received, 0% packet loss
round-trip min/avg/max = 0.400/1.052/1.795 ms

2D-

Neonetapp1> vscan

Virus scanning is disabled.

No vscan servers are connected.

List of extensions to scan:
Extensions-to-scan list is empty.

List of extensions not to scan:
Extensions-not-to-scan list is empty.

Number of files scanned:  0
Number of scan failures:  0
Number of throttled requests:  0

3-

Neonetapp1> sysconfig -a

        NetApp Release 7.3.4: Thu May 27 17:52:48 PDT 2010

        System ID: 0135077937 (Neonetapp1); partner ID: 0135077939 (Neonetapp2)

        System Serial Number: 600000000755 (Neonetapp1)

The network interface is 1Gb and not 10Gb and my appliance is FAS 2020 and not 2240.

Thanks

shaunjurr
25,392 Views

Hi,

The CIFS/SMB protocoll is not very performant. If you have newer clients than win2k3 and XP, SMB2 will give you much better performance. I see, however, that you aren't using it. Enable it on the filer CLI with 'options cifs.smb2.enable on' .  Do take some time to read the documentation as well. Many of your questions will be answered there with much better authority than you will get on Communities.

You might also want to try updating to a much newer 7.3.x release to take advantage of some performance enhancements in ONTap.

S.

DATACENTER_NEO
25,392 Views

Ok,

but on other netapp i have the same problem, on ontap is 8.x, interface is 10Gb (FAS2240) and smb 2 enable.

can i sent the same informations for your analysis ?

Thanks

DATACENTER_NEO
25,391 Views

In this case i have this configurations:

Options

=========================================================

Netapp01> options cifs

cifs.LMCompatibilityLevel    1

cifs.W2K_password_change     off

cifs.W2K_password_change_interval 4w

cifs.W2K_password_change_within 3600h

cifs.audit.account_mgmt_events.enable on

cifs.audit.autosave.file.extension

cifs.audit.autosave.file.limit 0

cifs.audit.autosave.onsize.enable off

cifs.audit.autosave.onsize.threshold 75%

cifs.audit.autosave.ontime.enable off

cifs.audit.autosave.ontime.interval 1d

cifs.audit.enable            off

cifs.audit.file_access_events.enable on

cifs.audit.liveview.allowed_users

cifs.audit.liveview.enable   off

cifs.audit.logon_events.enable on

cifs.audit.logsize           31457280

cifs.audit.nfs.enable        off

cifs.audit.nfs.filter.filename

cifs.audit.saveas            /etc/log/adtlog.evt

cifs.bypass_traverse_checking on

cifs.client.dup-detection    ip-address

cifs.comment

cifs.enable_share_browsing   on

cifs.gpo.enable              off

cifs.gpo.trace.enable        off

cifs.grant_implicit_exe_perms off

cifs.guest_account

cifs.home_dir_namestyle

cifs.home_dirs_public_for_admin on

cifs.idle_timeout            28800

cifs.ipv6.enable             off

cifs.max_mpx                 1124

cifs.ms_snapshot_mode        xp

cifs.neg_buf_size            65535

cifs.netbios_aliases

cifs.netbios_over_tcp.enable on

cifs.nfs_root_ignore_acl     off

cifs.oplocks.enable          on

cifs.oplocks.opendelta       8

cifs.per_client_stats.enable off

cifs.perfmon.allowed_users

cifs.perm_check_ro_del_ok    off

cifs.perm_check_use_gid      on

cifs.preserve_unix_security  off

cifs.restrict_anonymous      0

cifs.restrict_anonymous.enable off

cifs.save_case               on

cifs.scopeid

cifs.search_domains

cifs.show_dotfiles           on

cifs.show_snapshot           off

cifs.shutdown_msg_level      2

cifs.sidcache.enable         on

cifs.sidcache.lifetime       1440

cifs.signing.enable          off

cifs.smb2.enable             on

cifs.smb2.signing.required   on

cifs.smb2_1.branch_cache.enable on

cifs.smb2_1.branch_cache.hash_time_out 3600       (value might be overwritten in takeover)

cifs.snapshot_file_folding.enable off

cifs.symlinks.cycleguard     on

cifs.symlinks.enable         on

cifs.tcp_window_size         2096560

cifs.trace_dc_connection     off

cifs.trace_login             off

cifs.universal_nested_groups.enable on

cifs.widelink.ttl            10m

cifs.wins_servers            192.168.200.6,192.168.200.5

=========================================================

=========================================================

Netapp01> cifs stat

                              reject        0  0%

                               mkdir        0  0%

                               rmdir        0  0%

                                open        0  0%

                              create        0  0%

                               close      436  0%

                             X&close        0  0%

                               flush        0  0%

                             X&flush        0  0%

                              delete       53  0%

                              rename        5  0%

                            NTRename        0  0%

                              getatr        0  0%

                              setatr        0  0%

                                read        0  0%

                              X&read        0  0%

                               write        0  0%

                             X&write        0  0%

                                lock        0  0%

                              unlock        0  0%

                               mknew        0  0%

                              chkpth        0  0%

                                exit        0  0%

                               lseek        0  0%

                            lockread        0  0%

                          X&lockread        0  0%

                         writeunlock        0  0%

                            readbraw        0  0%

                           writebraw        0  0%

                              writec        0  0%

                           gettattre        0  0%

                           settattre        0  0%

                            lockingX       72  0%

                                 IPC       67  0%

                               open2        0  0%

                         find_first2     1280  0%

                          find_next2       27  0%

                       query_fs_info       91  0%

                     query_path_info     2303  0%

                       set_path_info        2  0%

                     query_file_info      141  0%

                       set_file_info      170  0%

                         create_dir2        0  0%

                        Dfs_referral       50  0%

                          Dfs_report        0  0%

                                echo    19378  0%

                          writeclose        0  0%

                               openX        0  0%

                               readX   101608  1%

                              writeX  5086492 42%

                           findclose        0  0%

                                tcon        0  0%

                                tdis     2260  0%

                             negprot      690  0%

                               login      215  0%

                              logout       43  0%

                               tconX       64  0%

                             dskattr        0  0%

                              search        0  0%

                              fclose        0  0%

                           NTCreateX     1881  0%

                       NTTransCreate        0  0%

                        NTTransIoctl      156  0%

                       NTTransNotify      310  0%

                       NTTransSetSec        0  0%

                     NTTransQuerySec        0  0%

                    NTNamedPipeMulti        0  0%

                         NTCancel CN       14  0%

                      NTCancel Other        0  0%

                            SMB2Echo        0  0%

                         SMB2Negprot      290  0%

                    SMB2TreeConnnect     2371  0%

                  SMB2TreeDisconnect     2213  0%

                           SMB2Login      989  0%

                          SMB2Create   310890  3%

                            SMB2Read  2798078 23%

                           SMB2Write  2740381 23%

                            SMB2Lock      720  0%

                          SMB2Unlock      710  0%

                      SMB2OplkBrkAck     1766  0%

                          SMB2ChgNfy     6220  0%

                           SMB2CLose   275385  2%

                           SMB2Flush      138  0%

                          SMB2Logout      846  0%

                          SMB2Cancel     2689  0%

                       SMB2IPCCreate     2040  0%

                         SMB2IPCRead     2112  0%

                        SMB2IPCWrite     2112  0%

                        SMB2QueryDir    94857  1%

              SMB2QueryFileBasicInfo        0  0%

               SMB2QueryFileStndInfo        0  0%

                SMB2QueryFileIntInfo      613  0%

                 SMB2QueryFileEAInfo   126267  1%

                SMB2QueryFileFEAInfo        0  0%

               SMB2QueryFileModeInfo        0  0%

                SMB2QueryAltNameInfo        0  0%

             SMB2QueryFileStreamInfo   125656  1%

                SMB2QueryNetOpenInfo    18024  0%

                SMB2QueryAttrTagInfo        0  0%

                 SMB2QueryAccessInfo        0  0%

            SMB2QueryFileUnsupported        0  0%

                SMB2QueryFileInvalid        0  0%

                  SMB2QueryFSVolInfo    55514  0%

                 SMB2QueryFSSizeInfo      391  0%

                  SMB2QueryFSDevInfo        0  0%

                 SMB2QueryFSAttrInfo    55516  0%

               SMB2QueryFSFullSzInfo      920  0%

                SMB2QueryFSObjIdInfo        0  0%

                  SMB2QueryFSInvalid       56  0%

               SMB2QuerySecurityInfo   130384  1%

                    SMB2SetBasicInfo    67037  1%

                   SMB2SetRenameInfo       85  0%

                 SMB2SetFileLinkInfo        0  0%

                 SMB2SetFileDispInfo      569  0%

                   SMB2SetFullEAInfo        0  0%

                     SMB2SetModeInfo        0  0%

                    SMB2SetAllocInfo    24397  0%

                      SMB2SetEOFInfo    36632  0%

                  SMB2SetUnsupported        5  0%

                  SMB2SetInfoInvalid        0  0%

                 SMB2SetSecurityInfo     2634  0%

             SMB2FsctlPipeTransceive     2214  0%

                   SMB2FsctlPipePeek        0  0%

              SMB2FsctlEnumSnapshots       31  0%

               SMB2FsctlDfsReferrals     1072  0%

                  SMB2FsctlSetSparse        0  0%

                SMB2FsctlSecureShare        0  0%

            SMB2FsctlFileUnsupported     2078  0%

             SMB2FsctlIpcUnsupported     1370  0%

                         cancel lock        0

                           wait lock        0

                       copy to align        5

                        alignedSmall   194139

                        alignedLarge   305982

                     alignedSmallRel        0

                     alignedLargeRel        0

                       FidHashAllocs       63

                       TidHashAllocs        0

                       UidHashAllocs        0

                            mbufWait        0

                             nbtWait        0

                            pBlkWait        0

                    BackToBackCPWait        0

                             cwaWait        0

                   short msg prevent      320

                         multipleVCs       16

                         SMB signing      909

                    mapped null user        0

                          PDCupcalls        0

                           nosupport        0

                      read pipe busy        0

                     write pipe busy        0

                     trans pipe busy        0

                    read pipe broken        0

                   write pipe broken        0

                   trans pipe broken        0

                     queued writeraw        0

                      nbt disconnect      955

                      smb disconnect       13

                      dup disconnect        9

                  OpLkBkXorBatchToL2     1766

                OpLkBkXorBatchToNone        0

                      OpLkBkL2ToNone      315

                    OpLkBkNoBreakAck        0

                  OpLkBkNoBreakAck95        0

                  OpLkBkNoBreakAckNT        0

                    OpLkBkIgnoredAck       72

                OpLkBkWaiterTimedOut        0

                  OpLkBkDelayedBreak      209

                 SharingErrorRetries        6

                        FoldAttempts        0

                         FoldRenames        0

                  FoldRenameFailures        0

                       FoldOverflows        0

                      FoldDuplicates        0

                     FoldWAFLTooBusy        0

                     NoAllocCredStat        0

                   RetryRPCcollision        0

                        TconCloseTID        0

                     GetNTAPExtAttrs        0

                     SetNTAPExtAttrs        0

                          SearchBusy        0

                      ChgNfyNoMemory        0

                      ChgNfyNewWatch     1678

                     ChgNfyLastWatch     1676

                   UsedMIDTblCreated       14

                 UnusedMIDTblCreated        6

                   InvalidMIDRejects        0

                SMB2InvalidSignature       10

           SMB2DurableCreateReceived   245363

          SMB2DurableCreateSucceeded   112391

          SMB2DurableReclaimReceived        1

         SMB2DurableReclaimSucceeded        0

          SMB2DurableHandlePreserved       11

             SMB2DurableHandlePurged        1

            SMB2DurableHandleExpired       10

                     SMB2FileDirInfo        0

                 SMB2FileFullDirInfo        0

               SMB2FileIdFullDirInfo        0

                 SMB2FileBothDirInfo       20

               SMB2FileIdBothDirInfo    47425

                   SMB2FileNamesInfo       14

              SMB2FileDirUnsupported        0

                       SMB2QueryInfo   513341

                         SMB2SetInfo   131359

                           SMB2Ioctl     6765

              SMB2RelatedCompRequest   234328

            SMB2UnRelatedCompRequest        0

                     SMB2FileRequest  6874826

                     SMB2PipeRequest    14651

                  SMB2_1_LeaseBreaks        0

                SMB2_1_LeaseUpgrades        0

            SMB2_1_LeaseBreakExcuses        0

        SMB2_1_LeaseBreakAckTimeouts        0

            SMB2_1_HandleLeaseBreaks        0

            SMB2_1_LeaseBreaksToNone        0

        SMB2_1_LeaseBreakAcksIgnored        0

                       SMB2nosupport      325

Max Multiplex = 1023, Max pBlk Exhaust = 62874, Max pBlk Reserve Exhaust = 0

Max FIDs = 1038, Max FIDs on one tree = 1036

Max Searches on one tree = 1, Max Core Searches on one tree = 0

Max sessions = 10

Max trees = 11

Max shares = 7

Max session UIDs = 2, Max session TIDs = 4

Max locks = 1040

Max credentials = 12

Max group SIDs per credential = 12

Max pBlks = 1024 Current pBlks = 1024 Num Logons = 0

Max reserved pBlks = 32 Current reserved pBlks = 32

Max gAuthQueue depth         = 5

Max gSMBBlockingQueue depth  = 3

Max gSMBTimerQueue depth     = 5

Max gSMBAlfQueue depth       = 2

Max gSMBRPCWorkerQueue depth = 1

Max gOffloadQueue depth      = 2

Local groups: builtins = 6, user-defined = 1, SIDs = 6

RPC group count = 10, RPC group active count = 0

Max Watched Directories = 8, Current Watched Directories = 7

Max Pending ChangeNotify Requests = 8, Current Pending ChangeNotify Requests = 6

Max Pending DeleteOnClose Requests = 3072, Current Pending DeleteOnClose Requests = 0

=========================================================

=========================================================

Netapp01> cifs domaininfo

NetBios Domain:           domainxx

Windows 2003 Domain Name: domainxx.local

Type:                     Windows 2003

Filer AD Site:            Default-First-Site-Name

Current Connected DCs:    \\AD02 and \\AD01

Total DC addresses found: 2

Preferred Addresses:

                          None

Favored Addresses:

                          192.168.200.5   AD01             PDC

                          192.168.200.6                    PDC

Other Addresses:

                          None

Connected AD LDAP Server: \\ad01.domainxx.local

Preferred Addresses:

                          None

Favored Addresses:

                          192.168.200.5

                           ad01.domainxx.local

                          192.168.200.6

                           ad02.domainxx.local

Other Addresses:

                          None

=========================================================

RTT

=========================================================

Netapp01> ping -s 192.168.200.5

64 bytes from 192.168.200.5 (192.168.200.5): icmp_seq=0 ttl=127 time=0.218 ms

64 bytes from 192.168.200.5 (192.168.200.5): icmp_seq=1 ttl=127 time=0.245 ms

64 bytes from 192.168.200.5 (192.168.200.5): icmp_seq=2 ttl=127 time=0.236 ms

64 bytes from 192.168.200.5 (192.168.200.5): icmp_seq=3 ttl=127 time=0.216 ms

--- 192.168.200.5 ping statistics ---

4 packets transmitted, 4 packets received, 0% packet loss

round-trip min/avg/max = 0.216/0.228/0.245 ms

Netapp01> ping -s 192.168.200.6

64 bytes from 192.168.200.6 (192.168.200.6): icmp_seq=0 ttl=127 time=0.189 ms

64 bytes from 192.168.200.6 (192.168.200.6): icmp_seq=1 ttl=127 time=0.192 ms

64 bytes from 192.168.200.6 (192.168.200.6): icmp_seq=2 ttl=127 time=0.231 ms

64 bytes from 192.168.200.6 (192.168.200.6): icmp_seq=3 ttl=127 time=0.219 ms

--- 192.168.200.6 ping statistics ---

4 packets transmitted, 4 packets received, 0% packet loss

round-trip min/avg/max = 0.189/0.207/0.231 ms

=========================================================

on my tests i have this values:

Windows 2008 -> NetApp = ~35MB/s

Windows 2012 -> NetApp = ~35MB/s

Windows 2008 -> Windows 2012 = ~100MB/s

Windows 2012 -> Windows 2008 = ~90MB/s

Windows 2008 and 2012 are in within a LUN on NetApp.


shaunjurr
25,391 Views

Hi,

It's still all a bit confusing because you are not specific enough about which Netapp you have sent results for. The first results here show that you aren't using smb2. The new cifs stat output looks more like you are using smb2.

Try to limit your posts to a minimum of relevant information and keep the tested combinations well labeled.  Also, watch the output of 'sysstat -x 1' while you are doing your tests. Use a new empty volume without any dedupe/compression. Try using a single identical file for all tests or a single config in a program like "iometer".

The 35MB/s you post is pretty typical "smb1" speed.

S.

DATACENTER_NEO
25,391 Views

Ok,

On first situation (CIFS on FAS2020) i enabled the SMB2 and my throughput is 30MB/s yet.

i need change more some cifs options?

DATACENTER_NEO
25,391 Views

Result of command "sysstat -x 1" while test file tranfer.

=====================================================================================================

CPU   NFS  CIFS  HTTP   Total    Net kB/s   Disk kB/s     Tape kB/s Cache Cache  CP   CP Disk    FCP iSCSI   FCP  kB/s iSCSI  kB/s

                                  in   out   read  write  read write   age   hit time  ty util                 in   out    in   out

80%     0   628     0     639 40152   817   7229  42777     0     0     2s 100%  63%  F   57%      0    11     0     0   551     0

79%     0   663     0     674 41174   832   6080  39415     0     0     2s  99%  62%  Fn  56%      0    11     0     0   530     0

85%     0   511     0     512 34320   675   7304  60600     0     0     2s 100%  80%  Ff  64%      0     1     0     0     4     0

82%     0   559     0     559 38017   747   6776  58788     0     0     2s 100%  77%  F   63%      0     0     0     0     0     0

69%     0   531     0     565 31861   638   8672  34392     0     0     2s  99%  54%  Ff  48%      0    34     0     0   430     0

81%     0   582     0     582 37582   746  10078  47976     0     0     2s 100%  77%  Ff  74%      0     0     0     0     0     0

71%     0   514     0     528 32011   655   7656  40840     0     0     2s  99%  63%  F   55%      0    14     0     0    77     0

79%     0   634     0     640 41903   825   7904  40496     0     0     2s  99%  60%  F   53%      0     6     0     0    25     0

83%     0   496     0     502 33071   654  13626  43006     0     0     2s  99%  77%  Fn  70%      0     6     0     0    14     0

84%     0   535     0     539 35759   707   6300  64116     0     0     2s 100%  74%  Ff  69%      0     4     0     0    13     0

80%     0   593     0     669 40450   976  10868  36008     0     0     2s  99%  86%  Ff  62%      0    76     0     0     0   167

85%     0   581     0     583 34780   704   9660  59272     0     0     2s  99%  79%  F   68%      0     2     0     0     1     0

76%     0   650     0     650 43945   867   2796  38580     0     0     2   99%  41%  Fn  37%      0     0     0     0     0     0

77%     0   687     0     696 31356  1119   8731  42877     0     0     2s  99%  62%  Fn  56%      0     9     0     0    20   393

83%     0   505     0     505 34410   676  11796  49886     0     0     2s 100%  75%  Ff  69%      0     0     0     0     0     0

81%     0   564     0     644 33076   945   8616  60060     0     0     2s 100%  74%  Fs  68%      0    80     0     0     4   167

80%     0   594     0     596 39813   808   6028  49400     0     0     2s  99%  64%  F   59%      0     2     0     0     1     0

79%     0   586     0     591 39905  1200   8528  40588     0     0     2s  99%  65%  F   60%      0     5     0     0     0   401

79%     0   630     0     639 38961   773   6890  39313     0     0     2s 100%  67%  Fn  60%      0     9     0     0    53     0

84%     0   557     0     557 33615   672   9772  48912     0     0     2s  99%  86%  Fn  76%      0     0     0     0     0     0

CPU   NFS  CIFS  HTTP   Total    Net kB/s   Disk kB/s     Tape kB/s Cache Cache  CP   CP Disk    FCP iSCSI   FCP  kB/s iSCSI  kB/s

                                  in   out   read  write  read write   age   hit time  ty util                 in   out    in   out

80%     0   520     0     520 34861   686  10992  48168     0     0     2s  99%  75%  Fs  69%      0     0     0     0     0     0

82%     0   515     0     520 29489   594  11616  61648     0     0     2s  99%  79%  Ff  73%      0     5     0     0     5     0

30%     0   384     0     384 26170   512    188   5120     0     0     2s  99%   9%  :    7%      0     0     0     0     0     0

63%     0   435     0     436 27851   552   5744  40544     0     0     2s  99%  95%  F   61%      0     1     0     0     4     0

81%     0   545     0     548 37140   732   7460  42364     0     0     2s 100%  90%  2n  61%      0     3     0     0    12     0

85%     0   503     0     521 32365   644  12940  61436     0     0     2s 100%  77%  Ff  67%      0    18     0     0   688     0

79%     0   570     0     594 35560   809   7968  54220     0     0     2s  99%  65%  Ff  53%      0    24     0     0   170    33

77%     0   583     0     583 39806   780   5858  45902     0     0     2s  99%  58%  F   52%      0     0     0     0     0     0

60%     0   361     0     365 23982   473   6932  39744     0     0     2s 100%  54%  F   48%      0     4     0     0    16     0

  2%     0     0     0      30   424    10     16     24     0     0     2s  99%   0%  -    2%      0    30     0     0   406     0

  2%     0     3     0      44   416   488    688      0     0     0     2s  96%   0%  -    2%      0    41     0     0   394   459

  2%     0     2     0      28   749    84     96      8     0     0     2s  88%   0%  -    4%      0    26     0     0   718    66

Neonetapp1>

=====================================================================================================

Shazad_Khan
10,049 Views

Did you ever get this sorted out?

I have the same setting and the same slow results

kurtleta69
25,391 Views

CIFS signing causes a huge performance issue for us.  I see cifs.signing off but it is on for SMB2 in your post.  When signing is on in our environment we get 70-80MBs over 10GBe.  Turn it off and we get 500-900MBs depending on the client capability.

danpancamo
16,009 Views

I see Max pBlk Exhaust = 62874 in your output...


You appear to be suffering from same "pBlk Exhaust"  issue we are having on our 6290.   pBlk Exhaust had taken down our 8.1.3P1 6190 Windows Home Directory down once, and our 3160 3 times now over 2 years.

Causes include vcan and slow AD, however after 4 P1 cases, we have yet to find the root cause.

We continue to suspect an AD communication issue, since we do not have vcan setup.  However no direct proof that anything is wrong with the AD servers.


Public