Hi.
The size depends on the vendor and your network/compute resources.
Most of them can scan compressed archive files, again - it's up to the vendor to dictate how deep (cascaded files).
Encrypted archives are non-scanable and you can usually filter them out within the vendor product settings (Some encrypted content is sometimes allowed, password-protected docs, encrypted content in signed EXE files, etc...).
It's important to remember that all that ONTAP do is to put the files in administrative share and call the VSCAN agent to tell the AV product to come and scan it, and then it passes back a true/false response for ONTAP do deny or allow the write/read.