Oh geeze, where to start, there are so many differences.
The most predominant difference between iSCSI and NFS is that iSCSI is block level and NFS is file based. With NFS, the filesystem is managed by the NFS server, in this case, the Netapp Storage System and with iSCSI the filesystem is managed by the guest os. There are pros and cons and other implication of both. Neither is really better then the other. Block based protocols where the guest os manages the filesystem allows you to take advanced of tools like snapmanager for (exchange, oracle, SAP, sql, etc)
To put as simply as possible with snapmanager, the guest OS, application and storage system all work together and ensure the file system is in a consistent state when snapshots occur.
On the other hand, typically with NFS, 50 or more virtual machines live inside the same volume and are deduplciated against each other. Deduplication on a volume with a single lun inside of it only finds duplicate blocks within a single filesystem. It is possible to place more then one iSCSI lun inside the same volume, but best practice is a 1-1 relationship.
What is the difference between NFS and CIFS is another loaded question. To start, they are both NAS file level protocols. Your comment is mostly correct. CIFS is a windows thing and NFS is a linux/unix thing, but there is some cross over. Samba is a linux/unix implementation of a CIFS server, and it is possible to mount NFS storage on a windows system with a UNIX utilities package. The ACL translations can be tricky and not fully functional when crossing OS. Netapp does some nifty stuff to separate the protocol layer from the filesystem that allows you to do cool things like share the same volume out through CIFS and NFS. Again, some gotchas with ACLs. You want to pick NTFS or unix ACLs, there is a mixed mode, but it is...well, kind of an administrative headache in my opinion. Don't use mixed mode unless you know what you are getting into and have a specific use case.
Hope this helps. I know I am just scratching the surface here.
--JMS
