To actualy answer your question: No, it is currently not possible to shrink a VMFS Volume, neither using ESX 3.5 nor vSphere 4.
To further help on your issues: It is possible tho to keep the VMFS as it is but reclaim the space on the netapp. You should check the Thin Provisioning Guide and see which method fits your needs best. As already mentioned, volume guarantee none and space reservation off is a good start.
If you really want or need to shrink a datastore, then I'd highly recommend looking at using NFS from your NetApp to ESX -- growing and shrinking a datastore becomes trivial (can even do it from the vCenter interface with the RCU plugin from NetApp).
Actually that wasn't his question, he wanted to shrink the volume not the LUN/FS inside.
Anyway, the only way to non-disruptively shrink an ESX LUN datastore is to create a new (smaller) one, SVMotion all VMs into it, and then delete the old (larger) one. Problem is that you need more space temporarily, which most people who want to shrink the volume/LUN don't have
The answer to your question is yes - you can shrink a volume without impacting the LUN. Additionally, you could change the volume guarantee from volume to file and the 100GB that's unused would be returned to the containing aggregate.