From my experience, and things might have gotten a bit better since I had to do any migrations, mixed mode is problematic and I would avoid it.
Basically, the security style on a qtree tells the filer where to ask for user information. The file rights are admittedly different, NT-ACL's can be more complex if everything is used and have more "bit" as far as file permissions are concerned than unix files. The easiest method is to use ntfs file security wherever you mostly access the files from windows and let your unix users be the same as your windows users. Then the unix users will have access to all the files where that same windows user would. Using unix security style will reduce the complexity of the permissions, but then also a certain level of security.
If you have, for example, Oracle databases via NFS, then ntfs security styles are probably a bad idea, it just complicates authentication (and will cause problems if AD authentication is every problematic... like if time sync isn't within 5 minutes, etc....).
Sharing out the "ntfs" (or for that matter "unix"= qtrees is just a matter of exporting them to your unix servers rw, with authentication "sys" if you don't use kerberos or such and then mounting them from the servers either permanently or with automount.
Shaunjurr has the right idea....mixed is not really the way to go unless you have some application that absolutely requires is.
Your filers should be joined to your domain, if you have multiple, I hope they all trust one another.
Your filers need to be bound to LDAP.
if your usernames are the same in LDAP as they are in AD, simply run wcc -s DOMAIN\USERNAME to see if you are resolving properly. If you put in mydomain\myuser and you see something to this effect, you are probably good to go:
(NT - UNIX) account name(s): (mydomain\myuser - myuser) *************** UNIX uid = 1055 user is a member of group users (100) user is a member of group users (100)
NT membership BUILTIN\Administrators BUILTIN\Users User is also a member of Everyone, Network Users, Authenticated Users ***************
Verify all the NT/AD memberships for the user are correct and there you go. Otherwise you need to read up on the usermap.cfg file.
One thing to be sure you avoid if you have NTFS style permissions is setting ownership to creator/owner on files/directories. This seems to throw *nix systems off, I believe due to the fact that creator/owner is actually a wrapper and thus *nix has no way to map it back to anything viable.