As emphasized in some places including NetApp TR's. Mixed style should not be used in general - it's functionality that exists because NetApp could provide it and some customers wanted it. but it's poorly serve the purpose that most users would consider a "mix". in short - mixed means that on whatever protocol the individual file taken ownership with (creates as well). it would be the type of ACL for that file (NTFS for CIFS, UNIX for NFS).
This can create lot's of confusion. and not really giving any value.
what you should do instead is to set the volume/qtree to the file system it mainly going to be manged from. if it also need access from other platform you can consider using a client side implementation to connect to the non-native platform (such SAMBA for linux of NFS client for windows). if that's not possible, consider separating the dataset to separate volumes/qtrees and not allowing shared access (as it also have disadvantages) . or use the default/anonymous mapping. / create a one way map between one platform to another. (similar to how you did, however keep in ,ind it meant to use with external nameservices )
i can;t really tell if your command actually worked or not. but as you can see from the description above. you can always create a file with any type of ACL in mixed mode. but. the share used to access it and the folder you placing the file within. must have write access allowed on it (on whatever ACL style it has). i guess that if you give the folder 777 or everyone-modify in windows..you would be able to write to it from everywhere with or without the mapping.working. but again - please don't use mixed-mode.