It will not "truly" affect much. However there are two important areas to note. The system will still mirror the data.
The destination will be replicating at the time it has set, so if your replication is time sensitive, you need them in sync.
e.g.
Mirrors need to be in sync every 15 minutes after a data dump/load
SystemA has time 00:13:00 Source- Read/Write
SystemB has time 00:16:00 Destination - Read Only
You expect the replication to occur in 2 minutes, however the destination controls the mirror, it won't actually occur for 14 more
Also snapmirror status can actually show negative results depending on which system has the time lagging.
-Scott