Use aggr relocate if you don't have any volume on it and then delete it.
storage aggregate relocation start
Or add -force true while running aggr offline command.
Example:
nas-cm92::> set d
Warning: These diagnostic commands are for use by NetApp personnel only.
Do you want to continue? {y|n}: y
nas-cm92::*> aggr offline -aggregate test -force-offline true
Warning: The aggregate contains one or more volumes. In order to offline the
aggregate, all of these volumes must be unmounted. Do you want to
unmount all of the volumes hosted by the aggregate?
{y|n}: y
Aggregate offline successful on aggregate: test