topic Re: DFM.Snapshots table imports in Active IQ Unified Manager Discussions

DFM.Snapshots table imports

f_duranti — Thu, 05 Jun 2025 06:36:55 GMT

Hi, anyone have a good implementation of importing DFM snapshots table from DFM with finders and filters to share?

Thanks

Francesco

Re: DFM.Snapshots table imports

yaronh — Mon, 23 Jan 2012 22:17:04 GMT

Hi Francesco,

Snapshot caching will be part of 1.1 release (ETA March 2012)

Regards,

Yaron Haimsohn

Re: DFM.Snapshots table imports

f_duranti — Mon, 23 Jan 2012 23:40:56 GMT

Can it be exported and reimported in 1.0.2? It would be a nice addition to have without having to rewrite it.

Francesco

Re: DFM.Snapshots table imports

dekel — Tue, 24 Jan 2012 06:48:34 GMT

Hi Francesco,

It can not be directly imported into 1.0.2, but it can be used as reference both table structure and queries .

Can you give some details on the use case ,why do you need to cache snapshots ?

Dekel

Re: DFM.Snapshots table imports

f_duranti — Tue, 24 Jan 2012 08:09:13 GMT

I need the snapshot list for 2 use cases:

1) to check the existence of a static snapshot (before deleting it) used for a workflow that will clone a volume on a destination VSM volume

2) I need to find the most recent snapshot starting with a specified string to do workflow for disaster recovery

I was trying to create this one in a new cache table but got some proble with the object definition and linking to other table with foreign key (probably using wrong name somewhere).

Re: DFM.Snapshots table imports

f_duranti — Tue, 24 Jan 2012 10:01:29 GMT

Checking on how other table are done I was trying to do this:

1) create a cache table (I've done this in a different schema because It seems that I cannot them change the standard DFM provider storage tables:

CREATE TABLE storage2.snapshot (

`id` int NOT NULL AUTO_INCREMENT,

`name` varchar(255) DEFAULT NULL,

`vol_id` int,

`aggr_id` int,

`array_id` int,

`snapid` bigint,

`accessTime` timestamp NOT NULL,

PRIMARY KEY (`id`)) ENGINE=InnoDB;

I've set array_id, vol_id, name as natural key and put the foreign key for array_id on storage.array.id, vol_id on storage.volume.id and aggr_id on storage.aggregate.id

Then I've created a new DFM Data Source Table linking the storage2 schema and doing this selection from dfm snapshots table:

select

snapshotId as id,

snapshotName as name,

volId as vol_id,

aggrId as aggr_id,

hostId as array_id,

snapshotUniqueId as snapid,

accessTime as accessTime

from dfm.snapshots

In the dictionary I changed aggr_id, vol_id, array_id to aggregate, volume, array (with the correct type) and on the ref column I've put the aggr_id, array_id, vol_id.

My problem is that when i create the datasource and try to do the import I get a "Foreign Key to a non existent table array" error.

Any suggestion?

Re: DFM.Snapshots table imports

dekel — Wed, 25 Jan 2012 09:27:07 GMT

Hello ,

First let me say , Nice progress!

The problem is that the Snapsot query you have created contains ID references taken from the DFM (vol_id ,aggr_id ,array_id) and reference to already created tables in the storage schema .

(We definitely need to change the UI so such mistakes will not be done .)

In order to solve you problem you should

1) Remove the FK to foreign keys to array table (you will need to recreate the table )

2) Add to the Select query the volume Name and the Array IP (or host name) -do JOIN with the relevant DFM tables.

3) Do the JOIN in the Filter level between snapshot and Volumes .

Another simpler option is to wait for 1.1 where WFA contains embedded support for snapshot caching as part of the certified content

Another note :

If you choose to go on and implement the query on 1.0.2 ,note that the query you created caches deleted snapshots

you can use the below Create and Select queries as reference

CREATE TABLE storage.snapshot (

`id` INT NOT NULL AUTO_INCREMENT,

`name` VARCHAR(255) NOT NULL,

`timestamp` DATETIME DEFAULT NULL,

`volume_id` INT NOT NULL,

PRIMARY KEY (`id`)

) ENGINE=InnoDB

SELECT
snapshot.snapshotId AS id,
snapshot.snapshotName AS name,
snapshot.accessTime AS timestamp,
snapshot.volId AS volume_id

FROM
dfm.snapshots snapshot
JOIN
dfm.volumes volume ON volume.volId = snapshot.volId AND volume.volState IS NOT NULL
JOIN
dfm.aggrs aggregate ON volume.aggrId = aggregate.aggrId
JOIN
dfm.objects obj ON volume.volId = obj.objid AND obj.objDeleted IS NULL AND ((obj.objFlags & 0x20000000000000) = 0)

Re: DFM.Snapshots table imports

f_duranti — Wed, 25 Jan 2012 16:13:34 GMT

Hi dekel, thanks for the good explanation Thanks for the information about deleted snapshots (I was not thinking about it... )

If i've understood the problem is mostly that the array/volume/aggregate table are already created so when I define a new table in another schema those old table in the storage schema will not be updated and the data are not imported. So only tables in the same schema can reference each other, is this correct?

I'll probably go with this until 1.1 is out just to start to do some testing and put on some test workflow we will need but I'll wait for 1.1. to starting working with them.

Just a curiosity: I saw that I cannot put the snapshot table in the default Data Source Type for DFM (because it will not let me put in the query on the snapshot table) but can I create a new Data Source Type copying all the old query to the new with the added one for snapshot? This should create a schema with all the tables with the right foreign keys. Is this correct?

Francesco

Re: DFM.Snapshots table imports

dekel — Wed, 25 Jan 2012 16:32:18 GMT

Hi.

Your understanding is indeed correct .

Your suggested workaround will work as well ,but I advice against it .when you create your "own" storage schema you are detaching yourself from the mainline ,

You will not benefit from updates and bug fixes made for the certified "storage" schema .

Hope it answers your q

Dekel

Re: DFM.Snapshots table imports

f_duranti — Wed, 25 Jan 2012 17:23:52 GMT

You're right, I'll just go with a simple table and the join at filter level for know. I just need to check for snapshot existance or newer snapshot so it's not a problem to wait for 1.1 and the full implementation.

Thanks really much for the full explanation on how WFA is working, it helped really much.

Francesco

Re: DFM.Snapshots table imports

f_duranti — Fri, 27 Jan 2012 12:14:40 GMT

Done some tests and still have problems because it seems that in this case, with tables in different schemas it will not be possible to have the correct "volid" because the volid from the dfm snapshots table is a number based on DFM internal data while WFA volume.id is an autoincrement. When I join the 2 tables I don't get the match between volumes and snapshot. I'm trying to include the volname directly into the snapshot table so at least this will solve for now my problem (i should just check for snapshot existance).

Francesco

Re: DFM.Snapshots table imports

dekel — Sat, 28 Jan 2012 16:18:19 GMT

Hello ,

Please see my response from above, the second bullet ,you cannot join By ID in this case -one reflects the DFM row IDs and the other is the WFA ID

2) Add to the Select query the volume Name and the Array IP (or host name) -do JOIN with the relevant DFM tables.

Re: DFM.Snapshots table imports

f_duranti — Sat, 28 Jan 2012 19:28:03 GMT

You're right, I had misunderstood what was there I've done it that way and it works correctly.

Thanks

Francesco