Variable referencing not very flexible

[uptimenow]

Hi,

I am trying to make use of the variable referencing feature of SnapCreator to create a logfile name, and I am slowly coming to the conclusion that it is not as flexible as I had hoped it to be. I am using SC 3.6.0c.

It would appear to me that concatenating variables is only possible when using a dash "-" between the variable names. Any other character (I tried underscores "_" and dots ".") stops variable substitution.

I am running an executable as a POST_NTAP_DATA_TRANSFER_CMD and want to log any output from this exe to a logfile with a name like this "hksql4.SD.daily.out.20130115181730-qsm.log" under SC's logfile directory so that it can be picked up by SC's GUI. For this, I need to construct a string by using the following variables: %CONFNAME and %SNAP_TIME, but with ".out." between the two variables.

Is there any way to achieve this with any current versions of SC ? Maybe I am overlooking something, but this:

CUSTOMZERO=.out.

CUSTOMONE=%CONFNAME-%CUSTOMZERO

CUSTOMTWO=%CUSTOMONE-%SNAP_TIME-qsm.log

creates something very close, hksql4.SD.daily-.out.-20130115181730-qsm.log, but doesn't get picked up by the SC GUI report section because of the extra dashes in the logfile name.

Wouldn't it be nicer to reference variables via this construction %{VARIABLE_NAME} and allow more flexible parsing ?

(ps. for those of you who think I should just let my .exe print to stdout or stderr, and let SC's debug log pick up the output, I am confronted with an executable (perl2exe compiled perl script) that falls victim to this problem: http://support.microsoft.com/kb/971163 and a hotfix that is not working properly).

Thanks in advance,

Filip

[sivar]

echo a > %CONFNAME%.out.%SNAP_TIME%-qsm.log

Above command created a file like this with CONFNAME and SNAP_TIME variables in place of

Will this work in your script?

01/16/2013  01:50 PM

4 hqksql4.SD.daily.out.20130115181730-qsm.log

[uptimenow]

Hi,

Thanks for your feedback. What you describe works if you use the environment variables in a script called from SC, but it is not what I am trying to achieve.

I want to use the variables in the Snap Creator config file in a - somewhat complicated - concatenated way, and have SC replace them with the values of the variables regardless of what special characters (dashes, underscores and dots) are used.

More specifically, I want to run a script (which could have nothing to do with SC whatsoever) that takes the name of a logfile as an argument. This logfile name must be constructed from config variables by SC. I do not want to have the script itself look at environment variables that are passed to it ...

With this config:

POST_NTAP_DATA_TRANSFER_CMD02=SERVER: echo a > %CONFNAME%.out.%SNAP_TIME%-qsm.log

I get:

[Tue Jan 22 14:50:34 2013] INFO: Running post data transfer command POST_NTAP_DATA_TRANSFER_CMD02 [SERVER: echo a > hksql4_SD_daily%.out.%SNAP_TIME%-qsm.log]

[Tue Jan 22 14:50:34 2013] INFO: Running post data transfer command [ echo a > hksql4_SD_daily%.out.%SNAP_TIME%-qsm.log] finished successfully

which is not what I want...

Best regards,

Filip

[ktenzer]

SC only allows one %VAR per line

The only exception is SNAPSHOT_CREATE_CMDs which allow %SNAME-%SNAP_TYPE_recent or %SNAME-%SNAP_TYPE_%SNAP_TIME

The reason, doing this is incredible complex code-wise and nobody asked for it until now...we have to basically substitute and replace and re-read config whenever any actions may occur. We found when having multiple %VARs on a line things became unstable which is why we support only one. I definitely love the idea and hopefully in future we can figure out a mechanism to allow more flexibility. I will take this feedback to engineering.

BTW your syntax is not correct, should be:

POST_NTAP_DATA_TRANSFER_CMD02=SERVER: echo a > %CONFNAME.out.%SNAP_TIME-qsm.log

what I know would work is:

POST_NTAP_DATA_TRANSFER_CMD02=SERVER: echo a > configName.out.%SNAP_TIME-qsm.log

Keith

[ktenzer]

Oh actually I think after %VAR there needs to be space else we dont know what VAR is

SC will think in this case VAR is %SNAP_TIME-qsm.log instead of %SNAP_TIME

Try:

POST_NTAP_DATA_TRANSFER_CMD02=SERVER: echo a > configName.out.qsm.log.%SNAP_TIME

[uptimenow]

Hi Keith,

Thanks for the quick reply, and thinks for pointing out that only one %VAR per line is allowed.

I can understand the complexity behind this, but I would suggest that a syntax like this %{VAR} be made possible in the future. Basically in your examples you're putting the %VAR at the end to avoid the parsing problem, but we are looking at a specific logfile name with the timestamp in the middle of the logfile.

By using this syntax %{VAR}, it would be possible to reference variables with underscores in the name, *and* still append things like "-qsm.log" because the closing bracket avoids any possible parsing confusion.

Thanks for taking this into consideration.

Filip

(ps. I know %VAR% is the wrong syntax but I was following sivar's suggestion)

[ktenzer]

This is actually a really good idea, I have copied pasted this into a user story...cant give deliverable but this is definitely something we will improve and your idea may very well be what we implement but requirement is basically using %VARs all over the place and multiple on a line so got that.

I applologize that we dont have a better solution for this, I originally actually wrote the code for this and wish I had talked to you 3 years ago

Keith

[ktenzer · Accepted solution]

UPDATE:

This is now working in SC 4.0

Since SC 4.0 community release is available we would recommend you test this so we ensure we can meet this when 4.0 is released with NetApp support in March 2013.

https://snapcreator.netapp.com/download

Let us know

Keith

[uptimenow]

Hi Keith,

Are you sure this is working correctly on 4.0 yet. I tried this configuration:

BLABLA1=%SNAP_TIME-qsm.log

BLABLA2=%{SNAP_TIME}-qsm.log

POST_NTAP_DATA_TRANSFER_CMD01=SERVER: cmd /c echo %BLABLA1%

POST_NTAP_DATA_TRANSFER_CMD02=SERVER: cmd /c echo %BLABLA2%

and this yielded this:

########## POST DATA TRANSFER COMMANDS ##########

[2013-03-05 11:22:31,207] INFO: Executing Post data transfer command  cmd /c echo %SNAP_TIME-qsm.log on server

[2013-03-05 11:22:31,207] INFO: stdout:%SNAP_TIME-qsm.log

[2013-03-05 11:22:31,207] DEBUG: Post data transfer:  cmd /c echo %SNAP_TIME-qsm.log finished with exit code:  0

[2013-03-05 11:22:31,207] INFO: Executing Post data transfer command  cmd /c echo %{SNAP_TIME}-qsm.log on server

[2013-03-05 11:22:31,207] INFO: stdout:%{SNAP_TIME}-qsm.log

[2013-03-05 11:22:31,207] DEBUG: Post data transfer:  cmd /c echo %{SNAP_TIME}-qsm.log finished with exit code:  0

so it's still not what I initially wanted. Do you want me to create an issue for this on github ?

Best regards,

Filip

[ktenzer]

Your syntax is wrong, try this

BLABLA1=%SNAP_TIME-qsm.log

BLABLA2=%SNAP_TIME-qsm.log

POST_NTAP_DATA_TRANSFER_CMD01=SERVER: cmd /c echo %BLABLA1

POST_NTAP_DATA_TRANSFER_CMD02=SERVER: cmd /c echo %BLABLA2

Keith

[uptimenow]

Hi Keith, I think you were a little too fast with your reply 🙂

If you go back a little higher higher in this thread, I suggested to use a syntax like this %{VARIABLE_NAME} as a permanent solution to this variable substitution problem. You thought it was a good idea (thanks !).Then a little later you mentioned "it is now working in 4.0".

Because I didn't know whether you meant you fixed the original problem I reported OR perhaps implemented the suggested syntax with curly brackets, I created a configuration with both examples.

But the point I want to make is that none of them are getting substituted correctly. You corrected two things:

1) You removed the curly brackets {} from my BLABLA2 variable, effectively making it the same as the BLABLA1 variable, but I hope you understand my attempt to see if the solution with curly brackets had been implemented.

2) On the CMD line, you replaced the variable name %BLABLA1% (with a % at the end) by %BLABLA1  (ditto for BLABLA2). But the %BLABLA1% syntax is just required to access the environment variable from Windows/DOS. To prove this point, in my previous reply, you can see the output logfile clearly shows: "stdout:%SNAP_TIME-qsm.log" and "stdout:%{SNAP_TIME}-qsm.log" so %BLABLAx% is referring correctly to the variable, but the variable substitution is not happening in both occurences.

So I want to reiterate my point that something is still wrong 🙂

Best regards,

Filip

[ktenzer]

Ah ok yeah you need to do something like this

POST_NTAP_DATA_TRANSFER_CMD01=SERVER: cmd /c echo %SNAP_TIME-qsm.log

POST_NTAP_DATA_TRANSFER_CMD02=SERVER: cmd /c echo %SNAP_TIME-qsm.log

4.0 code has a limitation around this, however on plus side you can have multiple variables on the same line so there should be no need to do this. We can look at improving this in 4.1 if the current solution is too limiting.

Keith

[uptimenow]

Hi Keith,

Thanks for your persistence, this does indeed work. And specifying multiple variables on the same line also works, which was my initial question. So what I make of this is that this works:

POST_NTAP_DATA_TRANSFER_CMD02=SERVER: cmd /c echo %SNAP_TIME-qsm-%SNAPMIRROR_VOLUMES.log

(I'm using random variable names in my test, there's no point in creating a file with that name), yet this doesn't work:

BLABLA1=%SNAP_TIME-qsm.log

POST_NTAP_DATA_TRANSFER_CMD01=SERVER: cmd /c echo %BLABLA1

I think this construction used to work in 3.6 (albeit with a maximum of one variable per line), but no problem: this was already a kludge to try to get multiple variables per line working. The first construction, by using the variables directly on the POST_NTAP_DATA_TRANSFER_CMDxx line, works and is what I initially needed.

Best regards,

Filip