Archiving Workflow Job Execution Status Logs

[kiessl]

Hi community,

Hi developers,

in my WFA 2.2 database I have "collected" about 2000 workflow job execution logs. As those logs describe the lifecycle of storage entities I need to export those data in order to archive them in an external system.

Is there a possiblity to export the data of the workflow job execution status logs from the WFA database into ASCII files? I do not want to export each workflow individually, rather I'd like to do it with a batch job for several logs at once. Best would be to select the jobs to be exported by a execution time span. When exporting the logs I want to keep all the information which is shown in the execution status window for the workflow job (flow, details, user input, return parameter and command output).

I would appreciate if there is a ready to use solution for this. But I might be able to code such a batch job by myself if someone can tell me where this data is located and how to access it.

Thanks for your help.

Best Regards

Walter

[adaikkap]

Hi Walter,

     What kind of external system do you archive them to ? Is it some change management system or source control system ?

WFA exposes it cache db schema via a read only user which is documented in the product guide/workflow developer guide. But unfortunatley, that doesnt give access to schema other than the cache.

The workflow execution status are store in schema named WFA which is not exposed via the read only user.

Regards

adai

[kiessl]

Hi Adai,

I want to archive this data to a Snaplock device.

Do I understand right: There is no possibility for me to write a tool to export the workflow execution status logs. And there is no tool available from NetApp or the WFA community.

So what can I do?

Best Regards

Walter

[adaikkap]

Hi Walter,

     Let me summaries what I understood. You want to bulk archive the following to your snaplock device.

1. Execution Status of all or certain number of workflow based on some selection criteria like, time/user etc, like the starttime end time, which user invoked the workflow etc
2. You also want to archive the execution log that you can download today using the Exectuion portal individually.

You would like to do both of these programatically or in simpler way instead of gui clicks.

If my above summary is correct you basically are expecting one of the folloiwng.

1. A powershell function which can do this
2. Or a Equivalent REST API
3. SQL access

I know for sure option b and c aren't available.  But let me check on the option a with our Engg team and get back to you on the same.

Regards

adai

[kiessl]

Hi Adai,

your summary is right, with the following addition:

I do not want to click in the GUI for every workflow execution log because I already have about 2.000 execution logs. This would need too much time.

But I can use the GUI for a bulk archive if there is a possibilty to export the data I need and for the selection criteria you list in your summary. If this is not available or could not be made available in the GUI then a powershell function would be possible, too.

If using the GUI for a bulk archive I'd like to have the execution log of one workflow in a single ASCII file. All ASCII files of the bulk export may be put in a zip-archive and downloaded with http from WFA server to a client.

Thanks for your help!

Best Regards

Walter

[adaikkap]

Hi Walter,

     Unfortunately, there is no way to export from GUI as well in a bulk fashion.

I have opened a Request For Enahancement. Can you pls open a case with netapp and add the same to RFE # 841941 ?

Regards

adai

[abhit]

If you don't need to do very often, there is a probable workaround.

You can restore the wfa backup in another mysql DB for which you know the root password.

I installed XAMPP and restored the backup.

You can see the attached .png to know the tables from which you will be able to get the data.

Regards

Abhi

[adaikkap]

Thats a very nice workaround Abhi.

But on a long term basis, I think we should expose some of these tables, for compliance, data analytics or reporting purpose to end users.

If not tables at least some views.

Regards

adai

[kiessl]

Hi Abhi,

thanks for your help!

I still would prefer an official function for exporting several workflow job execution status logs in one step from the GUI to ascii files like described above. But until this is provided I'll use the method you suggest.

BR

Walter

[abhit]

Thanks Walter.

Agree to both Adai and you.

Regards

Abhi

[kiessl]

Hi,

after having waited for two years for an official function to export the execution status into ASCII files, I finally must implement the proposed workaround.

I have access to a WFA database from a backup and have already found the command log messages. But I cannot find the other information from the execution status window (like return parameters or the execution plan).

Can you please tell me, how to extract these information from the database?

Thanks again for your help!

BR

Walter

[sinhaa]

@kiessl

Hi Walter,

      I agree WFA hasn't given any ready to use way to get what you want. Product feature requirements have to go through many processes and approval before something can be given in the product.

But there is a very good thing about WFA, if you don't get it in product, build it. And you can build almost anything if you know how to do it. And I can guide you how to make such a solution.

I'll be using WFA rest APIs to get this done. I'll give the small writeup. Details will be given later.

Just see.

For every workflow,

1.  You can get its execution job IDs. Again there is no given API in WFA to do this, but as I said before you can do almost anything if you know how to. Lets assume this is List A

2. Once you have all the job IDS, you can filter them out according to your Time-span spread. This will be subset of A, let is be list B

3. In this list B jobs, you can get your details like Return Parameters, status.

4. There is another API WorkflowExecution Resource which for every job, provided the execution logs. This should also be able to give you execution plan.

5. You have all the data you need, now make your files as you want it.

sinhaa

[kiessl]

Hi Sinhaa,

thanks for the reply. I have no problem building this solution on my own. But I need some more details about how to. Can tell me how to do it?

Best Regards

Walter