Active IQ Unified Manager Discussions

WFA API/Workflow questions

DwayneM
3,366 Views

Hey NetApp Gurus!

There has been a recent drive to automate processes within my company and I have built a few self-service type tools for our users which were driven by bash scripts and php etc.

I recently built a full blown django project with views etc and now I am trying to leverage some of the WFA rest API calls to processes data and kick off jobs.

The problem I am running into now is that I want to pull data as a pre-determined selection, rather than accept input.

The problem:

The tool will give users the ability to add their own NFS export rules.

Current Process:

The users specifies the clientmatch, policy name, cluster and vserver.

New Process:

I grab all of the data from the rest api and present it to them in a drop down.

My idea:

I use a GET request to some rest API from WFA, which returns the current list of vservers depending on which cluster they select (which will be the only hard coded information)

Based on that selection, I use ajax to repopulate the second selection auto-magically using another GET request to pull all of the export policies based on what vserver they selected.

 

I found that you can run a GET against <filter/test which essentially tests what the filter will do and returns it back in a huge xml type table with <td> etc which I can work with, but the sql query wont only get the vservers from the cluster specified, it just gets all vservers...

 

Why is it so hard to get data out of WFA or am I just doing something wrong?

 

Thanks for your help!

Dwayne

6 REPLIES 6

mbeattie
3,320 Views

Hi Dwanyne,

 

It sounds like you've have built your own application portal internally which is then invoking WFA workflows to action the user request? WFA is designed to be a self service storage provisioning portal, it's not "really" designed to be a reporting portal that you can query for storage configuration...but having said that there are a number of methods to query WFA for storage configuration if you want to use it for ad-hoc reporting.

 

  • Create a WFA workflow which invokes a SQL query on the WFA server locally, formats and creates a report locally on the WFA server (in your file format of choice, EG .csv, .xml, .json) then adds the URL as return paramater  so you can download the file via https.
  • Create a remote connection to the WFA MySQL database from your application and invoke a query from within your application
  • Configure your application with a local MySQL database locally and pull the WFA data to your MySQL server on a scheduled basis to ensure that your user experience when selecting information from your application UI is responsive (IE you are not having into REST API's to query data remotely to popluate a drop down list)

 

Alternately you could explore alternate platforms than WFA which might be a more appropriate choice for building your own portal application. For example API-S is also freely freely available from NetApp and leverages REST API's:

 

https://mysupport.netapp.com/NOW/cgi-bin/software/?product=OnCommand+API+Services&platform=Linux

 

Finally you might consider querying OCUM directly instead of WFA. The latest release of OCUM has REST API's included

 

https://docs.netapp.com/ocum-94/index.jsp?topic=%2Fcom.netapp.doc.onc-um-ag%2FGUID-05C56EE7-30A0-4BF3-8920-635421864D4B.html

 

I hope this gives you some ideas and options. Please let me know if you have any questions.

 

/Matt

If this post resolved your issue, help others by selecting ACCEPT AS SOLUTION or adding a KUDO.

DwayneM
3,245 Views

Hey Matt,

 

Thanks for your response. I did come across the test REST API's and began trying to use those, but ultimately I could not get it to only return SVMs from a specifc cluster.

 

The other part of post post ended up putting in into a completely different direction.

 

I had never come across API-S, which is basically exactly what I am looking for. I had only using the WFA because I didnt know of any other way to GET or POST data, effectively allow me to create my own personal "self service" portal.

 

Today I have build another VM and deployed API-S, and connected it to OCUM and began to realize what I was missing. The only question now is, am I able to create workflows or custom POST requests which will low me to do various reoccuring tasks. NFS Export rules to the thousand linux guests we have is one of the biggest and most time consuming. With expanding volumes being the second, allowing them to expand it to up to 10% free through a portal would also prevent from being called in the middle of the night because someone decided to do a data dump.

 

At the very least I could use both products and probably accomplish everything.

 

Thanks so much for your help!

Dwayne

joele
3,271 Views

Hi Dwayne,

 

I'd generally agree with Matt's recommendations, particularly in querying the database directly as you can tweak your queries on the fly and not need to define custom WFA filters every time you want to make a change. 

 

That said what you proposed is very doable with WFA filters and rest APIs.  I replied to your reddit post as well but might as well put it here on the community site for reference in mulitple spaces:

 

"I spent a few minutes doing this in a lab environment if it helps.

 

You're on the right track with the the GET /rest/filters/uuid/test call against to use against an existing WFA filter. You can call an existing filter in WFA or create your own custom one where you can control the SQL query.

 

For this test I created a few dummy SVMs on my cluster and made a custom WFA filter. For the WFA filter I cloned the pre-existing vserver filter named 'Filter Storage Virtual Machines by key' and adjusted the syntax to only return vservers starting with string 'wfa':

 

SELECT
    vserver.name,
    vserver.type,
    vserver.uuid,
    vserver.name_service_switch,
    vserver.nis_domain,
    vserver.language,
    vserver.comment,
    vserver.admin_state,
    vserver.nfs_allowed,
    vserver.cifs_allowed,
    vserver.fcp_allowed,
    vserver.iscsi_allowed,
    cluster.primary_address AS 'cluster.primary_address'   
FROM
    cm_storage.vserver,
    cm_storage.cluster   
WHERE
    cluster.id = vserver.cluster_id               
    AND vserver.name LIKE 'wfa%'          
    AND (
        cluster.name = '${cluster_name}'                           
        OR cluster.primary_address='${cluster_name}'               
    )    

 

The new filter has a UUID of f4f9cf99-c5f1-4b31-a527-90c2e230b018 (found by using the GET /rest/filters/ call) and this filter requires an input parameter named 'cluster_name', so the URL I want to perform the GET call on is:

 

https://WFA_SERVER_IP/rest/filters/f4f9cf99-c5f1-4b31-a527-90c2e230b018/test?cluster_name=CLUSTER_NAME_REDACTED

 

And here's a CURL test which only returned the 3 vservers I hoped it would. The cluster has a total of 37 vservers so the custom filter worked great.

 

[USER@SERVER_NAME ~]# curl -k -u API_USER:API_USER_PASSWORD -X GET --header 'Accept: application/xml' 'https://WFA_SERVER_IP/rest/filters/f4f9cf99-c5f1-4b31-a527-90c2e230b018/test?cluster_name=CLUSTER_NAME_REDACTED' | xmllint --format -
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2285  100  2285    0     0  11769      0 --:--:-- --:--:-- --:--:-- 11839
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<filterTestResults>
  <filterName>wfa_vserver_filter_test</filterName>
  <dictionaryName>cm_storage.Vserver</dictionaryName>
  <parameters>
    <parameter key="cluster_name" value="CLUSTER_NAME_REDACTED"/>
  </parameters>
  <columns>
    <column>#</column>
    <column>admin_state</column>
    <column>cifs_allowed</column>
    <column>cluster.primary_address</column>
    <column>comment</column>
    <column>fcp_allowed</column>
    <column>iscsi_allowed</column>
    <column>language</column>
    <column>name</column>
    <column>name_service_switch</column>
    <column>nfs_allowed</column>
    <column>nis_domain</column>
    <column>type</column>
    <column>uuid</column>
  </columns>
  <rows>
    <row>
      <cell key="#" value="1"/>
      <cell key="admin_state" value="running"/>
      <cell key="cifs_allowed" value="1"/>
      <cell key="cluster.primary_address" value="IP_ADDRESS_REDACTED"/>
      <cell key="comment"/>
      <cell key="fcp_allowed" value="1"/>
      <cell key="iscsi_allowed" value="1"/>
      <cell key="language" value="en"/>
      <cell key="name" value="wfa_filter_test1"/>
      <cell key="name_service_switch" value="file"/>
      <cell key="nfs_allowed" value="1"/>
      <cell key="nis_domain"/>
      <cell key="type" value="data"/>
      <cell key="uuid" value="UUID_REDACTED"/>
    </row>
    <row>
      <cell key="#" value="2"/>
      <cell key="admin_state" value="running"/>
      <cell key="cifs_allowed" value="1"/>
      <cell key="cluster.primary_address" value="IP_ADDRESS_REDACTED"/>
      <cell key="comment"/>
      <cell key="fcp_allowed" value="1"/>
      <cell key="iscsi_allowed" value="1"/>
      <cell key="language" value="en"/>
      <cell key="name" value="wfa_filter_test2"/>
      <cell key="name_service_switch" value="file"/>
      <cell key="nfs_allowed" value="1"/>
      <cell key="nis_domain"/>
      <cell key="type" value="data"/>
      <cell key="uuid" value="UUID_REDACTED"/>
    </row>
    <row>
      <cell key="#" value="3"/>
      <cell key="admin_state" value="running"/>
      <cell key="cifs_allowed" value="1"/>
      <cell key="cluster.primary_address" value="IP_ADDRESS_REDACTED"/>
      <cell key="comment"/>
      <cell key="fcp_allowed" value="1"/>
      <cell key="iscsi_allowed" value="1"/>
      <cell key="language" value="en"/>
      <cell key="name" value="wfa_filter_test3"/>
      <cell key="name_service_switch" value="file"/>
      <cell key="nfs_allowed" value="1"/>
      <cell key="nis_domain"/>
      <cell key="type" value="data"/>
      <cell key="uuid" value="UUID_REDACTED"/>
    </row>
  </rows>
</filterTestResults>
[USER@SERVER_NAME ~]#

 

Hope this helps.

mbeattie
3,259 Views

Hi Joele,

 

Nice one, I hadn't considering invoking a test of a WFA filter via that REST API (that's brilliant) it will certainly provide the results however there is one gotcha to consider (which is why you might chose to use OCUM REST API's or API-S instead)

 

WFA relies on OCUM as it's source of truth. If OCUM cluster discovery fails or the service is non-responsive (for any reason) then the WFA OCUM datasource aquisition can still suceed but the data WFA aquires from OCUM can be stale and hence the results from the filter has the potential to be out of sync with the storage.

 

If developing an application  i'd recommend "considering" using either OCUM REST API's or API-S. WFA will work as you've demonstrated but it's an additional application layer so there are a number of decisions to make for the applications design\architecture. Some performance testing of response times between REST WFA filters, OCUM REST API's and API-S queries would also be advisable.

 

WFA might be more responsive, i quess it depends what the requirements are.

 

/Matt

If this post resolved your issue, help others by selecting ACCEPT AS SOLUTION or adding a KUDO.

DwayneM
3,244 Views

Hey Joele,

 

Thanks for your response! I actually saw your response first on reddit, and was exactly what I was looking for before I found API-S. Ultimately I want to be able to query the data with a simple GET request returning some JSON and filter through it, which is just for restricting the user's selection on the form, then the form would execute a POST request using the selected data to submit the data or trigger a workflow to perform the work. In this particular case, I am trying to create a form, by which the user provide one or many clientmatches, then selects a cluster, which uses AJAX to reload the available SVM list, which then reloads the export policy, which will ultimately allow a user to request access to a specific NFS export.

 

In between obviously I will do some checking that the client match exists as well as send an email at the end with the results, was it successful or failure, which clients were specified and which export they were granted access to.

 

I need to dig a bit more into API-S but it looks like it has almost everything I need.

 

Thanks again for your help and extremely educated response! 🙂

 

Dwayne

mbeattie
3,232 Views

Hi Dwayne,

 

It's certainly possible to achieve your objectives, you might chose a combination of both WFA\API-S. I guess it depends if you think there would be an advantage to having a "self-service" storage provisioning portal externally to your application to enable administrators, service desk or application owners to provision or configure storage?.

 

I guess the advantage of WFA in this scenario is that workflow approvals are already a builtin feature and it does support RBAC and LDAP integration so you can restrict the ability of invoking workflows to specific users and categories. Using "approval points" in WFA workflows would help you put some limitations on processes depending on your requirements. Leveraging these features might save you the work of having to implement the same functionality in your own application.

 

Also of potential interest to you is that WFA can leverlage API-S by installing this:

 

https://automationstore.netapp.com/pack-detail.shtml?packUuid=8bfe3a86-0a4e-4d1f-b460-19c15c91dc35&packVersion=2.0.0

 

IE instead of WFA leveraging the PowerShell toolkit or invoking PERL ZAPI's it can invoke API-S via REST.

 

You may also consider automating event driven tasks before they result in a request or incident. For example you can configure OCUM alerts to attach a script (which invokes a WFA workflow) that remediates a potential issue within a specified threshold before it becomes an operational issue. See the following technical report:

 

https://www.netapp.com/us/media/tr-4585.pdf

 

Ultimately it depends on your application portal architecture? You could build your own application to invoke API-S directly or you could have your portal invoke a WFA workflow (which can invoke API-S if you choose)

 

Hope this is useful and provides some helpful ideas

 

/Matt

If this post resolved your issue, help others by selecting ACCEPT AS SOLUTION or adding a KUDO.
Public