Using Python in WFA: The solution.

[sinhaa]

Support for Python as a programming language for WFA commands/DataSources has been asked IIRC since 2013. Perl is largely outdated in programming world and its user base is shrinking.

Its a long wait, so let me not keep you waiting any longer. I have no clue when, if at all WFA is going to provide support for Python. I wish they do.

The following doc guides you to start using Python in WFA. This is the beauty of WFA. It allows room to be creative and build almost anything you want to.

The solution logic:

To run python language code, I'm going to use the infrastructure that is given in WFA. The Perl's infrastructure. So If I can get to know how Perl is being handled in WFA, I can make it to work with Python as well. And this is the core logic behind this solution.

WFA version needed:

Any version of WFA will do. Nothing specific.

The Setup

You need a time setup to be done.

Windows:

1. Install Python 2.7.X on your WFA windows system. You can install Python 3.X as well, but I've not tested it. Likely will work. Python 2.7 is the most popular python version by a long stretch. So I would recommend it.

 The default Python installation path is: C:\Python27

2. Install Request Module. If you want latest version, take it from here. http://docs.python-requests.org/en/master/user/install/#install. There multiple ways to install. pipenv or Tar ball as described in the link document.  If you are having difficulty, or okay with the version I've tried, I've also added the requests.dar. Rename the file extension to .zip and then unzip the content folder 'requests' in yout C:/Python27/Lib/Site-Packages

              Why this Module? Native python with urlib, urllib2, urllib3 etc do provide mechanism for making HTTP requests, but makes programming difficult. And I didn't want to do 90s style programming in 2017(Yes this solution was developed last year). 

3. Replace your existing  \WFA\perl\WFAWrapperl.pl with the one attached here ( Modify the file extension from .txt  to .pl), . Find the line as below and if required modify it as per your python installation path. Save it.

3. $python_path = "C://Python27//python.exe";

4. Download the attached WFAUtil.txt file and save it into your C:\Python27\Lib\Site-Packages. Modify the file extension to .py . The tool doesn't allow me to attach .pl, .py files. This is the language pack library which will allow you to use python in WFA.

That's all. Done.

Linux/CentOS:

1. If your Linux has python, you will not need to install it.
2. Same as step-2 above.
3. Take the WFAWrapper.pl attached here. Modify the above mentioned line to point to your python binaries,

That's all.

Test if your setup went fine.

1. Launch Python console and try the below lines. If they don't throw any error, we are good to go.

Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 20:42:59) [MSC v.1500 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> import WFAUtil
>>> import requests
>>> 

Your WFA is ready to use python.

Creating Commands:

WFA->Designer->Commands-> Add new.

Select language "Perl'. Yes this is what we have done. Python in WFA is now posing as Perl. See I can't change the hard-coded names in GUI .  So just read 'Python' where ever there is Perl written. Just like in a Perl command, you would need to add 'Parameter Definitions' and Parameter Mappings manually. If you need to understand how to do this I can elaborate. Post your queries in comments.

I've attached a sample Test Command. The test command gives example on how to use all the available WFA methods in python. 

How to see what methods are available to use? On the python console just type 

Python 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016, 20:42:59) [MSC v.1500 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> import WFAUtil
>>> import requests
>>> help(WFAUtil)
Help on module WFAUtil:

NAME
    WFAUtil

FILE
    c:\python27\lib\site-packages\wfautil.py

CLASSES
    WFAUtil
    
    class WFAUtil
     |  Methods defined here:
     |  
     |  __get_command_obj__(self, url)
     |  
     |  __get_xml_body__(self, root, hmap)
     |  
     |  __init__(self)
     |  
     |  __parse_xml_resp__(self, content)
     |  
     |  add_wfa_workflow_parameter(self, name, value, add_return=False)
     |      :param name: Name of the dynamic Parameter
     |      :param value: Value of the dynamic Parameter
     |      :param add_return: True/False to have the parameter as a return parameter for the workflow
     |      :return: none
     |  
     |  get_connection_details(self, host)
     |      :param host: Hostname or IP Address
     |      :return: Connection details Dictionary with keys userName, password, 'remoteSystemType',
     |  
     |  get_credentials(self, host, is_datasource=False)
     |      :param host: hostname or IP Address
     |      :param is_datasource: True/False
     |      :return: credential dictionary with keys 'userName', 'password'
     |  
     |  get_wfa_global_configuration(self, configkey)
     |      :param configkey: The global_config key whose value is required
     |      :return: The value of the config key (string)
     |  
     |  get_wfa_input_password(self, encrypted_password)
     |      :param encrypted_password: Parameter of type Password in encrypted format.
     |      :return: password in plain text
     |  
     |  get_wfa_rest_parameter(self, param)
     |      :param param: Parameter whose value needs to be obtained. Valid values are userName, jobId, workflowName, commandName, workflowId, Host, Port
     |      :return: The value of the Parameter (string)
     |  
     |  get_wfa_vendor(self)
     |      :return: The WFA Vendor (string)
     |  
     |  get_wfa_version(self, full=False)
     |      :param full: True/False to obtain the WFA version in full format
     |      :return: WFA version (sting)
     |  
     |  get_wfa_workflow_parameter(self, name)
     |      :param name: The Parameter whose value needs to be obtained.
     |      :return: The string value of the parameter
     |  
     |  report_progress(self, current, total, percentage, note='')
     |      :param current: Current completed out of the total
     |      :param total: Total number to be achieved
     |      :param percentage: current/total * 100
     |      :param note: Some note to pass
     |      :return: none
     |  
     |  send_log(self, message, level='DEBUG')
     |      :param message: message to log and print command execution
     |      :param level: log level DEBUG/INFO/WARN/ERROR
     |      :return: none

DATA
    EMPTY_NAMESPACE = None
    EMPTY_PREFIX = None
    StringTypes = (<type 'str'>, <type 'unicode'>)
    XMLNS_NAMESPACE = 'http://www.w3.org/2000/xmlns/'


>>> 

I've attached a sample code here. I'll also explain some code.

import argparse
from WFAUtil import WFAUtil
import sys
import traceback

I would need the above libraries. argparse is needed for parsing the arguments, WFAUtil is my WFA python module.

wfa=WFAUtil()

Creating the WFAUtil object.

Now call a method to do any task. Let's print some output log messages of info, warn and error. Just the way its done for any python code.

wfa.send_log("=== Testing Method send_log for all log levels ===", 'INFO')

wfa.send_log(message="=== INFO Message ===", level='INFO')
wfa.send_log( level='WARN', message="=== Warning Message ===")
wfa.send_log(message="=== Error Message ===", level='ERROR')
wfa.send_log(message="=== Debug Message ===")

Creating Data Source Type Script in Python.

Same for Data Source type.

Add a new Data Source Type, method = script, language= perl.

Now provide your python code

I've attached a sample Data Source script at well.

Note:

1. I've only provided methods which are WFA specific, which are needed to work with WFA. Methods like connecting to Cluster (Connect-WfaCluster equivalent) etc. are NOT provided. They are wrapper methods and can all be done by the users themselves.
2. Now that your Perl=Python, you will No longer be able to use Perl. If you want perl back, just get  back the old WFAWrapper.pl which came with WFA. That's all.
3. A lot of NetApp certified contents from WFA Automation store are only in Perl. So now, none of them will work. As mentioned above, you either get Perl or Python.

  4. This solution logic can be used for other languages like Ruby, Shell etc. as well. I can do it, if there are enough ask for those.  

[sinhaa]

Adding the example codes here.

[MartinRohrbach]

Although I'm not a Python guy I'm always interested in using tools to their full potential -- even if it means bending the rules a bit 🙂

For this particular hack I'm wondering though if it's not easier to go the other way round. By this I mean to use a Perl module to allow you to run python code from within the perl code, for example

http://search.cpan.org/dist/Inline-Python/Python.pod

Afaik the WFA perl distribution can be managed using PPM so it should be possible to install this package "site-local". This way, the NetApp certified commands keep running and a future WFA upgrade might actually work because it (hopefully) preserves any "site-local" perl packages as opposed to the wrappers you're replacing which will probably get overwritten.

Of course, you need to wrap your python code in a minimal perl scaffolding for each command, but this seems to be the lesser of the two evils if you prefer python over perl (or even powershell)...

Totally untested, just felt like adding my 2 cent.

[sinhaa]

@MartinRohrbach

Good to see this being discussed. I'll respond on the points brought out.

Thank you.

sinhaa

[sinhaa]

@MartinRohrbach

The purpose of inline-python is is have your classes and the containing methods defined in Python. And their class objects and the methods on those being called from below Perl code. Your major programming has to be in Perl. Which not the intent.

WFA generally commands do not use classes etc, Its makes not much sesnse here.

Also if your python is only going to have a simple Printing Hello World. this may work. Accepting parameters from commands and then using those in Python will NOT work.

In short, this is not going to work in WFA.

sinhaa

[marcusgross]

Hi,

maybe the user base of Perl is shrinking. But at the end, Perl is the perfect language for WFA on the Linux side.

It's available everywhere and compatible. 

Python failed on the step from 2.6 to 3.0. They tried to introduce new things without backward compatibility.

The community doesn't follow, so they moved features from 3 to 2.7.

RedHat 7.4 includes Python 2.7 it's now 8 years old. Current version is 3.7.  

I wrote all my RestAPI scripts in Python, it works great. And it's simple.

But when it comes to simple scripts and string parsing, it's Perl.

Marcus

[sinhaa]

@marcusgross

Marcus,

Perl is the perfect language for WFA on the Linux side

-------

     I didn't get why you say the above. Every method that is abaible on WFA Perl Module, i've provided in the python module. 

Python failed on the step from 2.6 to 3.0. They tried to introduce new things without backward compatibility.

The community doesn't follow, so they moved features from 3 to 2.7.

RedHat 7.4 includes Python 2.7 it's now 8 years old. Current version is 3.7.  

-------

RHEL 7.4 includes Python 2.7. But you can always install 3.7 or which ever latest you wish. The provided solution should work with Python 3.X as well. Are you facing any problems?

I wrote all my RestAPI scripts in Python, it works great. And it's simple.

-------------

Are you not using WFA? Those scripts can be in WFA  commands. This gives many advantages over a script. If there is a problem you are facing using python in WFA, let me help you.

But when it comes to simple scripts and string parsing, it's Perl.

-------------

Simplicity of python including parsing etc. is the reason for its popularity. Subjective preferance is a totally different thing.   

sinhaa