FileInfo write python example

[RobertSimac]

Hi Everybody,

I did quite an extensive search but was unable to find any functional examples of Python REST api doing the FileInfo writing. (Reading is ok).

The samples at https://library.netapp.com/ecmdocs/ECMLP2858435/html/resources/file_info.html#netapp_ontap.resources.file_info.FileInfo.get_collection are not functional.

An sample of writing to new, named file and writing to existing file would suffice.

I'm currently at Ontap 9.7

Thanks!

Robert

[ddegraaf]

Hi Robert, the docs you linked to are an older version of our documentation for netapp_ontap. Please refer to the examples in our latest docs (https://library.netapp.com/ecmdocs/ECMLP2882316/html/resources/file_info.html) and let me know if you have any issues writing to both new and existing files from the documentation found there. 

Thanks,

Dnaiel

[RobertSimac]

Daniel thanks for quick reply.

Unfortunately, the latest docs' example is also failing. I tried the 'write to new file' example verbatim and it throws an exception:

Caused by HTTPError('400 Client Error: Bad Request for url: https://devcd1.lod.netapp.com:443/api/storage/volumes/ddc41772-5a2b-4c5e-9e56-f33dd0b821e1/files'): Invalid JSON input. Unexpected character in stream: d around 0:3.

I tried to write both string and bytes, same error:

file.post(hydrate=True, data="Data to be written")

file.post(hydrate=True, data=b"Data to be written")

Can you please give that example a quick try, make sure it works as described, and let me know how it went. Also do let me know which ontap version you are testing against, to rule out possible ontap 9.7 (my version) limitations.

Also, from the 'writing to new file' example, I don't understand how to name such new file. The FileInfo object creating considers nothing but volume id, there is no file name as such.

Thanks

Robert

[ddegraaf]

Hi Robert, 

I did some testing against a client running 9.7 and realized that you must include the path of the new file that you would like to create as a positional argument when initializing the FileInfo resource. For example:

from netapp_ontap import HostConnection
from netapp_ontap.resources import FileInfo
with HostConnection("<mgmt-ip>", username="admin", password="password", verify=False):
    resource = FileInfo("54c06ce2-5430-11ea-90f9-005056a73aff", "newFileName")
    resource.post(hydrate=True, data="the data to be written to the new file")
    print(resource)  

 This will create a file "newFileName" and put the data into that file. This also answers your second question on how to name such a new file. 

I hope this fixes all of your issues. 

Thanks,

Daniel

[RobertSimac]

Thanks much, writing to new file works now.

Do you know who should be contacted to fix the documentation page so others don't run into same trouble. I could not find any links on the doc page itself...

One more question, about writing to existing file. It also fails when invoked as documented. At the beginning  of this sample I just demonstrated the existing file really exists, and then continued with the docs sample verbatim...

>>> netapp_ontap.config.CONNECTION=conn
>>> filename='rest_file.dat'
>>> data='sample_data'
>>> vol = netapp_ontap.resources.Volume.find(name="RSSIMPLE")
>>> files = netapp_ontap.resources.FileInfo.get_collection(vol.uuid, name=filename)
>>> files=list(files)
>>> files
[FileInfo({'_links': {'metadata': {'href': '/api/storage/volumes/ddc41772-5a2b-4c5e-9e56-f33dd0b821e1/files/rest_file%2Edat?return_metadata=true'}}, 'path': '', 'name': 'rest_file.dat', 'volume': {'uuid': 'ddc41772-5a2b-4c5e-9e56-f33dd0b821e1', '_links': {'self': {'href': '/api/storage/volumes/ddc41772-5a2b-4c5e-9e56-f33dd0b821e1'}}}, 'type': 'file'})]
>>> #example from docs
>>> resource = netapp_ontap.resources.FileInfo(vol.uuid, path="rest_file.dat")
>>> resource.patch(hydrate=True, data="*here is a little more data", byte_offset=39)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\user\AppData\Roaming\Python\Python39\site-packages\netapp_ontap\resources\file_info.py", line 1688, in patch
    return super()._patch(
  File "C:\Users\user\AppData\Roaming\Python\Python39\site-packages\netapp_ontap\utils.py", line 51, in wrapper
    return func(*args, **kwargs)
  File "C:\Users\user\AppData\Roaming\Python\Python39\site-packages\netapp_ontap\resource.py", line 1104, in _patch
    return self._poll(
  File "C:\Users\user\AppData\Roaming\Python\Python39\site-packages\netapp_ontap\resource.py", line 1288, in _poll
    response = self._get(fields='*')
  File "C:\Users\user\AppData\Roaming\Python\Python39\site-packages\netapp_ontap\utils.py", line 55, in wrapper
    return on_api_fail(erro)
  File "C:\Users\user\AppData\Roaming\Python\Python39\site-packages\netapp_ontap\utils.py", line 78, in on_api_fail
    raise NetAppRestError(cause=erro) from None
netapp_ontap.error.NetAppRestError:  Caused by HTTPError('500 Server Error: Internal Server Error for url: https://devcd1:443/api/storage/volumes/ddc41772-5a2b-4c5e-9e56-f33dd0b821e1/files/rest_file.dat?fields=%2A'): Not a directory
>>>

[RobertSimac]

Found the working solution for both writing to new file and overwriting existing one. Below is the sample method covering the file finding, opening, posting, reading and patching, all tested ok against ontap 9.7.

I still believe documentation should be updated to reflect the actual functionality. Mandatory use of host connection headers at get and patch is most important detail.

def writefile_rest(restconn, volname, filename, data):
    '''
    create if not existing or overwrite if existing filename at volname with supplied data
    tested against ontap 9.7
    '''

    try:
        existing = None
        with restconn:
            vol = netapp_ontap.resources.Volume.find(name=volname)

            #check if file already exists
            existing = netapp_ontap.resources.FileInfo.get_collection(vol.uuid, name=filename)
            existing = list(existing) #list() excercises the generator

            #recreating the file object, was not able to use the existing[0] retrieved object
            file = netapp_ontap.resources.FileInfo(vol.uuid, filename)
            
            if not existing:
                #post works only for new files
                file.post(hydrate=True, data=data)



        if existing: #file existing, demonstrate reading and then overwriting
            
            with restconn.with_headers({"Accept": "multipart/form-data"}): 
                #these headers are mandatory for reading and patching
                #note the file itself should be created/opened without above headers, with default connection
                
                file.get(byte_offset=0, length=4096)
                for file_data in file.response_files:
                    print(file_data)
                
                '''
                note this is overwriting existing data from the byte_offset to the data len
                the remaining data residing in the file will not be overwritten
                '''
                file.patch(hydrate=True, data=data, byte_offset=0)
        


    except Exception as e:
        logger.error(f'Failed writing file {filename} at volume {volname}: {e}')

[ddegraaf · Accepted solution]

Glad you found a work around. I agree. I created a ticket (1529956) to fix the documentation issues. Both to change the example for writing to a new file to be like the one I posted above, and to change the example of adding to an existing file to be like:

from netapp_ontap import HostConnection
from netapp_ontap.resources import FileInfo
  
headers = ({"Accept": "multipart/form-data"})
with HostConnection("<mgmt-ip>", username="admin", password="password", verify=False, headers=headers):
    resource = FileInfo("54c06ce2-5430-11ea-90f9-005056a73aff", path="aNewFile")
    resource.patch(hydrate=True, data="*here is a little more data", byte_offset=39)

Sorry about the issues with the documentation. They will be fixed.

Thanks,

Daniel 

[Shadari]

Hi folks. Can anyone provide curl examples of writing to and reading from files?