topic Re: Quota Management in NetApp PowerShell Toolkit broken? in Microsoft Virtualization Discussions

Quota Management in NetApp PowerShell Toolkit broken?

Wolfgang_Ratzka — Tue, 09 Jul 2024 14:37:12 GMT

I'm trying to port an old NetApp PowerShell Toolkit script running on an old server that is managing our home directory quota to the current version of the NetApp PowerShell Toolkit (9.14.1.2401) on an up-to-date server.

First strange effect: Unless I specify -ZapiCall, the Get-NcQuota and Get-NcQuotaReport are useless for me, as the QuotaTarget fields come up blank. So -ZapiCall it must be, even if that interface is deprecated.

Next problem: I have a quota entry, that clearly exists:

> Get-NcQuota -Type User -Target AD\ratzka_t -Volume vol_HRZ_H -VserverContext vsrz001 -Qtree ""

QuotaType QuotaTarget Volume Qtree DiskLimit FileLimit Vserver
--------- ----------- ------ ----- --------- --------- -------
user AD\ratzka_t vol_HRZ_H 36909875 - vsrz001

But trying to remove it gives me a confusing error message:

> Remove-NcQuota -Type User -Target AD\ratzka_t -Volume vol_HRZ_H -VserverContext vsrz001 -Qtree ""
Remove-NcQuota : Group parameter neither specified nor set
At line:1 char:1
+ Remove-NcQuota -Type User -Target AD\ratzka_t -Volume vol_HRZ_H -Vser ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Remove-NcQuota], ArgumentException
+ FullyQualifiedErrorId : Group is empty,DataONTAP.C.PowerShell.SDK.Cmdlets.Quota.RemoveNcQuota

Incidentally, searching for this error message turns up a suggestion to just do REST directly.

So is there anybody out there who is successfully using the NcQuota-cmdlets?

Kind Regards,

Wolfgang Ratzka

Re: Quota Management in NetApp PowerShell Toolkit broken?

mbeattie — Wed, 10 Jul 2024 07:33:01 GMT

Hi Wolfgang,

I had a look into the issue (both using an older PSTK version which works) and the latest version (which doesn't work). I think this might be related to the REST API implementation (which the PSTK is leveraging):

I will reach out to the developers. As a workaround you can use functions in your code that leverage either REST API or the private REST CLI to invoke commands over HTTPS. Here are some examples

Create a quota using the private REST CLI:

Param( [Parameter(Mandatory = $True, HelpMessage = "The Cluster Name or IP Address")] [String]$Cluster, [Parameter(Mandatory = $True, HelpMessage = "The Vserver Name")] [String]$Vserver, [Parameter(Mandatory = $True, HelpMessage = "The Volume Name")] [String]$Volume, [Parameter(Mandatory = $False, HelpMessage = "The Qtree Name")] [String]$Qtree, [Parameter(Mandatory = $True, HelpMessage = "The Quota Policy Name")] [String]$QuotaPolicy, [Parameter(Mandatory = $True, HelpMessage = "The Quota Type")] [ValidateSet("group","tree","user")] [String]$QuotaType, [Parameter(Mandatory = $True, HelpMessage = "The Quota Target")] [String]$Target, [Parameter(Mandatory = $True, HelpMessage = "The Disk Limit size measured in the storage unit (KB, MB, GB, TB)")] [Int]$DiskLimit, [Parameter(Mandatory = $True, HelpMessage = "The Disk Limit in the storage type")] [ValidateSet("KB","MB","GB","TB")] [String]$StorageUnit, [Parameter(Mandatory = $True, HelpMessage = "The Credential to authenticate to the Cluster")] [ValidateNotNullOrEmpty()] [System.Management.Automation.PSCredential]$Credential ) #'------------------------------------------------------------------------------ Function Get-OntapAuthorization{ [CmdletBinding()] Param( [Parameter(Mandatory = $True, HelpMessage = "The Credential to authenticate to the ONTAP cluster")] [ValidateNotNullOrEmpty()] [System.Management.Automation.PSCredential]$Credential ) #'-------------------------------------------------------------------------- #'Set the authentication header to connect to the cluster. #'-------------------------------------------------------------------------- $auth = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($Credential.UserName + ':' + $Credential.GetNetworkCredential().Password)) $headers = @{ "Authorization" = "Basic $auth" "Accept" = "application/json" "Content-Type" = "application/json" } Return $headers; }#'End Function Get-OntapAuthorization. #'------------------------------------------------------------------------------ Function New-OntapQuotaRule{ [CmdletBinding()] Param( [Parameter(Mandatory = $True, HelpMessage = "The Cluster Name or IP Address")] [String]$Cluster, [Parameter(Mandatory = $True, HelpMessage = "The Vserver Name")] [String]$Vserver, [Parameter(Mandatory = $True, HelpMessage = "The Volume Name")] [String]$Volume, [Parameter(Mandatory = $False, HelpMessage = "The Qtree Name")] [String]$Qtree, [Parameter(Mandatory = $True, HelpMessage = "The Quota Policy Name")] [String]$QuotaPolicy, [Parameter(Mandatory = $True, HelpMessage = "The Quota Type")] [ValidateSet("group","tree","user")] [String]$QuotaType, [Parameter(Mandatory = $True, HelpMessage = "The Quota Target")] [String]$Target, [Parameter(Mandatory = $True, HelpMessage = "The Disk Limit in Bytes")] [Int]$DiskLimit, [Parameter(Mandatory = $True, HelpMessage = "The Disk Limit in the storage type")] [ValidateSet("KB","MB","GB","TB")] [String]$StorageUnit, [Parameter(Mandatory = $True, HelpMessage = "The Credentials to authenticate to the destination cluster")] [System.Management.Automation.PSCredential]$Credential ) #'-------------------------------------------------------------------------- #'Enumerate the SnapMirror relationship. #'-------------------------------------------------------------------------- $headers = Get-OntapAuthorization -Credential $Credential [String]$command = "volume quota policy rule create -vserver $Vserver -policy-name $QuotaPolicy -volume $Volume -type $QuotaType -target $Target " If($Qtree){ $command += "-qtree $Qtree " }Else{ $command += "-qtree '' " } [String]$command += "-disk-limit $DiskLimit" [String]$uri = "https://$Cluster/api/private/cli/volume/quota/policy/rule" Write-Host "Executing Command`: $command" $cli = @{}; $cli.Add("vserver", $Vserver) $cli.Add("policy-name", $QuotaPolicy) $cli.Add("volume", $Volume) $cli.Add("type", $QuotaType) $cli.Add("target", $Target) $cli.Add("disk-limit", $($DiskLimit.ToString() + $StorageUnit)) If(($Qtree)){ $cli.Add("qtree", $Qtree) }Else{ $cli.Add("qtree", '') } $body = $cli | ConvertTo-Json Try{ $response = Invoke-RestMethod -Method POST -Body $body -Uri $uri -Headers $headers -ErrorAction Stop Write-Host "Created Quota Policy Rule on cluster ""$Cluster"" using URI ""$uri""" }Catch{ Write-Warning -Message $("Failed Creating Quota Policy Rule on cluster ""$Cluster"" using URI ""$uri"". Error " + $_.Exception.Message) Return $Null; } Return $response }#'End Function New-OntapQuotaRule. #'------------------------------------------------------------------------------ #'Set the Certificate Policy and TLS version. #'------------------------------------------------------------------------------ If(-Not("TrustAllCertsPolicy" -As [Type])){ Add-Type @" using System.Net; using System.Security.Cryptography.X509Certificates; public class TrustAllCertsPolicy : ICertificatePolicy { public bool CheckValidationResult( ServicePoint srvPoint, X509Certificate certificate, WebRequest request, int certificateProblem) { return true; } } "@ } [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]'Tls12' [System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy #'------------------------------------------------------------------------------ #'Create the Quota Policy Rule. #'------------------------------------------------------------------------------ $response = New-OntapQuotaRule -Cluster $Cluster -Vserver $Vserver -Volume $Volume -Qtree $Qtree -QuotaPolicy $QuotaPolicy -QuotaType $QuotaType -Target $Target -DiskLimit $DiskLimit -StorageUnit $StorageUnit -Credential $Credential #'------------------------------------------------------------------------------

Usage:

PS C:\Scripts\PowerShell\Projects\SetQuotaCli> .\SetQuotaCli.ps1 -Cluster 192.168.100.2 -Vserver vserver1 -Volume cifs_data_001 -Qtree "" -QuotaPolicy default -QuotaType user -Target TESTLAB\N2003909 -DiskLimit 1 -StorageUni t GB -Credential $credential Executing Command: volume quota policy rule create -vserver vserver1 -policy-name default -volume cifs_data_001 -type user -target TESTLAB\N2003909 -qtree '' -disk-limit 1 Created Quota Policy Rule on cluster "192.168.100.2" using URI "https://192.168.100.2/api/private/cli/volume/quota/policy/rule" cluster1::*> volume quota policy rule show -vserver vserver1 -volume cifs_data_001 -type user -policy-name default -target TESTLAB\N2003909 Vserver: vserver1 Policy: default Volume: cifs_data_001 Soft Soft User Disk Disk Files Files Type Target Qtree Mapping Limit Limit Limit Limit Threshold ----- -------- ------- ------- -------- ------- ------ ------- --------- user TESTLAB\N2003909 "" off 1GB - - - -

Here's an example using the REST API to remove a quota entry:

Param( [Parameter(Mandatory = $True, HelpMessage = "The Cluster Name or IP Address")] [String]$Cluster, [Parameter(Mandatory = $True, HelpMessage = "The Vserver Name")] [String]$Vserver, [Parameter(Mandatory = $True, HelpMessage = "The Volume Name")] [String]$Volume, [Parameter(Mandatory = $False, HelpMessage = "The Qtree Name")] [String]$Qtree, [Parameter(Mandatory = $True, HelpMessage = "The Quota Policy Name")] [String]$QuotaPolicy, [Parameter(Mandatory = $True, HelpMessage = "The Quota Type")] [ValidateSet("group","tree","user")] [String]$QuotaType, [Parameter(Mandatory = $True, HelpMessage = "The Quota Target")] [String]$Target, [Parameter(Mandatory = $True, HelpMessage = "The Credential to authenticate to the Cluster")] [ValidateNotNullOrEmpty()] [System.Management.Automation.PSCredential]$Credential ) #'------------------------------------------------------------------------------ Function Get-OntapAuthorization{ [CmdletBinding()] Param( [Parameter(Mandatory = $True, HelpMessage = "The Credential to authenticate to the ONTAP cluster")] [ValidateNotNullOrEmpty()] [System.Management.Automation.PSCredential]$Credential ) #'-------------------------------------------------------------------------- #'Set the authentication header to connect to the cluster. #'-------------------------------------------------------------------------- $auth = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($Credential.UserName + ':' + $Credential.GetNetworkCredential().Password)) $headers = @{ "Authorization" = "Basic $auth" "Accept" = "application/json" "Content-Type" = "application/json" } Return $headers; }#'End Function Get-OntapAuthorization. #'------------------------------------------------------------------------------ Function Remove-OntapQuotaRule{ [CmdletBinding()] Param( [Parameter(Mandatory = $True, HelpMessage = "The Cluster Name or IP Address")] [String]$Cluster, [Parameter(Mandatory = $True, HelpMessage = "The Vserver Name")] [String]$Vserver, [Parameter(Mandatory = $True, HelpMessage = "The Volume Name")] [String]$Volume, [Parameter(Mandatory = $False, HelpMessage = "The Qtree Name")] [String]$Qtree, [Parameter(Mandatory = $True, HelpMessage = "The Quota Policy Name")] [String]$QuotaPolicy, [Parameter(Mandatory = $True, HelpMessage = "The Quota Type")] [ValidateSet("group","tree","user")] [String]$QuotaType, [Parameter(Mandatory = $True, HelpMessage = "The Quota Target")] [String]$Target, [Parameter(Mandatory = $True, HelpMessage = "The Credentials to authenticate to the destination cluster")] [System.Management.Automation.PSCredential]$Credential ) #'-------------------------------------------------------------------------- #'Enumerate the quota. #'-------------------------------------------------------------------------- $headers = Get-OntapAuthorization -Credential $Credential [String]$uri = "https://$Cluster/api/storage/quota/rules?type=$QuotaType" If($QuotaType -eq "user"){ [String]$user = $Target.Replace("\", "%5C") [String]$uri += "&users.name=$user" } [String]$uri += "&svm.name=$Vserver&volume.name=$Volume&fields=users.name" Try{ $response = Invoke-RestMethod -Method GET -Uri $uri -Headers $headers -ErrorAction Stop Write-Host "Enumerated Quota Policy Rule on cluster ""$Cluster"" using URI ""$uri""" }Catch{ Write-Warning -Message $("Failed Enumerating Quota Policy Rule on cluster ""$Cluster"" using URI ""$uri"". Error " + $_.Exception.Message) Return $Null; } #'-------------------------------------------------------------------------- #'Enumerate the quota rule's UUID for deletion. #'-------------------------------------------------------------------------- If($response.num_records -eq 1){ [String]$uuid = $response.records.uuid }Else{ Write-Warning -Message "Failed Enumerating Quota Policy Rule on cluster ""$Cluster"" using URI ""$uri""" Return $Null; } #'-------------------------------------------------------------------------- #'Delete the quota rule. #'-------------------------------------------------------------------------- [String]$uri = "https://$Cluster/api/storage/quota/rules/$uuid" Try{ $response = Invoke-RestMethod -Method DELETE -Uri $uri -Headers $headers -ErrorAction Stop Write-Host "Deleted Quota Policy Rule on cluster ""$Cluster"" using URI ""$uri""" }Catch{ Write-Warning -Message $("Failed Deleting Quota Policy Rule on cluster ""$Cluster"" using URI ""$uri"". Error " + $_.Exception.Message) Return $Null; } Return $response }#'End Function Remove-OntapQuotaRule. #'------------------------------------------------------------------------------ #'Set the Certificate Policy and TLS version. #'------------------------------------------------------------------------------ If(-Not("TrustAllCertsPolicy" -As [Type])){ Add-Type @" using System.Net; using System.Security.Cryptography.X509Certificates; public class TrustAllCertsPolicy : ICertificatePolicy { public bool CheckValidationResult( ServicePoint srvPoint, X509Certificate certificate, WebRequest request, int certificateProblem) { return true; } } "@ } [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]'Tls12' [System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy #'------------------------------------------------------------------------------ #'Delete the Quota Policy Rule. #'------------------------------------------------------------------------------ $response = Remove-OntapQuotaRule -Cluster $Cluster -Vserver $Vserver -Volume $Volume -Qtree $Qtree -QuotaPolicy $QuotaPolicy -QuotaType $QuotaType -Target $Target -Credential $Credential #'------------------------------------------------------------------------------

Usage:

PS C:\Scripts\PowerShell\Projects\RemoveQuotaCli> .\RemoveQuotaCli.ps1 -Cluster 192.168.100.2 -Vserver vserver1 -Volume cifs_data_001 -Qtree "" -QuotaPolicy default -QuotaType user -Target "TESTLAB\N2003909" -Credential $cre dential Enumerated Quota Policy Rule on cluster "192.168.100.2" using URI "https://192.168.100.2/api/storage/quota/rules?type=user&users.name=TESTLAB%5CN2003909&svm.name=vserver1&volume.name=cifs_data_001&fields=users.name" Deleted Quota Policy Rule on cluster "192.168.100.2" using URI "https://192.168.100.2/api/storage/quota/rules/83406eaf-3e8c-11ef-ac71-005056a0bb1e" cluster1::*> volume quota policy rule show -vserver vserver1 -volume cifs_data_001 -type user -policy-name default -target TESTLAB\N2003909 There are no entries matching your query.

Hope that helps

/Matt

Re: Quota Management in NetApp PowerShell Toolkit broken?

Wolfgang_Ratzka — Wed, 10 Jul 2024 14:12:18 GMT

I have continued looking for a solution myself and found it easier to use Invoke-NcSsh together with the volume quota policy rule commands, as this requires less "glue" and fits into the original script quite nicely. (Well, the quota in my first attempt where too generous by a factor of 1024...)

I will, however, look into your suggestions as I am not sure that we shouldn't move from PowerShell Toolkit to REST-API or REST-CLI.

Thanks a lot

Wolfgang

Re: Quota Management in NetApp PowerShell Toolkit broken?

mbeattie — Thu, 11 Jul 2024 04:34:40 GMT

Hi Wolfgang

There is also an "Invoke-NcCli" CmdLet that can be used as an alternative to SSH which uses the private REST CLI but I prefer to define my own functions. Anyway i did ping the PSTK developers to take a look into the issues with these CmdLet's so hopefully it gets fixed in a future release.

/Matt

Re: Quota Management in NetApp PowerShell Toolkit broken?

mbeattie — Thu, 11 Jul 2024 06:29:45 GMT

Hi Wolfgang,

Note that you can also use the "Invoke-NcCli" CmdLet (which can be used as an alternative to SSH). Essentially this will invoke the private REST CLI (so if there's ever an issue using a CmdLet or a command that has no associated CmdLet then you can resort to this method). Here's an example of creating a Quota:

Connect-NcController -Name 192.168.100.2 -Credential $credential | Out-Null Invoke-NcCli -Command "volume quota policy rule create" -Body '{"type": "user", "vserver": "vserver1", "policy-name": "default", "target": "TESTLAB\\N2003905", "qtree": "", "disk-limit": "5GB", "volume": "cifs_data_001"}'

This might be an option for you if you prefer not to add custom functions to your code to invoke the REST API and just use native CmdLet's within the PSTK.

Hope that helps

/Matt

Re: Quota Management in NetApp PowerShell Toolkit broken?

saharsh — Mon, 15 Jul 2024 18:10:33 GMT

In Rest QuotaTarget is No Rest Equivalent

quota-target

NO REST EQUIVALENT

The REST API uses one of the three separate fields instead of quota-target: users/group/qtree.

Re: Quota Management in NetApp PowerShell Toolkit broken?

saharsh — Mon, 15 Jul 2024 18:16:31 GMT

For Remove-NcQuota could you please share the debug logs with us on ng-ontap-pstk-rest DL it will help us to identify the issue more efficiently.

You can enable the debug logs using Set-NaToolkitConfiguration debug command.