Solved: Re: Quota Management in NetApp PowerShell Toolkit broken?

Wolfgang_Ratzka · ‎2024-07-09

I'm trying to port an old NetApp PowerShell Toolkit script running on an old server that is managing our home directory quota to the current version of the NetApp PowerShell Toolkit (9.14.1.2401) on an up-to-date server.

First strange effect: Unless I specify -ZapiCall, the Get-NcQuota and Get-NcQuotaReport are useless for me, as the QuotaTarget fields come up blank. So -ZapiCall it must be, even if that interface is deprecated.

Next problem: I have a quota entry, that clearly exists:

> Get-NcQuota -Type User -Target AD\ratzka_t -Volume vol_HRZ_H -VserverContext vsrz001 -Qtree ""

QuotaType QuotaTarget Volume Qtree DiskLimit FileLimit Vserver
--------- ----------- ------ ----- --------- --------- -------
user AD\ratzka_t vol_HRZ_H 36909875 - vsrz001

But trying to remove it gives me a confusing error message:

> Remove-NcQuota -Type User -Target AD\ratzka_t -Volume vol_HRZ_H -VserverContext vsrz001 -Qtree ""
Remove-NcQuota : Group parameter neither specified nor set
At line:1 char:1
+ Remove-NcQuota -Type User -Target AD\ratzka_t -Volume vol_HRZ_H -Vser ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Remove-NcQuota], ArgumentException
+ FullyQualifiedErrorId : Group is empty,DataONTAP.C.PowerShell.SDK.Cmdlets.Quota.RemoveNcQuota

Incidentally, searching for this error message turns up a suggestion to just do REST directly.

So is there anybody out there who is successfully using the NcQuota-cmdlets?

Kind Regards,

Wolfgang Ratzka

mbeattie · ‎2024-07-10

Hi Wolfgang

There is also an "Invoke-NcCli" CmdLet that can be used as an alternative to SSH which uses the private REST CLI but I prefer to define my own functions. Anyway i did ping the PSTK developers to take a look into the issues with these CmdLet's so hopefully it gets fixed in a future release.

/Matt

If this post resolved your issue, help others by selecting ACCEPT AS SOLUTION or adding a KUDO.

View solution in original post

mbeattie · ‎2024-07-10

Hi Wolfgang,

I had a look into the issue (both using an older PSTK version which works) and the latest version (which doesn't work). I think this might be related to the REST API implementation (which the PSTK is leveraging):

I will reach out to the developers. As a workaround you can use functions in your code that leverage either REST API or the private REST CLI to invoke commands over HTTPS. Here are some examples

Create a quota using the private REST CLI:

Param(
    [Parameter(Mandatory = $True, HelpMessage = "The Cluster Name or IP Address")]
    [String]$Cluster,
    [Parameter(Mandatory = $True, HelpMessage = "The Vserver Name")]
    [String]$Vserver,
    [Parameter(Mandatory = $True, HelpMessage = "The Volume Name")]
    [String]$Volume,
    [Parameter(Mandatory = $False, HelpMessage = "The Qtree Name")]
    [String]$Qtree,
    [Parameter(Mandatory = $True, HelpMessage = "The Quota Policy Name")]
    [String]$QuotaPolicy,
    [Parameter(Mandatory = $True, HelpMessage = "The Quota Type")]
    [ValidateSet("group","tree","user")]
    [String]$QuotaType,
    [Parameter(Mandatory = $True, HelpMessage = "The Quota Target")]
    [String]$Target,
    [Parameter(Mandatory = $True, HelpMessage = "The Disk Limit size measured in the storage unit (KB, MB, GB, TB)")]
    [Int]$DiskLimit,
    [Parameter(Mandatory = $True, HelpMessage = "The Disk Limit in the storage type")]
    [ValidateSet("KB","MB","GB","TB")]
    [String]$StorageUnit,
    [Parameter(Mandatory = $True, HelpMessage = "The Credential to authenticate to the Cluster")]
    [ValidateNotNullOrEmpty()]
    [System.Management.Automation.PSCredential]$Credential
)
#'------------------------------------------------------------------------------
Function Get-OntapAuthorization{
    [CmdletBinding()]
    Param(
        [Parameter(Mandatory = $True, HelpMessage = "The Credential to authenticate to the ONTAP cluster")]
        [ValidateNotNullOrEmpty()]
        [System.Management.Automation.PSCredential]$Credential
    )
    #'--------------------------------------------------------------------------
    #'Set the authentication header to connect to the cluster.
    #'--------------------------------------------------------------------------
    $auth    = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($Credential.UserName + ':' + $Credential.GetNetworkCredential().Password))
    $headers = @{
        "Authorization" = "Basic $auth"
        "Accept"        = "application/json"
        "Content-Type"  = "application/json"
    }
    Return $headers;
}#'End Function Get-OntapAuthorization.
#'------------------------------------------------------------------------------
Function New-OntapQuotaRule{
    [CmdletBinding()]
    Param(
        [Parameter(Mandatory = $True, HelpMessage = "The Cluster Name or IP Address")]
        [String]$Cluster,
        [Parameter(Mandatory = $True, HelpMessage = "The Vserver Name")]
        [String]$Vserver,
        [Parameter(Mandatory = $True, HelpMessage = "The Volume Name")]
        [String]$Volume,
        [Parameter(Mandatory = $False, HelpMessage = "The Qtree Name")]
        [String]$Qtree,
        [Parameter(Mandatory = $True, HelpMessage = "The Quota Policy Name")]
        [String]$QuotaPolicy,
        [Parameter(Mandatory = $True, HelpMessage = "The Quota Type")]
        [ValidateSet("group","tree","user")]
        [String]$QuotaType,
        [Parameter(Mandatory = $True, HelpMessage = "The Quota Target")]
        [String]$Target,
        [Parameter(Mandatory = $True, HelpMessage = "The Disk Limit in Bytes")]
        [Int]$DiskLimit,
        [Parameter(Mandatory = $True, HelpMessage = "The Disk Limit in the storage type")]
        [ValidateSet("KB","MB","GB","TB")]
        [String]$StorageUnit,
        [Parameter(Mandatory = $True, HelpMessage = "The Credentials to authenticate to the destination cluster")]
        [System.Management.Automation.PSCredential]$Credential
    )
    #'--------------------------------------------------------------------------
    #'Enumerate the SnapMirror relationship.
    #'--------------------------------------------------------------------------
    $headers         = Get-OntapAuthorization -Credential $Credential
    [String]$command = "volume quota policy rule create -vserver $Vserver -policy-name $QuotaPolicy -volume $Volume -type $QuotaType -target $Target "
    If($Qtree){
        $command += "-qtree $Qtree "
    }Else{
        $command += "-qtree '' "
    }
    [String]$command += "-disk-limit $DiskLimit"
    [String]$uri     = "https://$Cluster/api/private/cli/volume/quota/policy/rule"
    Write-Host "Executing Command`: $command"
    $cli = @{};
    $cli.Add("vserver", $Vserver)
    $cli.Add("policy-name", $QuotaPolicy)
    $cli.Add("volume", $Volume)
    $cli.Add("type", $QuotaType)
    $cli.Add("target", $Target)
    $cli.Add("disk-limit", $($DiskLimit.ToString() + $StorageUnit))
    If(($Qtree)){
        $cli.Add("qtree", $Qtree)
    }Else{
        $cli.Add("qtree", '')
    }
    $body = $cli | ConvertTo-Json
    Try{
        $response = Invoke-RestMethod -Method POST -Body $body -Uri $uri -Headers $headers -ErrorAction Stop
        Write-Host "Created Quota Policy Rule on cluster ""$Cluster"" using URI ""$uri"""
    }Catch{
        Write-Warning -Message $("Failed Creating Quota Policy Rule on cluster ""$Cluster"" using URI ""$uri"". Error " + $_.Exception.Message)
        Return $Null;
    }
    Return $response
}#'End Function New-OntapQuotaRule.
#'------------------------------------------------------------------------------
#'Set the Certificate Policy and TLS version.
#'------------------------------------------------------------------------------
If(-Not("TrustAllCertsPolicy" -As [Type])){
Add-Type @"
    using System.Net;
    using System.Security.Cryptography.X509Certificates;
    public class TrustAllCertsPolicy : ICertificatePolicy {
    public bool CheckValidationResult(
    ServicePoint srvPoint, X509Certificate certificate,
    WebRequest request, int certificateProblem) {
        return true;
    }
}
"@
}
[System.Net.ServicePointManager]::SecurityProtocol  = [System.Net.SecurityProtocolType]'Tls12'
[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
#'------------------------------------------------------------------------------
#'Create the Quota Policy Rule.
#'------------------------------------------------------------------------------
$response = New-OntapQuotaRule -Cluster $Cluster -Vserver $Vserver -Volume $Volume -Qtree $Qtree -QuotaPolicy $QuotaPolicy -QuotaType $QuotaType -Target $Target -DiskLimit $DiskLimit -StorageUnit $StorageUnit -Credential $Credential
#'------------------------------------------------------------------------------

Usage:

PS C:\Scripts\PowerShell\Projects\SetQuotaCli> .\SetQuotaCli.ps1 -Cluster 192.168.100.2 -Vserver vserver1 -Volume cifs_data_001 -Qtree "" -QuotaPolicy default -QuotaType user -Target TESTLAB\N2003909 -DiskLimit 1 -StorageUni
t GB -Credential $credential
Executing Command: volume quota policy rule create -vserver vserver1 -policy-name default -volume cifs_data_001 -type user -target TESTLAB\N2003909 -qtree '' -disk-limit 1
Created Quota Policy Rule on cluster "192.168.100.2" using URI "https://192.168.100.2/api/private/cli/volume/quota/policy/rule"

cluster1::*> volume quota policy rule show -vserver vserver1 -volume cifs_data_001 -type user -policy-name default -target TESTLAB\N2003909

Vserver: vserver1          Policy: default           Volume: cifs_data_001

                                               Soft             Soft
                         User         Disk     Disk   Files    Files
Type   Target    Qtree   Mapping     Limit    Limit   Limit    Limit  Threshold
-----  --------  ------- -------  --------  -------  ------  -------  ---------
user   TESTLAB\N2003909  "" off        1GB        -       -        -          -

Here's an example using the REST API to remove a quota entry:

Param(
    [Parameter(Mandatory = $True, HelpMessage = "The Cluster Name or IP Address")]
    [String]$Cluster,
    [Parameter(Mandatory = $True, HelpMessage = "The Vserver Name")]
    [String]$Vserver,
    [Parameter(Mandatory = $True, HelpMessage = "The Volume Name")]
    [String]$Volume,
    [Parameter(Mandatory = $False, HelpMessage = "The Qtree Name")]
    [String]$Qtree,
    [Parameter(Mandatory = $True, HelpMessage = "The Quota Policy Name")]
    [String]$QuotaPolicy,
    [Parameter(Mandatory = $True, HelpMessage = "The Quota Type")]
    [ValidateSet("group","tree","user")]
    [String]$QuotaType,
    [Parameter(Mandatory = $True, HelpMessage = "The Quota Target")]
    [String]$Target,
    [Parameter(Mandatory = $True, HelpMessage = "The Credential to authenticate to the Cluster")]
    [ValidateNotNullOrEmpty()]
    [System.Management.Automation.PSCredential]$Credential
)
#'------------------------------------------------------------------------------
Function Get-OntapAuthorization{
    [CmdletBinding()]
    Param(
        [Parameter(Mandatory = $True, HelpMessage = "The Credential to authenticate to the ONTAP cluster")]
        [ValidateNotNullOrEmpty()]
        [System.Management.Automation.PSCredential]$Credential
    )
    #'--------------------------------------------------------------------------
    #'Set the authentication header to connect to the cluster.
    #'--------------------------------------------------------------------------
    $auth    = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($Credential.UserName + ':' + $Credential.GetNetworkCredential().Password))
    $headers = @{
        "Authorization" = "Basic $auth"
        "Accept"        = "application/json"
        "Content-Type"  = "application/json"
    }
    Return $headers;
}#'End Function Get-OntapAuthorization.
#'------------------------------------------------------------------------------
Function Remove-OntapQuotaRule{
    [CmdletBinding()]
    Param(
        [Parameter(Mandatory = $True, HelpMessage = "The Cluster Name or IP Address")]
        [String]$Cluster,
        [Parameter(Mandatory = $True, HelpMessage = "The Vserver Name")]
        [String]$Vserver,
        [Parameter(Mandatory = $True, HelpMessage = "The Volume Name")]
        [String]$Volume,
        [Parameter(Mandatory = $False, HelpMessage = "The Qtree Name")]
        [String]$Qtree,
        [Parameter(Mandatory = $True, HelpMessage = "The Quota Policy Name")]
        [String]$QuotaPolicy,
        [Parameter(Mandatory = $True, HelpMessage = "The Quota Type")]
        [ValidateSet("group","tree","user")]
        [String]$QuotaType,
        [Parameter(Mandatory = $True, HelpMessage = "The Quota Target")]
        [String]$Target,
        [Parameter(Mandatory = $True, HelpMessage = "The Credentials to authenticate to the destination cluster")]
        [System.Management.Automation.PSCredential]$Credential
    )
    #'--------------------------------------------------------------------------
    #'Enumerate the quota.
    #'--------------------------------------------------------------------------
    $headers      = Get-OntapAuthorization -Credential $Credential
    [String]$uri  = "https://$Cluster/api/storage/quota/rules?type=$QuotaType"
    If($QuotaType -eq "user"){
        [String]$user = $Target.Replace("\", "%5C")
        [String]$uri += "&users.name=$user"
    }
    [String]$uri += "&svm.name=$Vserver&volume.name=$Volume&fields=users.name"
    Try{
        $response = Invoke-RestMethod -Method GET -Uri $uri -Headers $headers -ErrorAction Stop
        Write-Host "Enumerated Quota Policy Rule on cluster ""$Cluster"" using URI ""$uri"""
    }Catch{
        Write-Warning -Message $("Failed Enumerating Quota Policy Rule on cluster ""$Cluster"" using URI ""$uri"". Error " + $_.Exception.Message)
        Return $Null;
    }
    #'--------------------------------------------------------------------------
    #'Enumerate the quota rule's UUID for deletion.
    #'--------------------------------------------------------------------------
    If($response.num_records -eq 1){
        [String]$uuid = $response.records.uuid
    }Else{
        Write-Warning -Message "Failed Enumerating Quota Policy Rule on cluster ""$Cluster"" using URI ""$uri"""
        Return $Null;
    }
    #'--------------------------------------------------------------------------
    #'Delete the quota rule.
    #'--------------------------------------------------------------------------
    [String]$uri = "https://$Cluster/api/storage/quota/rules/$uuid"
    Try{
        $response = Invoke-RestMethod -Method DELETE -Uri $uri -Headers $headers -ErrorAction Stop
        Write-Host "Deleted Quota Policy Rule on cluster ""$Cluster"" using URI ""$uri"""
    }Catch{
        Write-Warning -Message $("Failed Deleting Quota Policy Rule on cluster ""$Cluster"" using URI ""$uri"". Error " + $_.Exception.Message)
        Return $Null;
    }
    Return $response
}#'End Function Remove-OntapQuotaRule.
#'------------------------------------------------------------------------------
#'Set the Certificate Policy and TLS version.
#'------------------------------------------------------------------------------
If(-Not("TrustAllCertsPolicy" -As [Type])){
Add-Type @"
    using System.Net;
    using System.Security.Cryptography.X509Certificates;
    public class TrustAllCertsPolicy : ICertificatePolicy {
    public bool CheckValidationResult(
    ServicePoint srvPoint, X509Certificate certificate,
    WebRequest request, int certificateProblem) {
        return true;
    }
}
"@
}
[System.Net.ServicePointManager]::SecurityProtocol  = [System.Net.SecurityProtocolType]'Tls12'
[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
#'------------------------------------------------------------------------------
#'Delete the Quota Policy Rule.
#'------------------------------------------------------------------------------
$response = Remove-OntapQuotaRule -Cluster $Cluster -Vserver $Vserver -Volume $Volume -Qtree $Qtree -QuotaPolicy $QuotaPolicy -QuotaType $QuotaType -Target $Target -Credential $Credential
#'------------------------------------------------------------------------------

Usage:

PS C:\Scripts\PowerShell\Projects\RemoveQuotaCli> .\RemoveQuotaCli.ps1 -Cluster 192.168.100.2 -Vserver vserver1 -Volume cifs_data_001 -Qtree "" -QuotaPolicy default -QuotaType user -Target "TESTLAB\N2003909" -Credential $cre
dential
Enumerated Quota Policy Rule on cluster "192.168.100.2" using URI "https://192.168.100.2/api/storage/quota/rules?type=user&users.name=TESTLAB%5CN2003909&svm.name=vserver1&volume.name=cifs_data_001&fields=users.name"
Deleted Quota Policy Rule on cluster "192.168.100.2" using URI "https://192.168.100.2/api/storage/quota/rules/83406eaf-3e8c-11ef-ac71-005056a0bb1e"

cluster1::*> volume quota policy rule show -vserver vserver1 -volume cifs_data_001 -type user -policy-name default -target TESTLAB\N2003909
There are no entries matching your query.

Hope that helps

/Matt

If this post resolved your issue, help others by selecting ACCEPT AS SOLUTION or adding a KUDO.

Wolfgang_Ratzka · ‎2024-07-10

I have continued looking for a solution myself and found it easier to use Invoke-NcSsh together with the volume quota policy rule commands, as this requires less "glue" and fits into the original script quite nicely. (Well, the quota in my first attempt where too generous by a factor of 1024...)

I will, however, look into your suggestions as I am not sure that we shouldn't move from PowerShell Toolkit to REST-API or REST-CLI.

Thanks a lot

Wolfgang

mbeattie · ‎2024-07-10

Hi Wolfgang

There is also an "Invoke-NcCli" CmdLet that can be used as an alternative to SSH which uses the private REST CLI but I prefer to define my own functions. Anyway i did ping the PSTK developers to take a look into the issues with these CmdLet's so hopefully it gets fixed in a future release.

/Matt

If this post resolved your issue, help others by selecting ACCEPT AS SOLUTION or adding a KUDO.

mbeattie · ‎2024-07-10

Hi Wolfgang,

Note that you can also use the "Invoke-NcCli" CmdLet (which can be used as an alternative to SSH). Essentially this will invoke the private REST CLI (so if there's ever an issue using a CmdLet or a command that has no associated CmdLet then you can resort to this method). Here's an example of creating a Quota:

Connect-NcController -Name 192.168.100.2 -Credential $credential | Out-Null 
Invoke-NcCli -Command "volume quota policy rule create" -Body '{"type":  "user", "vserver":  "vserver1", "policy-name":  "default", "target":  "TESTLAB\\N2003905", "qtree":  "",  "disk-limit":  "5GB", "volume":  "cifs_data_001"}'

This might be an option for you if you prefer not to add custom functions to your code to invoke the REST API and just use native CmdLet's within the PSTK.

Hope that helps

/Matt

If this post resolved your issue, help others by selecting ACCEPT AS SOLUTION or adding a KUDO.

saharsh · ‎2024-07-15

In Rest QuotaTarget is No Rest Equivalent

quota-target

NO REST EQUIVALENT

The REST API uses one of the three separate fields instead of quota-target: users/group/qtree.

saharsh · ‎2024-07-15

For Remove-NcQuota could you please share the debug logs with us on ng-ontap-pstk-rest DL it will help us to identify the issue more efficiently.

You can enable the debug logs using Set-NaToolkitConfiguration debug command.