OCUM - how to extract all active events?

[rome]

Hello, dear All 🙂

I am a newbie in the NetApp world and we are moving from EMC to NetApp.

I would like to know everyting about issues/problems with our systems and I need to transfer all (possible) events to our CA NIMSOFT Dashboards. So, I have OCUM installed on Windows 2012 R2 server. Everything is working fine, and I want to

- Connect to OCUM via PowerShell and read events (preffered)

- Connect to OCUM's REST API and read events (via PowerShell too, I hope)

Any other idea like sending e-mails or SNMP traps is "not my cup of tea" so....

So. The first item in my list is seems difficult. I did not find any PS module for OCUM 9.5

The second is much better, but... 

I need to know how can I create a link that contains REST API command AND user name and password.

Because the main OCUM's web page is not providing a form for previding user name and password, I cannot connect to OCUM via "stantard" approach.

Can somebody help me?

[gaurav_verma]

I am not a PS guy but I guess this should help. 

Curl:

curl -u username:password -X GET --header 'Accept: application/vnd.netapp.object.inventory.hal+json' 'https://<OCUM_Server>/rest/events?limit=20'

PS:

add-type @"
using System.Net;
using System.Security.Cryptography.X509Certificates;
public class TrustAllCertsPolicy : ICertificatePolicy {
public bool CheckValidationResult(
ServicePoint srvPoint, X509Certificate certificate,
WebRequest request, int certificateProblem) {
return true;
}
}
"@
[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy

<# Above code will Ignore Certificates #>

$Username = 'Username'
$Password = 'Password'
$pass = ConvertTo-SecureString -AsPlainText $Password -Force
$Cred = New-Object System.Management.Automation.PSCredential($Username,$pass)

$response = Invoke-RestMethod 'https://nc2twnaocum01/rest/events?limit=20' -Credential $Cred -Method Get
echo $response

[rome]

Hm....

I got an error 401 - Unauthorized, despite I provided credentials and they are OK when I log in via web.

Any suggestions? 

[gaurav_verma]

@rome  I suggest to test it first with Postman (https://www.getpostman.com/). 

If you are using your user name with domain (domain\username), I suggest do not use domain name as OCUM does not allow domain name some time. 

[rome · Accepted solution]

This is THE solution.

Sorry, without comments but it is simle enough:

cls


function Convert-FromUnixDate ($UnixDate)
{
    [timezone]::CurrentTimeZone.ToLocalTime(([datetime]'1/1/1970').AddMilliseconds($UnixDate))
}


Add-Type @"
using System.Net;
using System.Security.Cryptography.X509Certificates;
public class TrustAllCertsPolicy : ICertificatePolicy {
public bool CheckValidationResult(
ServicePoint srvPoint, X509Certificate certificate,
WebRequest request, int certificateProblem) {
return true;
}
}
"@
$AllProtocols = [System.Net.SecurityProtocolType]'Tls12'
[System.Net.ServicePointManager]::SecurityProtocol = $AllProtocols
[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy


Import-Module SQLPS -DisableNameChecking

$DatabaseName = 'dbname'
$DatabaseUserName = 'username'
$DatabasePassword = 'password'
$DatabaseServerInstance = 'server.domain.org\dom,1436'


$username="monitoring"
$pass = "monitoring"
$password = ConvertTo-SecureString -AsPlainText $pass -Force


$bytes = [System.Text.Encoding]::UTF8.GetBytes("$username`:$pass")
$encodedCredentials = [System.Convert]::ToBase64String($bytes)
$headers = @{
"Authorization" = "Basic " + $encodedCredentials
"Accept" = "application/vnd.netapp.object.inventory.hal+json"
}

$events = Invoke-RestMethod  -Headers $headers -Uri https://ocum.domain.org/rest/events?limit=9999999


$query = 'DELETE FROM [dbname].[dbo].[NetApp]'
Invoke-Sqlcmd -Database $DatabaseName -Username $DatabaseUserName -Password $DatabasePassword -Query $Query -ServerInstance $DatabaseServerInstance


foreach ($event in $events._embedded."netapp:eventDtoList")
{
    Write-Host $event.name - $event.conditionMessage
    
    $query = "INSERT INTO [dbname].[dbo].[NetApp] 
    (
    objectId,
    name, 
    conditionMessage, 
    timestamp, 
    severity, 
    impactArea, 
    impactLevel, 
    sourceFullName, 
    state, 
    resolvedTimestamp, 
    resolvedBy,
    acknowledgedTimestamp,
    acknowledgedBy,
    sourceResourceType,
    daysOutstanding
    ) 
    values   
    (
    '$($event.objectId)',
    '$($event.name)',
    '$($event.conditionMessage -replace "'", """)',
    '$(Convert-FromUnixDate ($event.timestamp))',
    '$($event.severity)',
    '$($event.impactArea)',
    '$($event.impactLevel)',
    '$($event.sourceFullName)',
    '$($event.state)',
    '$(Convert-FromUnixDate ($event.resolvedTimestamp))',
    '$($event.resolvedBy)',
    '$(Convert-FromUnixDate ($event.acknowledgedTimestamp))',
    '$($event.acknowledgedBy)',
    '$($event.sourceResourceType)',
    '$($event.daysOutstanding)'
    )
    "
    #Write-Host $query
    Write-Host
    Invoke-Sqlcmd -Database $DatabaseName -Username $DatabaseUserName -Password $DatabasePassword -Query $Query -ServerInstance $DatabaseServerInstance

}


$username = 'user'
$pass = 'password'
$password = ConvertTo-SecureString ($pass) -AsPlainText -Force
$cred = New-Object System.Management.Automation.PSCredential ($username, $password)
Connect-NcController -Credential $cred -name 172.17.114.150

$volumes = Get-NcVol
foreach ($volume in $volumes)
{    
    $lastSnapshotDate = Get-NcSnapshot -Volume $volume | Sort-Object Created | Select-Object Created, Name -Last 1 
    
    $dateNow = (Get-Date).AddDays(0)
    
    $timeSpan = 1

    if ($volume.Name -eq 'data_CIFS_volume' -or $volume.Name -eq 'nvrvideo_CIFS_volume')    {        $timeSpan = 7    }

       
    if ((New-TimeSpan -Start $lastSnapshotDate.Created -End $dateNow).Days -gt $timeSpan)
    {                
        $lastSnaphotErrorString = "The last snapshot of the volume `"$volume`" was created at $(($lastSnapshotDate.Created).ToString(`"dd/MM/yyyy,`")) $((New-TimeSpan -Start $lastSnapshotDate.Created -End $dateNow).Days) days ago"
        Write-host $lastSnaphotErrorString 

        $query = "INSERT INTO [dbname].[dbo].[NetApp] 
        (
        objectId,
        name, 
        conditionMessage, 
        timestamp, 
        severity, 
        impactArea, 
        impactLevel, 
        sourceFullName, 
        state, 
        resolvedTimestamp, 
        resolvedBy,
        acknowledgedTimestamp,
        acknowledgedBy,
        sourceResourceType,
        daysOutstanding
        ) 
        values   
        (
        '-1',
        'Last snapshot',
        '$lastSnaphotErrorString',
        null,
        'error',
        null,
        null,
        '$volume',
        'NEW',
        null,
        null,
        null,
        null,
        null,
        null
        )
        "    
        Invoke-Sqlcmd -Database $DatabaseName -Username $DatabaseUserName -Password $DatabasePassword -Query $Query -ServerInstance $DatabaseServerInstance
    } 
    
}


<#




SELECT objectId , 'NetApp ' + sourceFullName + ': ' + name + ': ' + conditionMessage as message FROM [dbname].[dbo].[NetApp] where state = 'NEW' and severity <> 'information'


----------------------------------------------------------------------------------------------------------

  USE [dbname]
GO

/****** Object:  Table [dbo].[NetApp]    Script Date: 3/26/2019 12:22:40 PM ******/
SET ANSI_NULLS ON
GO

SET QUOTED_IDENTIFIER ON
GO

CREATE TABLE [dbo].[NetApp](
	[objectId] [int] NULL,
	[name] [nvarchar](max) NULL,
	[conditionMessage] [nvarchar](max) NULL,
	[timestamp] [datetime] NULL,
	[severity] [nchar](15) NULL,
	[impactArea] [nchar](15) NULL,
	[impactLevel] [nchar](10) NULL,
	[sourceFullName] [nvarchar](max) NULL,
	[state] [nvarchar](50) NULL,
	[resolvedTimestamp] [datetime] NULL,
	[resolvedBy] [nvarchar](50) NULL,
	[acknowledgedTimestamp] [datetime] NULL,
	[acknowledgedBy] [nvarchar](50) NULL,
	[sourceResourceType] [nvarchar](50) NULL,
	[daysOutstanding] [int] NULL
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]

GO
----------------------------------------------------------------------------------------------------------


#>

[Jim_Robertson]

---

@rome wrote:

$username="monitoring"
$pass = "monitoring"
$password = ConvertTo-SecureString -AsPlainText $pass -Force

$bytes = [System.Text.Encoding]::UTF8.GetBytes("$username`:$pass")
$encodedCredentials = [System.Convert]::ToBase64String($bytes)

$headers = @{
"Authorization" = "Basic " + $encodedCredentials
"Accept" = "application/vnd.netapp.object.inventory.hal+json"
}

---

Thanks for posting this, this was a huge help.  I was able to get it working with the code you have posted, but I was wondering if there was a way to do this without having the password in clear text in the code?  I've tried putting the password into a secure file.  i.e.:

Read-Host -assecurestring -Prompt 'Password' | convertfrom-securestring | out-file C:\cred.txt
$pass = get-content C:\cred.txt | convertto-securestring

But, when I convert it to a secure string, it messes up the encoding.  I want to have this run as an automated process, and there is no way that I can have a password hard coded into it.

Thanks!

[mbeattie]

Hi Jim,

As an "example" you can use a PowerShell credential object as an input paramater.

If invoking an OCUM REST API from WFA you can use:

$credentials = Get-WfaCredentials $ocumIp

If you just want to run a PowerShell script that accepts a Credential object as an input paramater, the following example enables you to query OCUM events for the past X hours:

Param(
   [Parameter(Mandatory=$True, HelpMessage="The OCUM server's hostname or IP Address")]   
   [String]$Hostname,
   [Parameter(Mandatory=$True, HelpMessage="The number of hours to enumerate OCUM events for")]   
   [Int]$EventRangeHours,
   [Parameter(Mandatory=$True, HelpMessage="The OCUM credentials used to authenticate the REST API request")]
   [System.Management.Automation.PSCredential]$Credentials
)
Add-Type @"
   using System.Net;
   using System.Security.Cryptography.X509Certificates;
   public class TrustAllCertsPolicy : ICertificatePolicy {
   public bool CheckValidationResult(
   ServicePoint srvPoint, X509Certificate certificate,
   WebRequest request, int certificateProblem) {
      return true;
   }
}
"@
[String]$username = $Credentials.GetNetworkCredential().UserName
[String]$password = $Credentials.GetNetworkCredential().Password
[System.Net.ServicePointManager]::SecurityProtocol  = [System.Net.SecurityProtocolType]'Tls12'
[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
$encodedCredentials = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes("$username`:$password"))
$headers = @{
   "Authorization" = "Basic " + $encodedCredentials
   "Accept"        = "application/vnd.netapp.object.inventory.hal+json"
}
[String]$uri = $("https://$HostName/rest/events?triggeredTime=LAST`_" + $EventRangeHours + "h")
Try{
   $events = Invoke-RestMethod -Headers $headers -Uri $uri -ErrorAction Stop
   Write-Host "Enumerated OCUM events within the past $EventRangeHours hours using URI ""$uri"""
}Catch{
   Write-Warning -Message $("Failed enumerating OCUM Events within the past $EventRangeHours hours using URI ""$uri"". Error " + $_.Exception.Message)
   Break;
}
ForEach($event In $events._embedded.'netapp:eventDtoList'){
   $event
}

Hope that helps?

/Matt

[Jim_Robertson]

Thank you, @mbeattie !  That is exactly what I needed.  Took me a little while to figure out what you were doing, but I think the 

.GetNetworkCredential().

was the piece I was missing. 

[mbeattie]

Hi Jim,

Yes that's it, use the .GetNetworkCredential() method of the [System.Management.Automation.PSCredential]. If you don't want to pass credentials as an input paramater (IE you running your script on a server as a scheduled task then you can cache and encrypt the credentials either locally to a file or to the registry then decrypt the credentials using the Windows Data Protection API). I can post an example if that's what you are trying to do?

/Matt

[Jim_Robertson]

@mbeattie , yep, that is exactly what I'm trying to do.  I got it working, but if you have a different way of doing it, I would be interested in seeing it.  For the record, I'm just trying to pull a list of clusters that are in OCUM.  I thought it would be a good way to pull a list of controllers to run PS scripts against, since all active controllers should always be added to OCUM once they are in production.

So, I first created a secure password file:

(Get-Credential).Password | ConvertFrom-SecureString | Out-File "C:\OCUM-Password.txt"

Then I create a PS object from that in line 4.  Then use the Get-NetCredential to pull the password from the PS Credentials like in your example.

$url = "https://ocum-server-dns/rest/clusters?limit=200"
$User = "ocum-admin"
$File = "C:\OCUM-Password.txt"
$Credentials = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $User, (Get-Content $File | ConvertTo-SecureString)

# Ignore certificate requests from OCUM server
Add-Type @"
   using System.Net;
   using System.Security.Cryptography.X509Certificates;
   public class TrustAllCertsPolicy : ICertificatePolicy {
   public bool CheckValidationResult(
   ServicePoint srvPoint, X509Certificate certificate,
   WebRequest request, int certificateProblem) {
      return true;
   }
}
"@
[String]$username = $Credentials.GetNetworkCredential().UserName
[String]$password = $Credentials.GetNetworkCredential().Password

[System.Net.ServicePointManager]::SecurityProtocol  = [System.Net.SecurityProtocolType]'Tls12'
[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
$encodedCredentials = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes("$username`:$password"))
$headers = @{
   "Authorization" = "Basic " + $encodedCredentials
   "Accept"        = "application/vnd.netapp.object.inventory.hal+json"
}

$clusters = Invoke-RestMethod  -Headers $headers -Uri $url

foreach ($cluster in $clusters._embedded."netapp:clusterInventoryList")
{
    Write-Host $cluster.cluster.label
}

The only downside I am seeing right now is that the password file will only be readable by me on the server that I created it on (at least that is my understanding from what I've read).  If I want others to be able to read it, it's looking like I will have to setup a secure key for the file.

Thanks again!

[mbeattie]

Hi Jim,

There are a few options:

• Encrypt credentials to a file
• Encrypt credentials to the registry
• Cache credentials using "Add-NcCredential" (See Get-Help Add-NcCredentials)

Here is an example using the cached credential method demonstrating how to add and retrieve credentials from the cache.

Param(
   [Parameter(Mandatory=$True, HelpMessage="The hostname, IP Address or FQDN of the system to cache credentials for")]
   [String]$HostName
)
#'------------------------------------------------------------------------------
#'Add credentials to the cache.
#'------------------------------------------------------------------------------
Import-Module -Name DataONTAP -ErrorAction SilentlyContinue
Try{
   Add-NcCredential -Name $HostName -Credential $(Get-Credential) -ErrorAction Stop
   Write-Host "Added Credentials for ""$HostName"""
}Catch{
   Write-Warning -Message $("Failed Adding Credentials for ""$HostName"". Error " + $_.Exception.Message)
   Break;
}
#'------------------------------------------------------------------------------
#'Enumerate the credentials from the cache.
#'------------------------------------------------------------------------------
Try{
   $credentials = Get-NcCredential -Name $HostName -ErrorAction Stop
}Catch{
   Write-Warning -Message $("Failed enumerating Credentials for ""$HostName"". Error " + $_.Exception.Message)
   Break;
}
[String]$username = $credentials.Credential.GetNetworkCredential().UserName
[String]$password = $credentials.Credential.GetNetworkCredential().Password
Write-Host "Username`: $username. Password`: $password"
#'------------------------------------------------------------------------------

Encrypting credentials to the registry is slightly more abstracted and offers some advantage to encrypting to a file (IE it avoids risk of the credential file being accidently deleted and your automation potentially failing as a result of a missing credential file). Caching credentials using the PSTK is the simplest method. If you don't have the Data ONTAP powershell toolkit (PSTK) you can download it here:

https://mysupport.netapp.com/tools/info/ECMLP2310788I.html?productID=61926&pcfContentID=ECMLP2310788

Hope that's useful.

/Matt

[rome]

Wow! Thank you very much!

Tomorrow I'll check this 🙂

[Tas]

I would look at CA's plugins.  You do not necessarily need to read OCUM, as that will give you RBAC and Performance metrics.  NetApp can trap errors, however, the best solution is to use the CA plugin for ONTAP.

So here is what I think you should look at in order of preference for events, if you can't get the plugin:

1.  Use OCUM's Events Dashboard;  it is a very good Event notifier, although only for NetApp

2.  Use email in OCUM to send event alerts to the CA product

3. Trying to use the rest will be very difficult.

[rome]

Thank you very much for your answer.

Unfortunately, CA's netapp probe leaves to be desired (like vmware too...)

Using PowerCLI I build a little script that moves ESXi alarms to the SQL table and then I show them on the dashboards.

Using emails is not a solution because we (do not laught!) do not use internal email server and we are "black site"

Using OCUM's dashboard is beautiful solution for me, but not for all our team. And, of course, we would like to stare on one dashbaord instead of tousands 🙂

So, for me if much simpler to use PowerShell and bring events/alarms streight to dashboards.