Data Backup and Recovery

Data Broker Scripting

amayacitta
2,843 Views

Hi all,

 

Has anyone any experience getting scripts to call from the data broker policy- aka snapcentre plugin for vSphere? I have a perl script setup to do the same thing as this script in PowerShell for windows. If I run it within the data broker itself, it works fine, but calling it from the policy fails.

 

Documentation is a little sparse, it just says it needs to be perl (which it is) and to call it, I'm putting an absolute path in the pre-post area. Documentation is here for reference: https://library.netapp.com/ecm/ecm_download_file/ECMLP2861085

 

The path I'm putting in the policy is /home/scripts/dc1_srm_mgmt_01_volumes.pl

 

If I run "perl /home/scripts/dc1_srm_mgmt_01_volumes.pl" locally on the data broker it works just fine. I know with the PowerShell script I ran for windows for some reason it used the SYSTEM account not the service account for the agent. I'm wondering what account it uses to run the perl script on the data broker appliance? It might be a permission thing, I've tried chmod 777 for fun but no joy. Not sure about the owner, it's currently the diag user. Perhaps I should change this to root or maint?

 

Any help would be appreicated. Oh and yes please develop native cascade support, fan-out is too limited unless you've got enough bandwidth to run a mirror and a second vault across your line 😞

2 REPLIES 2

amayacitta
2,752 Views

It's easy when you know how huh? I commented the top of my script which broke the possibility to execute the script with ./scriptexample.pl. When I was doing it manually I was running 'perl scriptexample.pl'

 

I moved #!/usr/bin/perl back to line 1 and also made sure the file had execute rights with chmod, and was owned by root with chown root:root scriptexample.pl

 

It now calls fine through data broker. Hope that helps someone else who is learning Perl fast lol.

 

Just need to fix the BACKUP_PHASE environmental variable. It passes down as POST_BACKUP twice, so I can't separate the two! Known bug though I think.

 

amayacitta
2,627 Views

I think it would be worth putting some basic templates of perl scripts within the documentation. I'm sure this could be done better, however it certainly works. For those that want a template perl script for the data broker, here is one. This adds support for cascaded vault, which isnt natively supported in the GUI. You can only inject labels with fan out using the GUI.

 

This will do the following:

  1. Inject labels into daily, weekly, monthly snapshots at source
  2. Perform snapmirror update to destination system
  3. The vault schedule on the destination NetApp can then happily use the labels for longer term retention.
  4. I never bothered to do the vault update as well in the script, just stagger the vault schedule far enough in advance for the first mirror to complete

 

#!/usr/bin/perl

# SRM Management 01 Volumes Perl Post Backup Script
# Modifies snapmirror-labels in cascade situation
# SnapCenter only supports fan out and won't add labels at source
# Job is ran from data broker job as root so ensure owner and permissions are set to allow root access
# Connect to console of databroker and enable diag account to create script files and directories
# Cascade update is done outside of this script by cron destination cluster
# Jobs need to be adequately staggered to allow mirror update to complete before hand

# Pre-reqs for this script to run successfully are commented below
# Data broker needs the following lines added to /etc/apt/sources.list
#
# echo "deb http://deb.debian.org/debian stretch main" | sudo tee -a /etc/apt/sources.list
# echo "deb-src http://deb.debian.org/debian stretch main" | sudo tee -a /etc/apt/sources.list
#
# Run the following lines in sequence to install OpenSSH module into perl environment
# Data broker requires internet access for these commands run run
# sudo apt-get update -y
# sudo apt-get install -y apt-utils
# sudo apt-get install -y libnet-openssh-perl
# sudo cpan -f install Net::OpenSSH
#
# Amayacitta Gill - Simplify IT

use strict;
use warnings;
use Net::OpenSSH;

# Grab the time and place in variable for log file
my @now = localtime();
my $timestamp = sprintf("%04d%02d%02d%02d%02d%02d",
$now[5]+1900, $now[4]+1, $now[3],
$now[2], $now[1], $now[0]);

# Environment variable will be set by SnapCentre to pickup PRE or POST jobs through this script
# These will not work until data broker 1.0.1D2 (or later) check the below URL for availability
# Until then this script will end up running twice as POST_BACKUP is always shown for the BACKUP_PHASE env variable
# https://mysupport.netapp.com/products/netapp_databroker/1.0.1D2/index.html
# Use if statement to run PRE or POST according
# $ENV{'BACKUP_PHASE'} = 'PRE_BACKUP';
# $ENV{'BACKUP_PHASE') = 'POST_BACKUP';

# Variables
# If a password contains an @ symbol the symbol must be preceded by a \ otherwise it's mistaken as an array
my $timeout  = "60";
my $postlogfilename = "post_srm_mgmt_01_volumes$timestamp.log";
my $sourcehostname = "DC1PNETAPP01.xxx.xxx";
my $sourceusername = "snapcentre";
my $sourcepassword = "xxxxx";
my $desthostname = "DC3PNETAPP01.xxx.xxx";
my $destusername = "snapcentre";
my $destpassword = "xxxxx";
my $smupdate = "snapmirror update *srm_mgmt_01*";
my $daily = "vol snapshot modify -volume *srm_mgmt_01* -snapshot *daily* -snapmirror-label daily -vserver DC1VSVM01";
my $weekly = "vol snapshot modify -volume *srm_mgmt_01* -snapshot *weekly* -snapmirror-label weekly -vserver DC1VSVM01";
my $monthly = "vol snapshot modify -volume *srm_mgmt_01* -snapshot *monthly* -snapmirror-label monthly -vserver DC1VSVM01";

# Test code for BACKUP_NAME as this is empty on PRE and populated on POST
# On testing this variable is not populated for either job runs!
# If BACKUP_NAME is not empty or failed then run post-backup code
#if ($ENV{'BACKUP_NAME'} ne '' && ne 'BACKUP_FAILED') {

# Run only when the BACKUP_PHASE is POST
if ($ENV{BACKUP_PHASE} = 'POST_BACKUP') {

# Start Logging
open (STDOUT, "| tee -ai /home/scripts/logs/$postlogfilename");

# Display current BACKUP_BAME variable and user script is running as for debugging
print "Running Post-Backup Code...\n";
print "BACKUP_NAME is $ENV{'BACKUP_NAME'}\n";
print "This script is running as user ";
print scalar getpwuid $<;
print "\n\n"; 

# Source connection details for OpenSSH Perl Module with error check
my $sourcessh = Net::OpenSSH->new(
        $sourcehostname,
        user        => $sourceusername,
        password    => $sourcepassword,
        timeout     => $timeout,
        master_opts => [ -o => "StrictHostKeyChecking=no" ]
);
$sourcessh->error and die "Unable to connect to remote host: " . $sourcessh->error;

# Modify source snapshots with snapmirror labels these are to be used later for vault cascade
# Comment out labels that are not required
$sourcessh->system("$daily");
#$sourcessh->system("$weekly");
#$sourcessh->system("$monthly");

# Start snapmirror update so vault cascade has updated label
# Destination connection details for OpenSSH Perl Module with error check
my $destssh = Net::OpenSSH->new(
        $desthostname,
        user        => $destusername,
        password    => $destpassword,
        timeout     => $timeout,
        master_opts => [ -o => "StrictHostKeyChecking=no" ]
);
$destssh->error and die "Unable to connect to remote host: " . $destssh->error;

# Run comamnds based on variables above
# Will perform snap mirror update to destination
print "Perforning snapmirror update of volumes...\n";
$destssh->system("$smupdate");

# Remove logs older than 30 days
print "Removing log files older than 30 days...\n";
system("find /home/scripts/logs/post** -type f -mtime +30 -print -exec rm {} '\;'");

}

Enjoy!

Public