Discovery problem with snap creator and kvm [ Edited ]

[#1]

Hi,

I'm trying to use snap creator 4.1 to backup our kvm vm images. Image files are in qcow2 format and resides on a nfs volume that is mounted on kvm hosts.

I have installed snapcreator agent on the kvm host and configured a backup profile. When I try to backup the operation hangs for some minutes at this point:

########## Application Discovery ##########

[2014-01-29 09:28:45,180] INFO: Performing discovery on : kvm

And then fails, here's the complete log

[2014-01-29 09:28:43,991] DEBUG: Workflow : backup started with workflow id : 23

[2014-01-29 09:28:43,991] DEBUG: Version: Snap Creator Framework 4.1.0

[2014-01-29 09:28:43,992] DEBUG: Profile: kvm

[2014-01-29 09:28:43,992] DEBUG: Config: backup_vm_img

[2014-01-29 09:28:43,992] DEBUG: Action: backup

[2014-01-29 09:28:43,992] DEBUG: Plugin: kvm

[2014-01-29 09:28:43,992] DEBUG: Policy: daily

[2014-01-29 09:28:43,992] DEBUG: Volume Name: detect

[2014-01-29 09:28:43,993] DEBUG: Snapshot Name: scsnap-daily_20140129092843

[2014-01-29 09:28:44,112] INFO: Validating policy: daily finished successfully

########## Detecting Data ONTAP mode for auto ##########

[2014-01-29 09:28:44,566] INFO: STORAGE-03031: Getting system version details of [cafaro]

[2014-01-29 09:28:44,567] DEBUG: <system-get-version/>

[2014-01-29 09:28:44,567] DEBUG: <system-get-ontapi-version/>

[2014-01-29 09:28:44,568] INFO: STORAGE-03032: Getting system version details of [cafaro] finished successfully.

########## Agent validation ##########

[2014-01-29 09:28:44,894] INFO: Agent validation completed successfully for agent 192.168.86.166:9090

########## Plugin validation ##########

[2014-01-29 09:28:44,925] INFO: Plugin validation completed successfully for plugin kvm

########## Application Discovery ##########

[2014-01-29 09:28:45,180] INFO: Performing discovery on : kvm

[2014-01-29 09:38:51,134] ERROR: [192.168.86.166:9090 (4.1.0.1)] Operation failed. Reason: Executor timed out. Make sure plugins/wrapper/wrapper binary has execute permission set. Increasing WRAPPER_TIMEOUT_IN_MSEC in agent.properties may help, too.

[2014-01-29 09:38:51,136] ERROR: [192.168.86.166:9090(4.1.0.1)] SCF-00028: Auto Discovery for plug-in [kvm] failed with error [validation failed] and exit code [102], Exiting!

[2014-01-29 09:38:51,143] DEBUG: Workflow : backup_OnFailure started with workflow id : 24

[2014-01-29 09:38:51,143] DEBUG: Version: Snap Creator Framework 4.1.0

[2014-01-29 09:38:51,143] DEBUG: Profile: kvm

[2014-01-29 09:38:51,143] DEBUG: Config: backup_vm_img

[2014-01-29 09:38:51,143] DEBUG: Action: backup_OnFailure

[2014-01-29 09:38:51,143] DEBUG: Plugin: kvm

[2014-01-29 09:38:51,144] DEBUG: Policy: daily

[2014-01-29 09:38:51,144] DEBUG: Volume Name: detect

[2014-01-29 09:38:51,144] DEBUG: Snapshot Name: scsnap-daily_20140129092843

########## Application cleanup ##########

[2014-01-29 09:38:51,789] INFO: Performing cleanup on : kvm

[2014-01-29 09:39:02,159] ERROR: [192.168.86.166:9090 (4.1.0.1)] Operation failed. Make sure plugins/wrapper/wrapper binary has execute permission set. Also try increasing WRAPPER_TIMEOUT_IN_MSEC in scAgent/etc/agent.properties

[2014-01-29 09:39:02,161] ERROR: [192.168.86.166:9090(4.1.0.1)] SCF-00074: Cleanup for plug-in [kvm] failed with error [null] and exit code [102], Exiting!

########## Agent Workflow Finalization ##########

[2014-01-29 09:39:02,170] INFO: Agent Workflow Finalization started

[2014-01-29 09:39:12,985] INFO: [192.168.86.166:9090 (4.1.0.1)] Finalized workflow with id 23

[2014-01-29 09:39:12,985] INFO: Agent Workflow Finalization finished successfully

########## Snap Creator Framework 4.1.0 failed ##########

[2014-01-29 09:39:13,028] INFO: Pre Exit commands are not defined. Skipping !

I have checked against SCF-00028 with no success.

Can you help me fix this issue?

Thanks

Il messaggio è stato modificato da: Lorenzo Conti

The only way to fix is doing this setup  APP_AUTO_DISCOVER=N  in the following config files

/SCSERVERDIR/scServer4.1.0/engine/configs/PROFILE/CONF and
/SCSERVERDIR/scServer4.1.0/engine/etc/validation/plugins/kvm.conf

Is it the right way to do it?

[#2]

Please email me a scdump (available via Action->scdump in the GUI)   My email address sivar at netapp.com

APP_AUTO_DISCOVERY=Y along with KVM_VM_MAPPING=auto:detect:detect will make the kvm plugin to discover vms.

If you don't have any VMs in the hypervisor, the discovery will fail. (or if the KVM_VM_MAPPING has any syntax error)

Do you have these parameters in your config file? (Look for the required variables)

####################

### KVM Settings ###

####################

##################################################################################################################

# KVM_RESTORE_VM_LIST - (required only during restore) List of VMs to be restored.                               #

#                       ie: RHEL5U6,RHEL5U7                                                                      #

#                                                                                                                #

# KVM_VM_MAPPING - (required) List of mapping b/w VM and its associated storage controller, lun/file path.       #

#                  This param will be updated dynamically during discovery process.                              #

#                  ie: RHEL5U6:fas3170-201-76:/vol/vol_KVM/RHEL5U6;RHLE5U7:fas3170-201-76:/vol/vol_KVM/RHEL5U7;  #

#                                                                                                                #

# KVM_VM_CONFIG_DIR - (optional) Path to the directory where all the VM's XML configuration files will be stored.#

#                     Default value: /etc/libvirt/qemu                                                           #

#                                                                                                                #

# KVM_CMD_RETRY_COUNT - (optional) The number of times the command will be re-tried when failed in KVM plug-in.  #

#                       Default value: 3                                                                         #

#                                                                                                                #

##################################################################################################################

KVM_RESTORE_VM_LIST=

KVM_VM_MAPPING=

KVM_VM_CONFIG_DIR=

KVM_CMD_RETRY_COUNT=

These are the validate parameters file content for KVM plugin

SC_AGENT_TIMEOUT=600

APP_DEFINED_RESTORE=Y

APP_AUTO_DISCOVERY=Y

APP_CONF_PERSISTENCE=Y

USE_PROXY=N

[#3]

Hi Siva,

Here are our settings:

in

scServer4.1.0/engine/etc/validation/plugins/kvm.conf

SC_AGENT_TIMEOUT=600

APP_DEFINED_RESTORE=Y

APP_AUTO_DISCOVERY=Y

APP_CONF_PERSISTENCE=Y

USE_PROXY=N

in

scServer4.1.0/engine/configs/kvm/backup_vm_img.conf

########################################################################################################################

#     Application Options                                                                                              #

########################################################################################################################

APP_NAME=kvm

APP_CONF_PERSISTENCE=Y

APP_IGNORE_ERROR=N

APP_DEFINED_RESTORE=Y

PYTHON_OPTS=

APP_DEFINED_CLONE=N

JAVA_HOME=

PYTHON_HOME=

FS_NAME=

JVM_ARGS=

PERL_OPTS=

PERL_HOME=

PLUGINS_FORWARD=

VALIDATE_VOLUMES=

APP_AUTO_DISCOVERY=Y

JAVA_CLASSPATH=

PLUGINS_RESTORE=

APP_DEFINED_BACKUP=N

APP_DEFINED_UMOUNT=N

PLUGINS_REVERSE=

APP_DEFINED_MOUNT=N

########################################################################################################################

#     Archive Options                                                                                                  #

########################################################################################################################

ARCHIVE_LOG_RECURSIVE_SEARCH=N

ARCHIVE_LOG_DIR=

ARCHIVE_LOG_EXT=

ARCHIVE_LOG_RETENTION=

ARCHIVE_LOG_ENABLE=

########################################################################################################################

#     Plug-In Parameter                                                                                                #

########################################################################################################################

KVM_SUSPENDED_VMS=null

KVM_CMD_RETRY_COUNT=3

KVM_VM_CONFIG_DIR=/etc/libvirt/qemu

KVM_VM_MAPPING=auto:detect:detect

The only way to get snapshot done is disablin auto discovery as said in my first post.

Thank you

[#4]

Thank you. I will pass this on to the KVM plugin developer and get back to you with an answer.

[#5]

Hi Lorenzo,

would you be so kind and email me scAgent/logs/agent.log? I think that would be helpful to further lock down the issue.

If you could do a quick check, do the folders (in the scAgent) plugins/wrapper/in and plugins/wrapper/out exist? The file plugins/wrapper/wrapper should have execute permission. There was one similar know issue, which has been fixed. Unfortunately, I cannot tell you exactly, when the fix will be available.

Thanks,

Clemens

[#6]

Hi Clemens,

here is an ls of ~plugins/wrapper dir

# cd /SC_41/scAgent4.1.0
# ls -l plugins/wrapper/

totale 5696

drwxr-xr-x 2 root root       6 gen 30 11:11 in

drwxr-xr-x 2 root root       6 gen 30 11:11 out

-r-xr-xr-x 1 root root 5831584 gen 16 13:06 wrapper

Regarding the agent.log file, can you give me your email address in order to send it?

Thank you

Lorenzo