Snapcreator Agent 4.1.0 refuses to start on AIX, confusion about java-version

[ADVUNIMG2]

Hello,

last week we tried to install snapcreator-agent on AIX-6.1-Servers, but without success. In the installation-guide I found that java1.6 Update 24 is required. The output from "java -version" is as follows:

--------------------------------------- snip -------------------------------------------

@:/ > /usr/java6_64/bin/java -version

java version "1.6.0"

Java(TM) SE Runtime Environment (build pap6460sr15fp1-20140110_01(SR15 FP1))

IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 AIX ppc64-64 jvmap6460sr15-20131231_180656 (JIT enabled, AOT enabled)

J9VM - 20131231_180656

JIT  - r9_20130920_46510ifx3

GC   - GA24_Java6_SR15_20131231_1152_B180656)

JCL  - 20140107_01

--------------------------------------- snap -------------------------------------------

I must admit that I cannot determine whether this is 1.6 Update 24 or not, but I'd assume that. I'ts the most recent version available from IBM. Below is the output of scWatchdog start which complains about java:

--------------------------------------- snip -------------------------------------------

@:/ >  export JAVA_HOME=/usr/java6_64

@:/ > sh -x /opt/ NetApp/scAgent4.1.0/bin/scWatchdog start

+ JAVA_XMS=-Xms32m

+ JAVA_XMX=-Xmx128m

+ JAVA_PERM=-XX:MaxPermSize=64m

+ APP_NAME=watchdog

+ + basename /opt/NetApp/scAgent4.1.0/bin/scWatchdog

APP_BASE_NAME=scWatchdog

+ MAX_FD=maximum

+ cygwin=false

+ msys=false

+ darwin=false

+ uname

+ false

+ PRG=/opt/NetApp/scAgent4.1.0/bin/scWatchdog

+ [ -h /opt/NetApp/scAgent4.1.0/bin/scWatchdog ]

+ + pwd

SAVED=/

+ dirname /opt/NetApp/scAgent4.1.0/bin/scWatchdog

+ cd /opt/NetApp/scAgent4.1.0/bin/..

+ + + pwd -P

APP_HOME=/opt/NetApp/scAgent4.1.0

+ cd /

+ + . /opt/NetApp/scAgent4.1.0/etc/agent_memory.properties

+ AGENT_MIN_MEMORY=64

+ AGENT_MAX_MEMORY=256

+ AGENT_PERMSIZE=128

+ WATCHDOG_MIN_MEMORY=32

+ WATCHDOG_MAX_MEMORY=128

+ WATCHDOG_PERMSIZE=64

+ JAVA_XMS=-Xms32m

+ JAVA_XMX=-Xmx128m

+ JAVA_PERM=-XX:MaxPermSize=64m

+ JAVA_OPTS=-Xms32m -Xmx128m -XX:MaxPermSize=64m

+ CLASSPATH=/opt/NetApp/scAgent4.1.0/lib/scAgent-4.1-core.jar

+ [ -n /usr/java6_64 ]

+ [ -x /usr/java6_64/jre/sh/java ]

+ JAVACMD=/usr/java6_64/bin/java

+ [ ! -x /usr/java6_64/bin/java ]

+ [ false = false -a false = false ]

+ + ulimit -H -n

MAX_FD_LIMIT=unlimited

+ [ 0 -eq 0 ]

+ [ maximum = maximum -o maximum = max ]

+ MAX_FD=unlimited

+ ulimit -n unlimited

+ [ 0 -ne 0 ]

+ false

+ exec /usr/java6_64/bin/java -Xms32m -Xmx128m -XX:MaxPermSize=64m -classpath /opt/NetApp/scAgent4.1.0/lib/scAgent-4.1-core.jar com.netapp.snapcreator.agent.nextgen.watchdog.external.ExternalWatchdogStarter start

Snap Creator Agent requires Java version 1.6.0_24 or greater!

Stopping watchdog...Stopping watchdog thread...Stopping threads...Done.

--------------------------------------- snap -------------------------------------------

The same happens when I switch to 32-bit-Java.

snapcreator 4.0P1 works like a charm:

--------------------------------------- snip -------------------------------------------

@:/opt/NetApp/scAgent4.0p1 > /opt/NetApp/scAgent4.0p1/bin/scAgent  sta rt

Starting scAgent:

Checking Status of scAgent:

Running on port 9090

--------------------------------------- snap -------------------------------------------

Do I have to use a different java-version or is the check too strict?

Best regards

     Martin Glora

[stanton]

The check seems to be off on AIX.  Try upgrading the java to a 1.7 version.  It has proven to give better results.

[LEIGHTONN]

Hi,

We hit the same issue in our environment. Here's the fix and explanation provided by our system admins.

Okay. Two things.

1. JAVA_HOME has to be set when running "scAgent start". It's not set anywhere in the script. I set it to /usr/java7
2. This is the issue:

In scWatchdog and scAgent-core they search two locations for the java command in JAVA_HOME:
$JAVA_HOME/bin/java
$JAVA_HOME/jre/sh/java

The second location is preceded by the comment:
# IBM's JDK on AIX uses strange locations for the executables

Which is really funny because AIX doesn't have java in either location. It needs to be searching for
$JAVA_HOME/jre/bin/java. I have no idea what version of AIX/java they were using when they decided
that java was in jre/sh.

I changed the check in scWatchdog and scAgent-core, ran scAgent start and:

[root@server1 bin]$ ./scAgent status
Checking Status of scAgent:
Watchdog: Running
Agent: Running

Thanks,

Leighton

[ADVUNIMG2]

Hi and thanks for all helpful tips!

As far as snapcreator4.0P1 ist currently working I have to arrange a date with the customer in order to try java1.7.  But we also have one AIX5.3 installation and as far as I can see http://www.ibm.com/developerworks/java/jdk/aix/service.html there ist no java1.7 for AIX5.

From my point of view JAVA_HOME is set and evaluated correctly, thats why I posted the output of "sh -x ./scWatchdog start" in the beginning. java ist found and started via "exec /usr/java6_64/bin/java -Xms32m -Xmx128m -XX:MaxPermSize=64m -classpath /opt/NetApp/scAgent4.1.0/lib/scAgent-4.1-core.jar com.netapp.snapcreator.agent.nextgen.watchdog.external.ExternalWatchdogStarter start"

BTW, can somebody confirm that IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 AIX ppc64-64... really means 1.6 Update 24?

Best regards

Martin