GUI Internal scheduler strangeness

[f_duranti]

Hi, I'm doing some tests with the internal scheduler and it seems that it will do something strange in some  circumstance.

It seems that if I start the gui but don't login to the browser interface the scheduling do not start so if I reboot the backup server and don't login to the gui I will get no backup job starting (not good at all).

More then this if I connect to the gui and there's some job that should have started in the past (when the gui scheduler was shutted down) it will start the backup job immediately and this is also not good because I have for example some oracle database that will do offline backup (shutting down and restarting oracle) and in this way It could happen that I get the database shutted down during working hour.

This is the step I've done to reproduce it (I have 1 configuration file only):

1) I've scheduled a backup to happen at 12.59.

2) At 12.56 and I shutted down the gui and started back it, waited until 13.03 and the backup was not run.

3) At this time I logged on the gui and as soon as I logged into the gui the backup started.

Is this the normal behaviour, a bug or there's something I've misconfigured ?

The snapcreator is version 3.4.0 installed under redhat linux 5.

Thanks

Francesco

[ktenzer · Accepted solution]

Hi Francesco,

Yes I am sorry to say this is a bug in both SC 3.3.0 and SC 3.4.0 (but wasnt reported or found till after release of SC 3.4.0). BURT 516935 has been created for this issue. The problem is that we use Quartz for our scheduler and we only start the Quartz process when someone logs or attempts to log into the GUI. This BURT has been fixed (starting Quartz when Jetty / GUI starts not when someone tries to log-in) and we will as far as I know be releasing a 3.4p1 (no confirmed date yet, we want to wait a few weeks to see what other bugs surface) with this fix as well as some others. The release notes are also being updated with this bug and some other's that were reported.

WORKAROUND:

1. After restarting GUI, log-in once so scheduler starts

2. Dont use SC Internal Scheduler instead use crontab (UNIX)

3. Dont use SC Internal Scheduler instead use task-manager (Windows)

Regards,

Keith

[f_duranti]

Hi Keith, thanks for the fast answer. We had the problem with 3.3.0 but we didn't check what was causing it because we was not using it, just testing so we never discovered that the problem was that it was needed to login into the product.

Anything about the other "issue" regarding the start of all past jobs? It seems not correct or at least I think it could be better to have a config option/startup option to define if when the gui start it should start all the past jobs or should wait until the next run cycle...

Do you think this can be solved/implemented too?

Regarding the workaround, so you think it's possible in some way to access a webpage on the server using something like wget to login automatically via script?

Regards

Francesco

[ktenzer]

Hi Francesco,

As for the GUI starting past jobs immediately after logging in. Does this happen with CRON style or normal style jobs? Do all jobs start immediately?

You could script the GUI to login or fake a login, all you need to do is attempt to login, I havent tried this but I would give it a shot. If wget works that could be a workaround.

Regards,

Keith

[f_duranti]

Hi Keith,

As for the GUI starting past jobs immediately after logging in. Does this happen with CRON style or normal style jobs? Do all jobs start immediately? 

it happen scheduling with frequence Daily and also with cron.

To test with cron I've configured it with a date/time in the future:"0 32 17 ? *  *", stopped/started the gui, waited until past of 17.32.00 and then logged on the gui.

As soon as I logged on the gui the backup started.

I've also some problem with "once".... I've just changed the schedule from cron to "once" and the backup started immediately.

Regards

Francesco

[Arora_Kapil]

Hi Francesco and Keith,

I looked into the code and found that "running immideately after start" is a default setting for these jobs and can only be corrected by a code change.

I have raised  BURT 518258 for this and we will be fixing this in the next release.

Please let me know if we should provide following two options or just set the functionality to 2, in case of misfire (schedular was down at the scheduled time) :

1) Run now

2) Don't do anything and run on next schedule only.

About updating a cron job to once:

If you selected today's date then I think it is working fine, if not I will try and recreate that too.

Please let me know.

Thanks,

Kapil

[f_duranti]

Hi Kapil,

In my opinion the better will be, if possible, an option on every job that will allow or deny (just a checkbox) the automatic run of the job if it was not run at the last scheduled time (in case of a backup server failure).

With an option like this it will be possible to decide how to run each job in case of failure so, for example, offline database jobs will not be run while online database jobs or "high priority" backup jobs can be run just after the server restart in case of failure.

Regards

Francesco

[Arora_Kapil]

Thanks Fransesco, really appreciate your feedback.

We will provide this option as part of the fix.

Please feel free to post any suggestions or comments you have.

~Kapil

[f_duranti]

Hi Kapil, Thanks really much

I hope the fix will be out soon we would like to start configuring all the backup of our dev/test Oracle Database server with SC3.4.0 but we do nightly offline backup so I would not like to risk that if the server restart for any reason I will get all the database shutted down during the day...

Regards and thanks again

Francesco

[cferbrache]

What is the work around for this. I can't have my clone jobs kicking off in the middle of the day if I log into the GUI. Do I just schedule them through cron and not use the scheduler?

[ktenzer]

Don't use GUI scheduler is only workaround at this point

We are providing fix for GUI scheduler not restarting automatically without logging-in after restart of GUI and providing option to handle what to do with jobs after restart.

It looks like the GUI not automatically restarting fix will go into a 3.4p1 (sometime in semptember is plan) and the option for handling jobs after restart in the next major release in december.

Keith

[f_duranti]

The time to resolve those 2 issue seems really long ... 2 more months for the first and 5 months for the job restarting options (without which restarting snapmanager gui and working with offline database backup will result in all database going down for backup ...).

No way to try to push those 2 issue at the developer attention for a hotfix release or something like that?

The product is really nice but those 2 issue, at least for my database administrator are a no-go for going in production with the product...

Francesco

[ktenzer]

The more NGS cases the faster the fix can be made available, right now the scheduler issues arent viewed as super critical because you can use cron, task manager, or any other scheduling system to schedule SC. Though I agree would be really nice to provide a patch quicker.

It's not the coding, it is the process behind products that always takes the longest time. 

Keith

[f_duranti]

Thanks for the suggestion.

Francesco

[f_duranti]

Hi Keith, just some help to open the case with NGS... on NOW support site when I go to open a case I should put a serial number or choose between one of the serial number we have registered... should I choose a storage to open the case for Snap Creator or what?