90day eval AltaVault - Not allowing to evict or clear out cache of backup cancelled.

[taniaeortega]

We installed AVA-v2 / v4.4#4591830 on ESX platform and configured NFS which our NetBackup server has mounted and performed a backup test.

(1) How to evict or delete that which is in Cache?

Because of bandwidth issues, we needed to immediately cancel the backup which we did via NetBackup, yet AltaVault continued to transfer it to AWS Glacier.

We then attempted a few things we read here, i.e. delete all files seen via the NFS mounts, Supspend Replication via Cloud Settings and even Stopped Storage Optimization Service.

Yet, once we restart the services and replication it starts up again and starts sending the data which we no longer see via NFS.

How can we first see what remains queued, then  flush / evict / delete it?

We have tried various datastore commands (even 'enable' the shell) and it doesn't recognize them? 

Is it because this is an eval license?

(2)  The primary network for management was configured via a different vLAN then the other three (3) IPv4 interfaces.  The network to AWS Glacier is off e0c, yet we see of the traffic going through the primary interface.  

Default IPv4 gateway was set to the same network as the primary vLAN and individual routes were added for each of the other interfaces, i.e.:

primary IP: 10.0.123.123 with default IPv4 gateway 10.0.123.254

e0c IP:  10.0.124.123 with these routes:

    Destination   Subnet Mask      Gateway          Interface     Status

    10.0.124.0    255.255.255.0    0.0.0.0             e0c    User Configured

     10.0.124.0   255.255.255.0    10.0.124.254   e0c   User Configured

Why would then the traffic going to AWS Glacier where S3 intial bucket is off the 124 still be going via the 123?

Is it because this is an evaluation license?

[chriswong]

Hi,

Here are the answers to address the questions:

1. You can use the command line to change the eviction threshold, which reduces the amount of data held on cache. SSH to the appliance, and issue:
enable

configure terminal

datastore eviction threshold XX (where XX is a % value, ie 50 = 50%)

Refer to the AltaVault documentation (CLI guide) for details on the command.

AltaVault will perform operations in order based on commands received. For example, it will perform subsequent delete operations after the backup job is replicated (i.e. the data must be replicated before AltaVault issues deletes). This is in order to ensure consistency of operations and data.

If you want to effectively "reset" AltaVault data and cloud contents, you can issue these commands instead to forcefully clear the cache and cloud contents:

enable

configure terminal

datastore format all

(note that once you issue this last command and confirm, your data is not recoverable).

2. Are you in a flat subnet? AltaVault interfaces should be on separate subnets, as recommended in TR-4416. If you have a flat subnet, the network layer may select the interface differently than that specified by your settings. We typically see this type of behavior in flat networks.

[taniaeortega]

Hi Chris,

Thanks for your lead and help.

(1) Before performing "datastore format all" we looked this up and it "Formats and deletes all data cached in AltaVault and in the cloud provider."

When it states "in the cloud provider" is it ONLY specific to the backup that was cancelled?

Is there a granualar view of what actually remains in the local AltaVault cache?

We could have a local cache of 8Tb and have to cancel a small 2Tb backup therefore formatting all seems so drastic.

(2) Regards the network type we have, AltaVault is in it's own subnet and we have set default gateways for each one as outlined earlier.  The primary via the vlan 123, the e0c via vlan 124 which is the one traffic which is routed via our switch to AWS Glacier.  Yet the routing is not working.

Thanks in advance.

Tania

[chriswong]

Hi Tania,

To answer your questions:

1. Yes, AltaVault only deletes data for the cloud provider (i.e. AWS) and bucket (as shown from the Cloud Settings page) that AltaVault is configured against. No other data in the cloud provider is removed. Remember the 'all' flag will clear the cache contents as well - this means, the AltaVault will be equivalent to 'day1' state after you restart the service following this action and will have no received data stored.

2. AltaVault 4.4 provides source based routing capabilities, for which you can assign different gateways for each interface. You can use the 'ip data-gateway' 'and 'ip data route' commands to establish the gatewy and routes for an inteface. 

Regards,

Christopher