Tech ONTAP Blogs
Tech ONTAP Blogs
Cloud Backup has now added “Search and Restore” support for On-Premise Connector (with internet access), which means you can now manage Indexed Catalog feature on on-premise, AWS, and Azure through the on-premise connector to search for a specific file from all backups stored on StorageGRID bucket, AWS S3 and Azure Blob storage and then perform a required restore. It provides users with a quick and focused search for files across all the backups in the StorageGRID bucket and gives options to narrow searches using multiple search criteria. The implementation has been done based on SnapDiff v3 with the full file system representation.
The Indexed catalog or “Search and Restore” On-Premise connector implementation can be done in 3 modes
1) Implementing Indexed catalog on-premise on the snapshots backed up to StorageGRID Bucket using On-prem connector
In this deployment mode, the On-premise connector will be deployed on a VM/Bare-metal within the data center and Cloud Backup will backup the data from the On-Prem ONTAP System to On-Prem StorageGRID. In this mode, the Indexed Cataloging feature utilizes DuckDB for Querying. The cataloging service continually scans backups for incremental changes in working environments that are enabled for indexing. It then updates DuckDB with both the changes and snapshot details. The SnapDiff response from ONTAP is converted to parquet format, it is stored in DuckDB. The cataloging service runs scheduled indexing tasks at regular intervals. Whenever a query is executed, the results are fetched from DuckDB tables and temporarily stored in Level DB.
2) Implementing Indexed catalog on AWS(using Athena and Glue) on the snapshots backed up to AWS S3 Bucket using On-prem connector
In this deployment mode, the On-premise connector will be deployed on a VM/Bare-metal within the data center and Cloud Backup will backup the data from the On-Prem ONTAP System to AWS S3 object store. When Indexed cataloging is enabled on the Cloud Backup UI, appropriate services (Athena and Glue )will be deployed to start the cataloging service for the data stored in AWS S3 Object store
3) Implementing Indexed catalog on Azure(using Synapse) on the snapshots backed up to Azure Blob using On-prem connector
In this deployment mode, the On-premise connector will be deployed on a VM/Bare-metal within the data center and Cloud Backup will backup the data from the On-Prem ONTAP System to the Azure Blob S3 object store. When Indexed cataloging is enabled on the Cloud Backup UI, appropriate services (Synapse )will be deployed to start the cataloging service for the data stored in the Azure Blob S3 object store.
Searches can be done based on the following criteria
To deploy on-premise BlueXP connector, you would need to download the installer from NetApp Support Site and install the Connector software on an existing Linux host in your network or in the cloud.
Please check out the VM host requirements for installing the On-Premise Connector
1) Download the Cloud Manager software from the NetApp Support Site, and then copy it to the Linux host.
2) Assign permissions to run the script.
chmod +x OnCommandCloudManager-V3.9.19.sh
3) Run the installation script.If you have a proxy server, you will need to enter the command parameters as shown below. The installer doesn’t prompt you to provide information about a proxy.
./OnCommandCloudManager-V3.9.19.sh [silent] [proxy=ipaddress] [proxyport=port] [proxyuser=user_name] [proxypwd=password]
4) Open a web browser and enter the following URL:
https://ipaddress
ipaddress can be localhost, a private IP address, or a public IP address, depending on the configuration of the host. For example, if the Connector is in the public cloud without a public IP address, you must enter a private IP address from a host that has a connection to the Connector host.
Sign up at NetApp Cloud Central or log in.
5) Once you log in make sure to select the appropriate NetApp account and give in a BlueXP Connector name.
6) This will redirect and open NetApp BlueXP page.
7) Now go ahead and add/discover your On-prem Cluster on the BlueXP canvas page and start backing up your data using Cloud Backup.
Once your data is backed up to the cloud , you can go ahead and start enabling Indexed catalog or Search and Restore on the Working Environment.
To get started with the Search & Restore feature, make sure that Cloud Backup is enabled.
5. Now you will see all the Working Environments and you can go ahead and enable Indexing on the Working Environment that you would like. Click “Enable Indexing” as shown in the figure.
6. Once the Indexed Catalog is enabled, appropriate services and databases will be created at the backend to start the cataloging features. Enabling Indexed Cataloging will result in the following:
7. Once the services are provisioned, the Indexed Catalog service for the Working Environment will be shown as “Active”.
Now let's go ahead and check out the new UI that has been introduced for the “Search and Restore” feature and understand its functionality.
2. To search for a file or a list of files having a common name, input the name of the file in the search bar and choose from where you would like to restore. You can search from “All Resources”, “Files”, or “Volumes”. To filter further, choose the “Filter by” options to filter by file types, file-creation dates, specific working environments, file sizes, and backup location.
3. Clicking on “View All Backups” will show all the backups both on the Cloud and Primary (local) which contains the particular file. Note that you will only be able to restore the selected file from the cloud backups. The “Restore” button on the Primary (local) backup copies will be grayed out as it's not supported as yet. Click on the "Restore" button on the available cloud backup.
4. Check the restore folder information, select the destination type and Click on Restore to complete the file restore process.
Now that you’ve read about our Cloud Backup: Search and restore feature support for On-Premise connector, we hope it entices you to go ahead try out this new preview feature .
If you would like more information, contact us [Bezalel, Aviram (Aviram.Bezalel@netapp.com), John Andathethu, Jacob (Jacob.A@netapp.com)] and we would be happy to answer your questions.