Introduction
Welcome to the second instalment in our two-part series on the practical application of quotas within Amazon FSx for NetApp ONTAP (FSx for ONTAP) file systems. In the first blog post, we explored the fundamentals of quotas, delved into a relevant case study, and discussed best practices in quota management. If you have not yet read the initial article, I recommend starting there to gain a foundational understanding of quotas before proceeding with this walkthrough. Now, ready to move from theory to practice, I will guide you through a detailed technical walkthrough to enforce and test quotas in qtrees on an FSx for ONTAP file system.
Scenario
As a storage administrator at E Corporation, you are tasked with creating shared directories for the Finance and Marketing teams in the new Singapore office. To align with the company's mandate to cut costs, you will also implement quotas to monitor and control resource usage as shown in Table 1.
|
Business requirements |
Corresponding IT requirements |
Quota Type |
|
Limit each department to 3 GiB of space |
Each qtree has a default quota of 3 GiB in the volume |
Derived default tree quota |
|
Limit each staff member to 100 MiB of space in their own department |
Each user has a default quota of 100 MiB in their qtree |
Derived default user quota |
|
Marketing department has an increased space allocation of 5 GiB |
Marketing qtree has an increased explicit quota of 5 GiB in the volume |
Explicit quota |
|
Monthly reporting of storage consumption by department |
A tracking quota for all groups is set on the volume |
Tracking quota |
Set up the demo environment
Access to an AWS environment is required to follow along with this technical walkthrough. This walkthrough will take an estimated 2 hours to complete.
To get started, deploy this AWS CloudFormation template that launches a new VPC containing a single-AZ FSx ONTAP file system and two EC2 instances serving as a bastion host and a NFS client as shown in Figure 1. This stack may take up to 60 minutes to deploy.
Note: Implementing this solution incurs billing for the associated AWS services. The estimated cost for this 2-hour walkthrough is USD 4. For more information, see the pricing details pages for those services.
Access NetApp ONTAP CLI
The NetApp ONTAP CLI allow a deeper level of disk space management that cannot be performed on the AWS Management Console.
SSH into the EC2 instance named “ONTAP Quotas Env Linux Bastion Host”, and log in to the NetApp ONTAP CLI to administer and manage file system resources throughout this walkthrough.
ssh fsxadmin@management.fs-01234567890abcabc.fsx.us-east-1.aws.com
Password: fsxadmin-password
This is your first recorded login.
FsxId01234567890abcabc::>
Steps to configure quotas
Create a quota policy
Creating a new quota policy is the first step in managing storage consumption effectively. The quota policy serves as a container for all the quota rules that will enforce limits.
FsxId01234567890abcabc::> volume quota policy create -vserver svm_cluster1 -policy-name singaporeQuotaPolicy
Add quota rules
Add the quota rules to the quota policy based on the requirements in Table 1.
IT Requirement 1: Each qtree has a default quota of 3 GiB in the volume.
Quota Type: Derived default tree quota.
Setting a default tree quota of 3GiB will create a derived default tree quota of 3 GiB for every qtree on the volume. A soft quota of 2.5 GiB is also set to alert administrators if a department is approaching their limits. This quota is crucial for preventing any department from exceeding its share of storage resources, which could impact other departments and overall system performance.
FsxId01234567890abcabc::> volume quota policy rule create -vserver svm_cluster1 -policy-name singaporeQuotaPolicy -volume Singapore -type tree -target "" -disk-limit 3GB -soft-disk-limit 2.5GB
IT Requirement 2: Each user has a default quota of 100 MiB in their qtree.
Quota Type: Derived default user quota.
In the previous step, a default tree quota for each department was configured in the volume. To limit individual storage usage, a default user quota of 100 MiB can be set in the volume. With this default user quota in place, each qtree will have a derived default user quota of 100 MiB. A soft quota of 80 MiB is also set to alert administrators if a user is approaching their limit. This approach automates the process, applying the quota to each user without manual intervention, which is particularly useful in a large organization with many users.
FsxId01234567890abcabc::> volume quota policy rule create -vserver svm_cluster1 -policy-name singaporeQuotaPolicy -volume Singapore -type user -target "" -qtree "" -disk-limit 100MB -soft-disk-limit 80MB
IT Requirement 3: Marketing qtree has an increased explicit quota of 5 GiB in the volume.
Quota Type: Explicit quota.
An explicit quota for the Marketing qtree is necessary to accommodate the department's larger data requirements. Setting an explicit tree quota of 5 GiB will override the derived default tree quota of 3 GiB set in the first rule. This quota type allows for flexibility and customization of storage limits to meet specific team needs. Similarly, a higher soft quota of 4.5 GiB is also set to alert administrators if the department is approaching its new limit.
FsxId01234567890abcabc::> volume quota policy rule create -vserver svm_cluster1 -policy-name singaporeQuotaPolicy -volume Singapore -type tree -target Marketing -disk-limit 5GB -soft-disk-limit 4.5GB
IT Requirement 4: A tracking quota for all groups is set on the volume.
Quota Type: Tracking quota.
Implementing a tracking quota on the volume enables the generation of monthly reports to monitor departmental storage usage without enforcing any limits. This data is vital for informed decision-making and helps the company stay on top of storage trends.
FsxId01234567890abcabc::> volume quota policy rule create -vserver svm_cluster1 -policy-name singaporeQuotaPolicy -volume Singapore -type group -target "" -qtree ""
These additional rules are added to ensure the quota tests can run successfully.
FsxId01234567890abcabc::> volume quota policy rule create -vserver svm_cluster1 -policy-name singaporeQuotaPolicy -volume Singapore -type user -target marketingAdmin -qtree "" -disk-limit 10GB
FsxId01234567890abcabc::> volume quota policy rule create -vserver svm_cluster1 -policy-name singaporeQuotaPolicy -volume Singapore -type user -target marketingAdmin -qtree Marketing -disk-limit 10GB
FsxId01234567890abcabc::> volume quota policy rule create -vserver svm_cluster1 -policy-name singaporeQuotaPolicy -volume Singapore -type user -target financeAdmin -qtree "" -disk-limit 10GB
FsxId01234567890abcabc::> volume quota policy rule create -vserver svm_cluster1 -policy-name singaporeQuotaPolicy -volume Singapore -type user -target financeAdmin -qtree Finance -disk-limit 10GB
Verify that all the quota policy rules are attached to the quota policy
After creating the quota policy and rules, it is important to verify that they are correctly attached to ensure that the quotas will be enforced as expected. The next three steps confirms that the policies are ready to be implemented and will function according to the business requirements.
FsxId01234567890abcabc::> volume quota policy rule show -vserver svm_cluster1 -policy-name singaporeQuotaPolicy
Vserver: svm_cluster1 Policy: singaporeQuotaPolicy Volume: Singapore
Soft Soft
User Disk Disk Files Files
Type Target Qtree Mapping Limit Limit Limit Limit Threshold
----- -------- ------- ------- -------- ------- ------ ------- ---------
tree "" "" - 3GB 2.50GB - - -
tree Marketing
"" - 5GB 4.50GB - - -
user "" "" off 100MB 80MB - - -
user financeAdmin
"" off 10GB - - - -
user financeAdmin
Finance
off 10GB - - - -
user marketingAdmin
"" off 10GB - - - -
user marketingAdmin
Marketing
off 10GB - - - -
group "" "" - - - - - -
8 entries were displayed.
Attach the newly created quota policy to the SVM
FsxId01234567890abcabc::> vserver modify -vserver svm_cluster1 -quota-policy singaporeQuotaPolicy
Verify that the new quota policy is attached to the SVM
FsxId01234567890abcabc::> vserver show -fields quota-policy -vserver svm_cluster1
vserver quota-policy
------------ --------------------
svm_cluster1 singaporeQuotaPolicy
Initialize the quota
Activating the quotas on the volume is a critical step because quotas will not take effect until they are initialized. This process is necessary to transition from the policy creation stage to the enforcement stage, where quotas begin to govern storage usage.
FsxId01234567890abcabc::> volume quota on -vserver svm_cluster1 -volume Singapore -foreground
[Job 41] Job is queued: "quota on" performed for quota policy "singaporeQuotaPolicy" on volume "Singapore" in Vserver "svm_cluster1"
[Job 42] Job succeeded: Successful
Check for initialization errors
If the quota state is:
- initializing: the quotas are in the process of being applied.
- on: the quotas are active and enforcing limits.
- If there is an error message, it will be displayed under 'Last Quota Error Message'.
FsxId01234567890abcabc::> volume quota show -vserver svm_cluster1 -volume Singapore
Vserver Name: svm_cluster1
Volume Name: Singapore
Quota State: on
Scan Status: -
Logging Messages: on
Logging Interval: 1h
Sub Quota Status: none
Last Quota Error Message: -
Collection of Quota Errors: -
Display the quota report
Generate the quota report to verify that the enforced quotas match the requirements.
FsxId01234567890abcabc::> volume quota report -vserver svm_cluster1 -volume Singapore
Vserver: svm_cluster1
----Disk---- ----Files----- Quota
Volume Tree Type ID Used Limit Used Limit Specifier
------- -------- ------ ------- ----- ----- ------ ------ ---------
Singapore
user root 0B - 3 -
Singapore
user marketingAdmin
0B 10GB 0 - marketingAdmin
Singapore
user financeAdmin
0B 10GB 0 - financeAdmin
Singapore
Finance user root 0B - 1 -
Singapore
Finance user financeAdmin
0B 10GB 0 - financeAdmin
Singapore
Marketing
user root 0B - 1 -
Singapore
Marketing
user marketingAdmin
0B 10GB 0 - marketingAdmin
Singapore
group root 0B - 1 -
Singapore
group finance 0B - 1 - *
Singapore
group marketing 0B - 1 - *
Singapore
Finance group finance 0B - 1 - *
Singapore
Marketing
group marketing 0B - 1 - *
Singapore
Finance tree 1 0B 3GB 1 - Finance
Singapore
Marketing
tree 2 0B 5GB 1 - Marketing
Singapore
user * 0B 100MB 0 - *
Singapore
Finance user * 0B 100MB 0 - *
Singapore
Marketing
user * 0B 100MB 0 - *
Singapore
group * 0B - 0 - *
Singapore
Finance group * 0B - 0 - *
Singapore
Marketing
group * 0B - 0 - *
Singapore
tree * 0B 3GB 0 - *
21 entries were displayed.
Test the configured quotas
Testing the configured quotas through simulated data writes is an important validation step. It confirms that the quotas are functioning correctly, and that the targets (users, groups, and qtrees) are restricted according to the policies set in place.
Verify if all qtrees are mounted
SSH into the EC2 instance named “ONTAP Quotas Env NFS Client”, and check that there are 3 qtrees mounted on the instance.
[ec2-user@ip-10-192-10-235 home]$ mount | grep qtree
svm-01234567890abcabc.fs-01234567890abcabc.fsx.us-west-2.amazonaws.com:/Singapore/Marketing on /home/marketingAdmin/marketing_qtree type nfs (rw,relatime,vers=3,rsize=65536,wsize=65536,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.192.20.102,mountvers=3,mountport=635,mountproto=udp,local_lock=none,addr=10.192.20.102)
svm-01234567890abcabc.fs-01234567890abcabc.fsx.us-west-2.amazonaws.com:/Singapore/Finance on /home/sphua/finance_qtree type nfs (rw,relatime,vers=3,rsize=65536,wsize=65536,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.192.20.102,mountvers=3,mountport=635,mountproto=udp,local_lock=none,addr=10.192.20.102)
svm-01234567890abcabc.fs-01234567890abcabc.fsx.us-west-2.amazonaws.com:/Singapore/Finance on /home/financeAdmin/finance_qtree type nfs (rw,relatime,vers=3,rsize=65536,wsize=65536,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.192.20.102,mountvers=3,mountport=635,mountproto=udp,local_lock=none,addr=10.192.20.102)
Quota Test 1: Derived default tree quota
Business Requirement 1: Limit each department to 3 GiB of space.
Test goal: Confirm that the total storage used within the Finance department's qtree does not exceed the 3 GiB limit.
Steps overview:
- Log in as financeAdmin, the administrative user for the Finance department.
- Write a 2 GiB file and observe that the system allows the creation of the file without error.
- Attempt to write a second 2 GiB file and expect an error indicating that the quota limit has been reached.
- Run the volume quota report command to generate a report and confirm that the Finance qtree's storage usage did not exceed 3 GiB.
- View the events logs in the EMS and confirm that there is a tree quota exceeded event generated.
[ec2-user@ip-10-192-10-235 ~]$ su - financeAdmin
[financeAdmin@ip-10-192-10-235 ~]$ cd /home/financeAdmin/finance_qtree/
[financeAdmin@ip-10-192-10-235 finance_qtree]$ dd if=/dev/zero of=2GB_file1 bs=1M count=2048
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 15.0804 s, 142 MB/s
[financeAdmin@ip-10-192-10-235 finance_qtree]$ dd if=/dev/zero of=2GB_file2 bs=1M count=2048
dd: error writing '2GB_file2': No space left on device
1128+0 records in
1127+0 records out
1181745152 bytes (1.2 GB, 1.1 GiB) copied, 6.69408 s, 177 MB/s
Expected Outcome:
- The system should allow the creation of the first 2 GiB file indicating that the quota of 3 GiB has not yet been exceeded.
- The second 2 GiB file creation attempt should fail with an error message, signalling that the 3 GiB limit is enforced.
FsxId01234567890abcabc::> volume quota report -vserver svm_cluster1 -volume Singapore
Vserver: svm_cluster1
----Disk---- ----Files----- Quota
Volume Tree Type ID Used Limit Used Limit Specifier
------- -------- ------ ------- ----- ----- ------ ------ ---------
--- TRUNCATED FOR BREVIETY ---
Singapore
Finance tree 1
3GB 3GB 3 - Finance
--- TRUNCATED FOR BREVIETY ---
Expected Report Output:
- The report should display the Finance qtree with exactly 3 GiB used, demonstrating that the derived default tree quota is correctly applied and actively managing the department's storage usage.
FsxId01234567890abcabc::> event log show -severity NOTICE -event *quota*
Time Node Severity Event
------------------- ---------------- ------------- ---------------------------
6/20/2024 07:25:10 FsxId01234567890abcabc-01
NOTICE wafl.quota.qtree.exceeded: tid 1: tree quota exceeded on volume Singapore@vserver:abcd1234-5678-90ab-cd12-34567890abcd. Additional warnings will be suppressed for approximately 60 minutes or until a 'quota resize' is performed.
Expected EMS Output:
- A tree quota exceeded event should be generated in the EMS.
‼️Note: At the time of publication (June 2024, FSx for ONTAP version 9.13.1), quota notifications are currently limited to hard tree quotas events (wafl.quota.qtree.exceeded and wafl.quota.qtree.correct). Notifications for hard user and group quotas, as well as for soft quota events, do not currently generate notifications. Notifications for these quota events will be progressively added in future updates.‼️
Quota Test 2: Derived default user quota
Business Requirement 2: Limit each staff member to 100 MiB of space in their own department.
Test goal: Verify that individual users cannot exceed their allocated 100 MiB within their department's qtree
Steps overview:
- Log in as user sphua in the Finance department.
- Delete the incomplete file “2GB_file2” created in the previous test.
- Write an 80 MiB file and observe that the system allows the creation of the file without error.
- Attempt to write another 80 MiB and expect an error indicating that the quota limit has been reached.
- Use the volume quota report command to validate that the user's storage consumption did not exceed 100 MiB.
[financeAdmin@ip-10-192-10-235 finance_qtree]$ su - sphua
[sphua@ip-10-192-10-235 ~]$ cd /home/sphua/finance_qtree/
[sphua@ip-10-192-10-235 finance_qtree]$ rm -f 2GB_file2
[sphua@ip-10-192-10-235 finance_qtree]$ dd if=/dev/zero of=80MB_file1 bs=1024 count=81920
81920+0 records in
81920+0 records out
83886080 bytes (84 MB, 80 MiB) copied, 0.370685 s, 226 MB/s
[sphua@ip-10-192-10-235 finance_qtree]$ dd if=/dev/zero of=80MB_file2 bs=1024 count=81920
dd: closing output file '80MB_file2': Disk quota exceeded
Expected Outcome:
- The system should allow the creation of the first 80 MiB file, indicating that the quota of 100 MiB has not been exceeded.
- The second 80 MiB file creation should fail, indicating the 100 MiB quota is enforced.
FsxId01234567890abcabc::> volume quota report -vserver svm_cluster1 -volume Singapore
Vserver: svm_cluster1
----Disk---- ----Files----- Quota
Volume Tree Type ID Used Limit Used Limit Specifier
------- -------- ------ ------- ----- ----- ------ ------ ---------
--- TRUNCATED FOR BREVIETY ---
Singapore
Finance user sphua
99.97MB 100MB 2 - *
--- TRUNCATED FOR BREVIETY ---
Expected Report Output:
- The report should reflect the user sphua has reached his hard limit of 100 MiB within the Finance qtree, demonstrating that the derived default user quota is correctly applied and actively managing the users’ storage usage.
Quota Test 3: Explicit quota
Business Requirement 3: Marketing team has an increased space allocation of 5 GiB.
Test goal: Confirm that the Marketing qtree can use up to 5 GiB without affecting other quotas.
Steps overview:
- Log in as the marketingAdmin user.
- Write two files of 2 GiB each and observe that the system permits the creation of the first two files without error.
- Attempt to write a third file of 2 GiB and expect an error indicating that the quota limit has been reached.
- Run the volume quota report command to validate that the Marketing qtree's storage consumption did not exceed 5 GiB.
[sphua@ip-10-192-10-235 finance_qtree]$ su - marketingAdmin
[marketingAdmin@ip-10-192-10-235 ~]$ cd /home/marketingAdmin/marketing_qtree/
[marketingAdmin@ip-10-192-10-235 marketing_qtree]$ dd if=/dev/zero of=2GB_file1 bs=1M count=2048 && dd if=/dev/zero of=2GB_file2 bs=1M count=2048
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 14.7905 s, 145 MB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 16.2606 s, 132 MB/s
[marketingAdmin@ip-10-192-10-235 marketing_qtree]$ dd if=/dev/zero of=2GB_file3 bs=1M count=2048
dd: error writing '2GB_file3': No space left on device
1114+0 records in
1113+0 records out
1167065088 bytes (1.2 GB, 1.1 GiB) copied, 7.23921 s, 161 MB/s
Expected Outcome:
- The system should allow the creation of the first two files (4 GiB in total), indicating that the quota of 5 GiB has not been exceeded.
- The third 2 GiB file creation should fail, indicating the 5 GiB quota is enforced.
FsxId01234567890abcabc::> volume quota report -vserver svm_cluster1 -volume Singapore
Vserver: svm_cluster1
----Disk---- ----Files----- Quota
Volume Tree Type ID Used Limit Used Limit Specifier
------- -------- ------ ------- ----- ----- ------ ------ ---------
--- TRUNCATED FOR BREVIETY ---
Singapore
Marketing
tree 2
5GB 5GB 4 - Marketing
--- TRUNCATED FOR BREVIETY ---
Expected Report Output:
- The volume quota report should list the Marketing qtree with exactly 5 GiB used, indicating that the explicit quota has allowed the Marketing department to write up to 5 GiB of data, overriding the default tree quota of 3 GiB.
Quota Test 4: Tracking quota
Business Requirement 4: Monthly reporting of storage consumption by department.
Test goal: Ensure that the tracking quota accurately reflects the storage consumption of each department.
Steps overview:
- Calculate the total amount of data written in previous tests per department.
- Run the volume quota report command to generate the storage consumption report.
- Confirm that the report matches the expected data usage for each department.
From the previous tests, the data consumption of each department should be:
|
Department |
User |
Storage consumed (GiB) |
|
Finance |
sphua |
0.1 |
|
financeAdmin |
2.0 |
|
|
Total |
2.1 |
|
|
Marketing |
marketingAdmin |
5.0 |
|
Total |
5.0 |
FsxId01234567890abcabc::> volume quota report -vserver svm_cluster1 -volume Singapore
Vserver: svm_cluster1
----Disk---- ----Files----- Quota
Volume Tree Type ID Used Limit Used Limit Specifier
------- -------- ------ ------- ----- ----- ------ ------ ---------
--- TRUNCATED FOR BREVIETY ---
Singapore
group finance
2.11GB - 4 - *
Singapore
group marketing
5GB - 4 - *
--- TRUNCATED FOR BREVIETY ---
Expected Outcome:
The report should accurately display the storage consumption for the Finance and Marketing groups, demonstrating that the tracking quota can accurately reflect the storage consumption of each department.
Clean up
After completing the tests, we will clean up the resources to avoid incurring future unintended charges. Navigate to the AWS CloudFormation console, select the stack you created at the start of this walkthrough, and click 'Delete Stack’.
Conclusion
In this technical walkthrough, we have built upon the foundational knowledge established in the first part of our series to demonstrate the practical steps of creating and enforcing quotas in FSx for ONTAP. This hands-on guide has taken you through the necessary actions to ensure that your data storage is not only efficient but also aligned with your operational and budgetary goals.
As we conclude this two-part series, I hope that the principles and practices that we have explored will be useful for managing your evolving data landscape effectively. Should you need further assistance, do not hesitate to reach out to a NetApp Cloud Specialist or consult the ONTAP quota documentation for additional guidance.
