Let's talk Shredded Storage and SnapManager for SharePoint

There are more and more discussions going on internally related to customers deploying Microsoft SharePoint Server 2013 and using shredded storage. The main question that comes up, "Is shredded storage supported in SnapManager for SharePoint (SMSP)?" The simple answer is Yes, there is nothing special to do in SMSP in order to take a backup with shredded storage, it’s the same process, same results, same restore workflow, etc.


A few details about shredded storage, it is on by default and can be set at the WebApp or Site level. Because it is on by default that means any new document uploaded to SharePoint will be shredded, any existing document will not be shredded until it is opened/saved back. Shredded storage gets a bit tricky with SMSP Real-time Storage Manager (RTSM) because RTSM functions off of file size criteria (eg. 1MB) and shredded storage is implemented using the FileWriteChunkSize property (64kb by default). Which means that when RTSM goes to extend content it doesn’t see a 1MB file coming through the SMSP RBS provider but rather a chunk of varying size in the range of 64kb; btw there is no absolute file size shredding logic. In order to get RTSM to work properly the FileWriteChunkSize needs to be set to a larger chunk size than the RTSM criteria. Changing FileWriteChunkSize is not recommended, but will it happen most definitely, will it causes problems possibly, but it will need to be reviewed on a case by case basis. The basic rule is large values improve throughput, small values improve latency.


Shredded storage is beneficial with document versioning, nothing else really as I see. But saving on document versioning means saving on database size which is fairly significant. We all know how Microsoft SharePoint Server 2010 handles storing 10 versions of a 10MB document totaling a 100MB record footprint; with shredded storage those 10 versions only take up incremental changes (aka shreds). What shredded storage doesn’t address is internal database deduplication, for example if you have File A (10MB) and store it in Document Library X and the same copy in Document Library Y there will be 2 distinct versions of that same document stored in the different libraries each with their own version history, etc. I have not verified this yet but this type of situation could benefit from NetApp deduplication but seeing the SQL Server databases really vary in deduplication savings it may or may not be a big net effect.


The main advantage of shredded storage when it comes to document versioning is I/O performance because it handles updates direct to the database vs using the WFE as an intermediary as it worked in SharePoint 2010. Any version of Microsoft Office that uses DAV or FSSHTTP gains benefits with shredded storage and SharePoint 2013. My research shows that Office documents see more bang for the buck shredded storage but all file types are supported.


Now there is a way to define how shredded storage operates using the FileOperationSettings property of the WebApp with the following settings: UseWebSetting (falls back to using the SPWeb.EnableIncrementalFileOperations setting), AlwaysDirectToShredded (default behavior) or NeverDirectToShredded (think old school inline BLOB storage). I have additional testing to do and will continue to update my blog with more details on shredded storage since this is one of the newest internal features to SharePoint operations that no one really sees. I will also publish more guidance on using shredded storage with SnapManager for SharePoint if I discover any particular details that need to be tweaked.