WHITEPAPER SERIES Shredded Strage in SharePint 2013 What des Shredded Strage mean, hw much des it actually save and hw t take advantage f it in SharePint 2013. What is Shredded Strage? Shredded Strage in SharePint 2013 is a new feature that makes better use f strage and perfrmance f yur SQL server and ultimately SharePint itself. Fr thse that dn t peak behind the SharePint curtains, all elements f SharePint are stred and referenced frm an underlying SQL database. Each site r list yu create becmes a reference stred in SQL and mre imprtantly each piece f cntent that is created r upladed is als stred in SQL as BLOBs. This article is nt intended t get t far int the weeds abut SQL, BLOBs, database tables and the like, I nly mentin these terms because yur Cntent Database and BLOBs are the main cmpnents f the new Shredded Strage feature. Fr a deep dive int these tpics, yu can review Micrsft s TechNet article (http://blgs.technet.cm/b/wbaer/archive/2012/11/12/intrductin-t-shredded-strage-insharepint-2013.aspx ). Prblems with BLOB Strage Befre jumping int SharePint 2013 s Shredded Strage, we need t jurney back t SharePint 2010 and briefly discuss the underlying issues with strage and I/O perfrmance there. In the next sectin I will dive int the actual numbers, but fr nw here is a basic explanatin. In SharePint 2010 when versin histry was enabled, each versin f a dcument created an additinal BLOB in yur cntent database. T put that int perspective, if yu had a 10MB PwerPint file that wuld create a 10MB BLOB. When versin 2 was created, that wuld result in an additinal 10MB BLOB. Yu are prbably thinking that this makes sense. The file was mdified and I may need t pull that particular versin ut f SharePint. Hwever, cnsider that sme mdificatins d nt invlve the actual cntent. Fr example if yu mdified a metadata value, SharePint wuld still add anther 10MB BLOB strage t the database. As a result, that 10MB PwerPint with 10 versins is nw taking up 100MB f space in yur cntent database, even thugh the file itself may never have been altered. Gd thing there are histry limits that can be placed n versins - smething I am sure yu have implemented, right? Due t these inherit inefficiencies with strage, peple lked utside f SQL fr answers. This is where Remte BLOB Strage (RBS) slutins fit in t the picture. These slutins take all that cntent (BLOBs) and externalize them t an alternate lcatin, where strage culd be ptimized and I/O imprved, and left smaller, mre efficient pinters in the database. SharePint can then retrieve the cntent when called upn. Fr SharePint 2010, these slutins became the backbne f many large rganizatins that relied n SharePint t stre massive amunts f cntent and deliver it efficiently t the users.
Back t present day and the release f SharePint 2013, with its Shredded Strage functin. Using the earlier example f the 10MB PwerPint file, what happens nw with SharePint 2013 is that each versin f the file is n lnger stred as 10MB in the cntent database, but rather it is chunked r shredded int smaller pieces, each representing nly the actual mdificatin (metadata r differential if cntent was altered). S the same 10MB file with 10 versins will nt take up 100MB f strage with Shredded Strage, but a significantly smaller ftprint. In an unlikely scenari where all 10 versin are limited t metadata changes, SharePint culd cnsume nly a little mre than the riginal 10MB. Even if the file was mdified, SharePint 2013 intelligently chunks each versin int smaller BLOBs cntaining nly the cntent deltas. This is a great thing. Did I mentin yet that it is enabled by default with n cnfiguratin required! Are yu still with me? Gd. Hw it wrks behind the scenes r hw SharePint recnstructs these BLOBs is a beynd the scpe f this article, s I wuld encurage yu t Bing It fr mre infrmatin if yu are interested. There are many technical articles nline written by experts describing the nks and crannies f Shredded Strage. Fr nw, we care that it is there, it wrks and it is a standard feature f SharePint 2013. Hw Des Shredded Strage Actually Effect Strage? Nw that yu have a basic understanding f what Shredded Strage des and why it was included in SharePint 2013, let s examine sme actual numbers t see where the savings ccur. Fr ur testing, we used the fllwing cnfiguratin: SharePint 2010 with SQL Server 2012, running n Server 2012 (Hyper-V) New Web Applicatin using default ptins 1 Site Cllectin created using the ut f the bx Team Site template Default Shared Dcuments library with Majr versins enabled, a single custm text clumn and a mdified All Dcuments view SharePint 2013 with SQL Server 2012, running n Server 2012 (Hyper-V) New Web Applicatin using default ptins 1 Site Cllectin created using the ut f the bx Team Site template Default Dcuments library with Majr versins enabled, a single custm text clumn and a mdified All Dcuments view PwerPint 2013 file File Size = 10.0MB (10,510,336 bytes)
Figure 1: Windws File Prperties f Test Dcument Measurements were taken using the SQL Server 2012 Management Interface, mre specifically the Object Explrer s Data Space Used value. All numbers are reprted in KB (kilbytes) unless therwise nted. On t the numbers! The first test is purely a baseline t cmpare hw efficiently (r inefficiently) SharePint 2010 and 2013 stred the same PwerPint file with multiple versins. Test 1: Baseline Cmparisn f SharePint 2010 & 2013 Default Strage Within my envirnment, the identical testing prcess fr bth 2010 and 2013 was; 1. Measure the baseline database space usage withut cntent 2. Uplad the PwerPint file and measure the size increase 3. Create a secnd versin by simply mdifying the metadata (the file was nt altered, just the metadata) and measure the increase 4. Repeat the test until 5 versins were created Here are the results fr SharePint 2010:
Actin Database Size (KB) Database Increase with Zer Baseline (KB) Empty Web App 13520 Site Cllectin (/) and Dcument Library created 15136 0 Upladed First Versin using brwser 25624 10488 Created v2 by mdifying metadata nly 36144 21008 Created v3 by mdifying metadata nly 46640 31504 Created v4 by mdifying metadata nly 57136 42000 Created v5 by mdifying metadata nly 67624 52488 Here are the results fr SharePint 2013: Actin Database Size (KB) Database Increase with Zer Baseline (KB) Empty Web App 16992 Site Cllectin (/) and Dcument Library created 19152 0 Upladed First Versin using brwser 29792 10640 Created v2 by mdifying metadata nly 29840 10688 Created v3 by mdifying metadata nly 29880 10728 Created v4 by mdifying metadata nly 29928 10776 Created v5 by mdifying metadata nly 29968 10816 The evidence is quite dramatic. As yu can clearly see in the third clumn Database Increase with Zer Baseline (KB), each metadata mdificatin in 2010 causes an additinal 10+MB f strage space t be utilized. With 2013 that same change nly resulted in a few extra KBs in yur cntent database. Represented as percent changes, yu have the fllwing view: SharePint 2010 SharePint 2013 Ttal Change frm Baseline Incremental Change frm Previus Ttal Change frm Baseline Incremental Change frm Previus 69.29% 69.29% 55.56% 55.56% 138.79% 41.06% 55.81% 0.16% 208.14% 29.04% 56.02% 0.13% 277.48% 22.50% 56.27% 0.16% 346.78% 18.36% 56.47% 0.13% Based n these numbers, the 2013 cntent database shws a 79% decrease in database size cmpared t 2010 which then translates int a 485% increase in database efficiency. Multiply that ut t cver the
GBs r TBs f cntent in yur SharePint farm and yu can quickly realize the benefits f Shredded Strage in SharePint 2013. Befre smene claims that this test is nt indicative f real-life scenaris, I admit that this was a very basic case using a single Open XML Office dcument, but it des shw the strength f the underlying technlgy. I will d a fllw-up pst later cmparing numbers in 2013 with varying types and sizes f cntent. Keep in mind that all cntent is shredded in SharePint 2013, nt just Office files. Can I Take Advantage f Shredded Strage when Upgrading? The numbers abve are very exciting fr new cntent, but yu may be asking yurself what abut all my GBs/TBs f existing SharePint 2010 cntent. Wn t that still take up the same amunt f space? Hw can I get that int 2013 and will it be shredded? In rder t prperly answer these questins, we need t review the supprted methds f getting yur cntent int SharePint 2013. In-place upgrades Database Attach (Backup/Restre) Third party migratin tls In-place upgrades are n lnger supprted in SharePint 2013, s we can crss that ff ur list. The next ptin is Database Attach and this is prbably the mst cmmnly used methd. This prcess requires the cnstructin f a new SharePint farm specifically fr 2013; detaching r backing up yur 2010 databases; mving them t the new farm; and re-attaching r restring them t 2013. An versimplified explanatin, yes, but that is the basic prcess. Fr a mre detailed descriptin, yu can dwnlad Micrsft s SharePint 2013 Upgrade Prcess frm http://www.micrsft.cm/enus/dwnlad/details.aspx?id=30371. Yu might be thinking that earlier I said that Shredded Strage nly wrks with new cntent. Isn t this database backup methd nly restring my existing data? The answer is emphatically, YES. Since SharePint des nt cnsider this cntent new, it is nt prcessed and therefre left un-shredded. Yur 100GBs f cntent in SharePint 2010 will take up the same 100GBs (give r take) in 2013. Yu will nly start t reap the benefits f Shredded Strage when users start adding new cntent OR mdifying sme f that 100GBs upgraded data. New versins and mdificatins f this cntent will be shredded. T illustrate this pint, let s g back t the testing and prceed with: Test 2: Database Backup/Restre First, I need t backup a 2010 Cntent Database and then restre it t 2013.
Figure 2: 2010 DB Restred t SharePint 2013 With that dne, nw I can Munt it t ne f my existing SharePint 2013 Web App (:27858). Figure 3: Database Successfully Munted Figure 4: Cntent Database Attached t Web App
As seen in Central Admin, it is nw available in my SharePint 2013 Web App. The ptin t update the User Experience which was nt perfrmed fr these tests. Figure 5: Accessed in SharePint 2013 With that prcess cmplete, we can examine the baseline size f each database in SQL. First is the SharePint 2010 WSS_Cntent_SP2010 which was used fr back up and then the SharePint 2013 which was restred, munted and accessed: Figure 6: SharePint 2010 Data Space Used (KB) Figure 7: SharePint 2013 Data Space Used (KB) A few extra MBs f space were added during the upgrade prcess, but as clearly illustrated the cntent cntained within this database (10MB PwerPint with 10 versins) was nt shredded. This emphasizes the pint that upgraded cntent databases are nt shredded when munted n a 2013 Web App.
I lgged back int the updated 2013 site and began mdifying my PwerPint file. I prceeded t add 5 additinal versins t the file, each ne cnsisting f a simple change t the metadata. Here is the resulting database after 15 versins f the file were created. Figure 8: Versin Histry Figure 9: Data Space Used after 15 Versins
The first new versin in SharePint 2013 (Versin 11.0 in Figure 8) adds 10MB f space t the database, but subsequent versins cnsume nly abut 50KB each (as shredding is utilized). Overall, an apprximate 11MB increase rather than the 50MB yu wuld have bserved in SharePint 2010 running the same test. This is great news fr thse that plan ahead and allcate enugh space t accmmdate their 2010 databases. Yu might ask --What abut that initial 125MBs, can that als be shredded? The answer is, Yes it can! We will cver that prcess in mre detail a little later. The final ptin t upgrade yur SharePint 2010 t SharePint 2013 is t use a third party migratin tl. I wn t get int the Prs and Cns f using a migratin tl vs. a database restre (that is definitely a tpic fr anther day). What I d want t stress here are the benefits yu can achieve frm the Shredded Strage feature during migratin. Using a migratin tl, SharePint 2013 treats all cntent as new cntent. This means it is immediately shredded reducing the verall database size. Again yu might be saying Really?! Is that true? Yes. If yu dn t want t take my wrd n it (I dn t blame yu), let s nce again g lk t the numbers. Test 3: Migrating frm SharePint 2010 t 2013 Fr this test, I created similar 2010 and 2013 envirnments: A single Web Applicatin using default settings A single rt site cllectin using the Team Site Template A single, empty dcument library with versins enabled, a custm text clumn and a mdified view T establish a base line, I am ging t migrate just the cntent during this test (again my PwerPint file with 5 versins), but I culd have just as easily migrated the entire site cllectin and received similar results. T illustrate the impact f Shredded Strage, I migrated the same cntent int anther SharePint 2010 site cllectin: Actin Database Size (KB) Database Increase with Zer Baseline (KB) Site Cllectin (/) and Dcument Library created 67776 Created identical destinatin Dcument Library 67792 0 Migrated versined PPTX 119760 51968 Figure 10: SharePint 2010 Space Usage after Migratin As expected there was a 50+ MB increase in size. And nw, here is hw the SharePint 2013 Database Space increased during the same test:
Actin Database Size (KB) Database Increase with Zer Baseline (KB) Empty Site Cllectin (/) 60016 Created identical destinatin Dcument Library 60040 0 Migrated versined PPTX 74704 14664 Figure 11: SharePint 2013 Space Usage after Migratin Impressive, right? What used t take ver a 50MB, nw takes as little as 15MB. Once cntent is migrated int SharePint 2013, the cntent is shredded immediately leading t a significant decrease in strage and increase in efficiency frm Day One! That can be a huge savings if yu extraplate that ver yur many GBs/TBs f SharePint cntent. T cnclude this sectin, it is evident that regardless f which methd yu take, Shredded Strage in SharePint 2013 has a psitive impact n yur strage requirements. With the database methd, the impact will be felt gradually as yur SharePint 2013 cntent grws with use, while the migratin path will prvide the same psitive impact immediately. Can I Still Take Advantage f Shredded Strage if I Use a Database Attach Methd? As I hinted earlier, even if yu perfrmed a Database Attach, yu can still take advantage f Shredded Strage. Befre we examine this ptin in mre detail, I want t disclse that I am ging t review a feature that we pineered in MetaVis Migratr; this is nt functinality that is native t SharePint 2013. If yu recall the previus scenari when I reviewed the Database Attach methd t upgrade frm SharePint 2010 t 2013, the riginal 120MB database still cnsumed the identical amunt f space in the 2013 farm. Of curse, any new cntent will be shredded, but the riginal cntent des nt take advantage f this. This is where MetaVis Migratr enters the picture. We engineered the ability t Cmpress Strage fr any cntent that was upgraded. Sunds t gd t be true? Let s lk at the numbers and shw yu what is pssible. Test 4: MetaVis Migratr s Cmpress Strage Feature fr SharePint 2013 Setting the stage fr this test, I attached a 2010 Cntent Database t SharePint 2013 which cntains the nw infamus 10MB PwerPint file with its 10 versins. Figure 12: 2010 DB restred t 2013
I then cnnected MetaVis Migratr t this site and navigated int Shared Dcuments, where the file is lcated. I selected my single PwerPint file and chse the Cmpress Strage ptin. Figure 13: Cmpress Strage in MetaVis Migratr I waited a few secnds and was greeted with a message letting me knw that the prcess was cmplete and the strage was cmpressed successfully. Figure 14: Cmpress Successfully Prcessed Excellent, I nw can g back int SQL Manager and see what actually happened t my database usage.
Befre: After: Nw we re talking! The database space usage was reduced frm a ttal f 125360KB dwn t 37896KB with abslutely n data lse. I still have all 10 versins f my PwerPint file and its metadata. T keep with the same cnsistency as the rest f my tests, I added 5 new versins (metadata changes nly) and measured the database usage alng the way. Here are the raw numbers fr each step f the prcess. SharePint 2013 Actin Database Size (KB) 2010 Database Restred t 2013 SQL 120040 2010 Database Munted t 2013 Web Applicatin 125352 2010 Database Befre MetaVis Cmpress 125360 2010 Database After MetaVis Cmpress 37896 2010 Database after v11 38136 2010 Database after v12 38288 2011 Database after v13 38448 2012 Database after v14 38592 2013 Database after v15 38744 In summary, the Cmpress Strage feature reduced the verall usage size by 87MB r rughly 70%. This is a significant reductin right frm the start. Each subsequent versin was shredded autmatically, but at this pint that was expected. T learn mre abut MetaVis Migratr s Cmpress Strage feature visit ur site (http://www.metavistech.cm ), dwnlad a free trial f Migratr and give it a sht in yur SharePint 2013 envirnment. We re sure yu will be quite pleased with the results.
All Gd Things As I cnclude this lng winded tpic, my intent is nt t imply that this was such a great piece f writing that it is sad t see it cme t an end (well, maybe a little tngue in cheek), but rather a nice phrase t summarize what Shredded Strage actually is, All Gd Things. It is a fantastically useful feature that is bundled int SharePint 2013. It requires n additinal installatin, cnfiguratin r management (I m lking at yu, 2013 Wrkflw Manager!). It des exactly what it claims t d and des it with great ease and efficiency: reduced strage ftprint, increased perfrmance and mre bang frm yur buck. What mre culd yu pssibly ask fr frm SharePint? If yu managed t stay with me and read thrugh this entire pst, thank yu fr yur time. If nt, I can t say that I blame yu, but I hpe that the pieces yu did read were helpful in gaining a better insight int Shredded Strage. I plan t write mre abut Shredded Strage in the future, including actual results with different file types (particularly PDFs), sizes and changes t cntent instead f just metadata. We can then delve int the numbers and see just hw efficient (r nt) SharePint 2013 turn ut t be. Until next time, thanks again fr yur time!