How To Write A Store On A Microsoft Database On A Flash (Orchestra) On A Linux Computer (Ortho) On An Ipro (Orroster) On Your Computer Or Your Computer (For Ahem) On The

Transcription

1 There and Back Again As Quick As a Flash: Tuning Performance in Stefan Winkler Independent Software Developer and IT Consultant Committer on Twitter Note: Extended version of these slides available on the ECE page of this talk! 1

2 in a nutshell 2

3 in a nutshell 3

4 in a nutshell ID attribute reference 3

5 in a nutshell ID ID attribute reference ID 3

6 in a nutshell ID ID attribute reference 3

7 in a nutshell ID ID attribute reference 3

8 in a nutshell ID attribute reference 4

9 in a nutshell ID ID attribute reference 4

10 in a nutshell ID ID attribute reference 4

11 in a nutshell ID attribute reference ID attribute reference 4

12 in a nutshell ID attribute reference ID attribute reference 4

13 Layers App Logic EMF Model IStore Code 5

14 App Logic EMF Model IStore Code 6

15 Choose the Right! time to read 10,000 objects (msec) time to write 10,000 objects (msec) MEM Lissome DB (h2 in-proc) DB (h2 net) DB (MySQL) Hibernate (MySQL) MongoDB (net) DB4O (in-proc)

16 Choose the Right! Store Usage Scenarios Pros Cons MEMStore prototyping, testing supports all features not persistent LissomeStore custom file-based store, optimized for performance performance, full support planned custom file format (no external access) DBStore common/default solution supports all features DB selection and tuning adds to complexity HibernateStore custom DB schema required custom DB schema and mapping only supports subset of features, slower MongoDB Store No SQL DB wanted, fast writing speeds required fast writing speeds slow reading speeds, limited commit size DB4OStore No SQL DB wanted, fast reading speeds required fast reading speeds slower writing speeds dual licence ObjectivityStore OODB with commercial support needed performance? (not measured) commercial licence, performance not measured 8

17 Choose the Right Database! time to read 10,000 objects time to write 10,000 objects (msec) h2 (net) MySQL (net) PostgreSQL (net)

18 Choose the Right Database! time to read 10,000 objects time to write 10,000 objects (msec) h2 (net) Standard Configurations!!! MySQL (net) PostgreSQL (net)

19 Choose the Right Database! time to read 10,000 objects time to write 10,000 objects (msec) h2 (net) and tweak it! Standard Configurations!!! MySQL (net) PostgreSQL (net)

20 App Logic EMF Model IStore Code 11

21 Choose the Right Repository Mode! Audit Mode v1 v2 v3 v4 v5 v6 v7 Time 12

22 Choose the Right Repository Mode! Non-Audit Mode v1 v2 v3 v4 v5 v6 v7 Time 13

23 Choose the Right Repository Mode! Branching Mode 0:v1 v2 v3 v4 v5 v6 1:v1 1:v2 2:v1 2:v2 2:v3 Time 14

24 Choose the Right DBStore Mapping! Audit: plain vs. range-based list mapping Branching: plain vs. range-based list mapping copy-lazy vs. copy-on-write list branching 15

25 Choose the Right DBStore Mapping! non-audit repo mode writes deltas Order date sum no historic ( audit ) views Product Product date Product date sum number sum price Order id version date sum 1 2 3/29/ Product id version number price Order_Product_List id version index value

26 Choose the Right DBStore Mapping! Order audit repo mode classic always writes complete list range-based writes deltas id version date sum 1 2 3/29/ Order_Product_List Product id version number price id version index value

27 Choose the Right DBStore Mapping! Order audit repo mode classic always writes complete list id version date sum 1 2 3/29/ Product id version number price range-based writes deltas Order_Product_List id from to index value 1 1 null null null

28 Choose the Right DBStore Mapping! branching repo mode classic vs. range-based copy-lazy (default) vs. copy-on-branch Order_Product_List id branch from to index value null null null

29 Choose the Right DBStore Mapping! branching repo mode classic vs. range-based copy-lazy (default) vs. copy-on-branch Order_Product_List id branch from to index value null null null 2 4 Order_Product_List id branch from to index value null

30 Choose the Right DBStore Mapping! branching repo mode classic vs. range-based copy-lazy (default) vs. copy-on-branch Order_Product_List id branch from to index value null null null 2 4 Order_Product_List id branch from to index value null

31 Choose the Right DBStore Mapping! branching repo mode classic vs. range-based copy-lazy (default) vs. copy-on-branch Order_Product_List id branch from to index value null null null

32 Choose the Right DBStore Mapping! branching repo mode classic vs. range-based copy-lazy (default) vs. copy-on-branch Order_Product_List id branch from to index value null null null 2 4 Order_Product_List id branch from to index value null null null

33 Choose the Right DBStore Mapping! time to insert 100 objects at beginning time to insert 100 objects in the middle time to insert 100 objects at the end non-audit audit branching branching w/ ranges 19

34 App Logic EMF Model IStore Code 20

35 Choose the Right Transport Layer! Computer 1 Computer 2 TCP (true client/server) TCP (same computer) 21

36 Choose the Right Transport Layer! Computer 1 Computer 2 TCP (true client/server) TCP (same computer) JVM (same JVM, direct calls) 21

37 A Word On Caching Revision Caches in use Soft references Caches grow until memory limit is reached View Cache can be configured to use strong or weak references instead View Cache (Soft Ref) Session Cache (Soft Ref) Repository Cache (Soft Ref) view.options().setcachereferencetype(referencetype.strong); 22

38 App Logic EMF Model IStore Code 23

39 Choose the Right Model Configuration! time to read 100,000 objects time to write 100,000 objects (msec) Native Reflective Legacy

40 App Logic EMF Model IStore Code 25

41 Make Use of Loading Policies! Lazy Collection Loading session.options().setcollectionloadingpolicy( Util.createCollectionLoadingPolicy( /* initial = */ 5, /* onresolve = */ 5)); Explicit Prefetching Object from = getmyobjectfrom(); from.cdoprefetch(/* depth = */ 3); Prefetching by Policy view.options().setrevisionprefetchingpolicy( Util.createRevisionPrefetchingPolicy( /* size = */ 5)); 26

42 Leftovers Things I did not talk about unordered collections store queries (use with caution!) LOBs & streaming 27

43 Summary Choose the Right! Choose the Right Database! Choose the Right Repository Mode! Choose the Right DBStore Mapping! Choose the Right Transport Layer! Choose the Right Model Configuration! Make Use of Loading Policies! More of : Now that I ve got my model - where is my Application? Thu 10:30-11:30 Newsgroup: eclipse.tools.emf 28