Introduction to the Microsoft Sync Framework. Michael Clark Development Manager Microsoft

Transcription

1 Introduction to the Michael Clark Developent Manager Microsoft

2 Agenda Why Is Sync both Interesting and Hard Sync Fraework Overview Using the Sync Fraework Future Directions Suary

3 Why Is Sync Iportant Coputing Device Proliferation More Coon Daily Access To Multiple PCs Devices Services Software + Services Iprove User Experience Iprove Network Utilization Better Availability/Offline Usage

4 Why Is Sync Hard Local Change Detection Change Enueration Avoiding Reflecting Changes Conflict Detection/Resolution Efficiently Handling Interruption & Restart Managing Deleted Ites Correct Cleanup of Tobstones Synchronization Loops

5 Agenda Why Is Sync both Interesting and Hard Sync Fraework Overview Using the Sync Fraework Future Directions Suary

6 What Is The Sync Fraework? Metadata Handles Coon Cases Efficiently Correctly Handles Corner Cases Useable in Any Topology Can be Used to Bridge Multiple Solutions Platfor Supports Low Level Use Of the Metadata Provider Model to Abstract Interaction Between Stores Provides Make it Siple Services Factored to Enable Expansion Infrastructure Coon Stores Coon Protocols Server & Services Integration

7 How to Use The Sync Fraework Write Sync Applications to synchronize stores Using other people s or your own providers Write Providers for your stores and apps Using the Fraework s Sync Runtie Choose your balance of perforance vs. coplexity

8 Exaple Synchronization Session Configuration Sync Application Sync Orchestrator changes changes Sync Provider Meta-data Interpretation Tools Provider Services Sync Provider Data Store changes Sync Runtie MD Store changes Data Store

9 Exaple Reote Sync Session Sync Application Sync Orchestrator Sync Provider Provider Proxy Provider Stub Local Store Reote Store

10 Data Model Concepts The fraework does not andate a data odel Just a few concepts that can be apped to ost odels Sync Scope: the set of objects being synchronized across a set of partners Change unit: granularity of change tracking in a store Granularity of change propagation: only changed units need be sent Granularity of conflict detection: independent changes to the sae change unit are a conflict Consistency unit: granularity of consistency All changes within the sae consistency unit are sent together Thus, sync can never be interrupted with part of a consistency unit applied Scope: User Mailbox Consistency: Message Body Flags Header Consistency: Message Consistency: Message Attachents

11 Basic Metadata Concepts Peers ake changes independently Synchronization: aking peers aware of all changes Each change has a globally-unique version Fundaental concept: Knowledge A concise description of the set of changes that a peer is aware of Knowledge is portable knowledge specification can be understood by any peer not pair-specific: not what I have received fro you Main operations on knowledge Test if a given knowledge covers a given change Add one piece of knowledge to another to produce cobined knowledge Each replica aintains its own knowledge User Mailbox Consistency: Message Body version Flags version Header version Consistency: Message Consistency: Message Knowledge Att 1 version Att 2 version

12 Increental Sync using Knowledge 1. RequestChanges A 3. ConveyChanges B 4. Apply Changes 2. Enuerate Changes RequestChanges: supply your knowledge Enuerate Changes. Is y version covered by your knowledge? If not, send. ConveyChanges: send along Version of the change Enuerator s knowledge what the peer aking the change knew when he ade it what the recipient will learn by applying this change Apply Changes: Conflict detection algorith Is your version covered by y knowledge? If not, you have a conflict

13 Basic Metadata Details Version The ID of the replica aking the change + replica-specific nuber Replica IDs are GUIDs Replica-specific nuber is ever-increasing at the replica Clock vector: X4 Y3 Z7 A set of (replica GUID, replica-specific nuber) pairs Seantics: all versions authored by this replica up to this nuber The siplest exaple of knowledge Gets ore coplex as failures, interruptions and such occur But quiesces to the siple for

14 Platfor Sync Orchestration Between Providers Siple Interaction for Applications Ipleentation of Core Metadata Services Knowledge anageent interfaces Learning new things: K new = K old + K learned Version-to-knowledge coparisons: v K Change enueration assistance Conflict detection Tobstone anageent, filtering, fidelity anageent, uch ore Core services are platfor, storage, and data-type-independent Applicable regardless of protocol being used Unanaged ipleentation for device portability Convenient anaged wrappers Make it Siple Services Support for Change Application Metadata Storage

15 Soe Infrastructure We Provide File Sync Provider Useable on FAT as well as NTFS Filesystes Relational Sync Provider Supporting any ADO.Net Enabled Database Feedsync Produce or Consue Feeds in RSS or ATOM SyncToy Useful UI for Configuring Filesync Partnerships Other

16 Agenda Why Is Sync both Interesting and Hard Sync Fraework Overview Using the Sync Fraework Future Directions Suary

17 Application Code Saple Using the built-in file sync provider Public Class MySyncController Public Sub SynchronizeFolders() Di SyncOrchestrator As New SyncOrchestrator Di LocalProvider As New FileSyncProvider(ySourceReplicaId, _ "c:\folder1") Di ReoteProvider As New FileSyncProvider(yDestinationReplicaId, _ "d:\folder2") With SyncOrchestrator.LocalProvider = LocalProvider.ReoteProvider = ReoteProvider.Synchronize() End With End Sub End Class

18 Recall: The Sync Session Sync Application Sync Orchestrator changes changes Sync Provider Meta-data Interpretation Tools Provider Services Sync Provider Data Store changes Sync Runtie MD Store changes Data Store

19 Provider Interactions GetSyncBatchParaeters Orchestrator Metadata Knowledge A15,B37,C8 Versions I: A12 J: A9 L: B25 M: C8 read Provider Code Sync Runtie Data

20 Provider Interactions GetChangeBatch(Kd) Orchestrator Metadata Knowledge A15,B37,C8 Versions I: A12 J: A9 L: B25 M: C8 read Provider Code read inlocaltick(kd)? iscontained in Kd? AddToBatch Sync Runtie ChangeBatch Data

21 Provider Interactions Orchestrator ProcessChangeBatch(changes, Ks) Metadata ApplyChanges Knowledge A15,B37,C8 Versions I: A12 J: A9 L: B25 M: C8 read write read write Provider Code write GetVersion SaveIte SaveKnowledge Sync Runtie Data

22 Metadata store Internal Databsae storage for provider s etadata Knowledge, versions, tobstones, etc Provider Code Sync Runtie Extreely useful for those who can t store etadata in their store E.g. FAT Data On change Metadata Store Makes it easy to write aintaining providers : Whenever you detect a change, tell etadata store It will update the etadata (new version, tobstone) Can happen in notifications, or during sync Change enueration is taken care of You just read the data fro the store Change application is largely taken care of You just write the data to the store and forward the calls

23 Change Enueration Code snippet public override ChangeBatch GetChangeBatch( uint batchsize, SyncKnowledge destinationknowledge, out object changedataretriever) { ChangeBatch batch = _etadata.getchangebatch(batchsize, destinationknowledge); changedataretriever = this; // this is where the transfer // echanis/protocol would go. // For an in eory provider, // this is sufficient return batch; }

24 Change Application Code Snippet public virtual void SaveIteChange(SaveChangeAction saveaction, IteChange change, SaveChangeContext savechangecontext) {... } switch (saveaction) { } case SaveChangeAction.Create: LocalId localid = MyStore.CreateIte(saveChangeContext.ChangeData); i = replicametadata.createitemetadata(change.iteid, localid); i.creationversion = change.creationversion; i.currentversion = change.changeversion; break; case SaveChangeAction.UpdateVersionAndData: i = replicametadata.finditemetadatabyid(change.iteid); i.currentversion = change.changeversion; MyStore.Update(i.LocalId, savechangecontext.changedata); break; case SaveChangeAction.DeleteAndStoreTobstone: i = replicametadata.finditemetadatabyid(change.iteid); i.isdeleted = true; i.currentversion = change.changeversion; MyStore.Delete(i.LocalId); break; replicametadata.saveitemetadata(i);

25 FeedSync Support FeedSync (previously known as SSE) is a set of extensions to RSS and ATOM to Enables bi-directional ulti-aster synchronization Spec publicly available on MSDN Intended to provide interoperability for Web Service synchronization FeedSync etadata is fully copatible with Sync Fraework Sync Fraework includes built-in support for generating and consuing FeedSync feeds Publish and consue feeds by pulling and pushing SSE feeds to their provider Sync Provider SSE Feed Generator Data Store SSE Feed (RSS / Ato)

26 Producing an SSE Feed Code Snippet void PublishAllItes( MySyncProvider provider, FeedIdConverter idconverter, FeedIteConverter iteconverter, Strea feedstrea) { FeedProducer feedproducer = new FeedProducer(provider, idconverter, iteconverter); } feedproducer.producefeed(feedstrea);

27 Agenda Why Is Sync both Interesting and Hard Sync Fraework Overview Using the Sync Fraework Future Directions Suary

28 Siple Ways to Write Providers Build on Existing Make it Siple Services Higher Level Abstraction Model Provider Focuses on Interacting with the Store How to Create Read Update and Delete data How to Perfor Local Change Detection Fast Anchor Based Detection for Stores Supporting One Really Siple Change Detection Based on Enuerating Contents if no Other Method is Available

29 Enhanced Filtering Support for Filtered Replicas This replica only ever syncs and stores a subset of the data More Tools for Rolling Window Scenarios Just Keep the Upcoing 2 weeks of calendar appointents Nearly Always Filtered Syncs Adds even ore flexibility to the concept of synchronization scope

30 Fidelity Sync between Stores with different scheas Endpoint 1 stores 3 eail addresses for a contact and syncs with Endpoint 2 that only stores 1 eail address Endpoint 1 only stores 45 characters for nae field and syncs with Endpoint 2 that only stores 30 Parts of the data ay be transfored as part of the sync operation to be ore useful on the destination Pictures are converted to 640x480 when transferred to a device Video is stored in a low bit-rate codec How can we ake this work within an arbitrary topology Challenge to record the loss of inforation if changes are passed through nodes with lower fidelity

31 Current and Future Offline Support Sync Services for ADO.Net (aka OCS) Take any ADO.Net Enabled Database Offline Work Locally Against Cached Data in SQL CE Sync Data Back to Central Store ADO.Net Data Services (aka Astoria) Extend Support to Stores Utilizing ADO.Net Data Services REST Style Data Access SQL Server Data Services Take Cloud Backed Data Services Offline Others?

32 Agenda Why Is Sync both Interesting and Hard Sync Fraework Overview Using the Sync Fraework Future Directions Suary

33 Layering 33

34 Flexibility: varying entry points I need to cache y (service) data offline Put your cache in SQL, use OCS No, I need to sync particular stores Use Sync Providers for those stores Use Orchestrator to orchestrate But how do I counicate y changes reotely? Use Haronica FeedSync support to generate and consue feeds Alternatively, extend or create your own protocol But there is no provider for this store Write one easily using Metadata Store (on SQL-CE) and Siple Provider odels I need better perforance and integration Using SQL? Use SQL 2008 Change Tracking to ake it siple Use Knowledge Services to store etadata yourself 10/1/2008 3:12 AM Microsoft Confidential 34

35 Conclusion V1 Shipped Sept 2008 Available for Download on MSDN V2 Currently Being Built More Details Will be Coing out in the Next Year Downloads Forus Saple Code

36 Questions? 36