SIF 3: A NEW BEGINNING

The SIF Implementation Specification Defines common data formats and rules of interaction and architecture, and is made up of two parts: SIF Infrastructure Implementation Specification defines the rules of interaction and messages framework, and SIF Data Model Implementation Specification an XML-based data model that models entities in the learning environment as SIF Data Objects to be shared between applications.

Existing SIF 2.x infrastructure Zone Integration Server (ZIS) Network Account MIS Library Automation SIF Agents Applications H.R. & Finance Post 16 SIF Data Objects Data Warehouse VLM Meals

Introduction to SIF 3

Getting started has never been easier... Simple. Secure. Standard. as well as flexible and scalable!

New Terminology SIF 2 SIF 3 Agent Adapter Zone Environment ZIS Middleware New SIF 3 environments Direct Brokered

SIF functionality now fits in with everything else deployed in your IT data centre. Your developers and administrators will appreciate that the SIF 3.0 infrastructure is REST based, and fully conformant with all accepted REST design patterns. A SIF 3 enabled application doesn t have to be in your data centre. It can be located in the Cloud. As a result, your SIF 3.0 solutions can span hybrid data centre / cloud service deployments. Middleware is no longer mandatory in deploying SIF 3.0 solutions. For example, a tablet based client application can use the SIF 3.0 standard to directly access and update the data maintained by your MIS or Data Warehouse, independent of which particular SIF-enabled supplier products you have deployed.

Data Security is baked into the SIF 3.0 standard from the beginning as it was in earlier releases. New security profile extensions standardise how sites currently utilising client certificates, shared secrets, and / or HMAC SHA 256 authentication technologies may leverage them when constructing SIF solutions. Data Privacy policies are enforceable through a combination of Data Profiles and administrative best practices. For example a School or Academy could configure and dedicate a separate External Environment in which all student data was de-identified, and then allow the access of any Local Authority or Academy Trust-level data analytics application to that environment.

SIF remains an established, community built, free and open data blueprint that is supported by hundreds of educational applications regardless of platform or vendor provider. SIF 3.0 is tighter in terms of defining requirements than its predecessors and as a result allows educational institution architects to select best of breed SIF-enabled components, assured in the knowledge that their choices will interoperate with little or no integration effort, and the value of their IT investments will be preserved.

SIF 3 Functionality Completely Independent of Data Model Leverages existing infrastructure technologies- REST Choice of data model SIF Implementation Data Model (UK) 2.0 OR X Data Model OR Y Data Model SIF Implementation Infrastructure Specification 3.0.1

Why is the separation of infrastructure & data model so important? (1) Each locale can publish a release as often as they wish, enabling responsiveness. The SIF Association can release the global infrastructure free from data driven pressures, increasing quality. Network architects now find many of the best practices they expect in the SIF specifications, promoting adoption. Integrators can apply generic tools to each layer, allowing for the better sharing of existing network resources. Network programmers can use common REST tools to develop SIF Solutions, saving time and money.

Why is the separation of infrastructure & data model so important? (2) Everyone implementing a SIF published data model may utilise data schema driven tools, greatly reducing effort. Software architects are freed to design simpler components that naturally fit together, increasing maintainability. Infrastructure products can be more flexible when working with modular data models, creating opportunities for reuse. Everyone following a few conventions is now empowered to use their own data model with the SIF infrastructure, encouraging innovation. Decision makers are free to chose the form of their data payloads including: alternate formats, efficient compression, and even end-to-end encryption, meeting more specialised needs.

The SIF 3 infrastructure defines: a secure REST-based transport protocol for defining both direct and brokered connections between software applications (which can act as REST servers or clients) a set of services to enable interactions between REST servers and clients, including creating environments, other administrative task, synchronous exchanges of data, and asynchronous message queues an abstract, platform-independent definition of the API between one software application and another or a broker.

SIF 3 infrastructure definitions: A SIF Environment is a distributed system that may consist of middleware (a broker) and one or more software applications with a SIF Adapter (a SIF-enabled application) sharing/reporting one or more SIF data objects over a network. A SIF Implementation consists of one or more SIF Environments deployed and configured to meet customer data sharing and reporting needs. The SIF Implementation Specification defines architecture requirements and communication protocols for software components and the interfaces between them; it makes no assumption of specific hardware or software products needed to develop SIF-enabled applications and/or middleware implementations, other than their ability to support technologies leveraged as the foundation for SIF, most prominently XML and HTTP(S) and including JSON and REST.

SIF 3 Environments (1) Direct Connects Consumers to its Services (no middleware) Support multiple Consumers Support mobile Consumers All Direct Consumers run in Brokered Environments

SIF 3 Environments (2) Brokered Enterprise Service Bus + other middleware All Direct Consumers run in Brokered Environments Centralised Security Policy & Workflow Preventative Maintenance & Diagnostics Highly scalable

SIF 3 Transport: REST Goal: HTTPS usage matches REST Design Patterns Service operations correspond to Get/Post/Put/Delete Service Provider is a REST Resource Class is URL, Object is URL/RefId (Students/1234) Responses are immediate Header arguments in: HTTPS Header fields URL Segments URL Matrix and / or Query Parameters as appropriate HTTPS payload is pure data Interactive Paged Query with ETags XML to JSON conversion conventions documented

SIF 3 infrastructure Direct Environment (optional) Consumer Management Information System Adapter Application

SIF 3 Direct Environment Implemented by a Central Data Service Example: MIS, Data Store Connects directly to Consumer Applications (no middleware) No 3 rd Party Services made available to Consumers Supports multiple Consumers Each Consumer is the only one in its Environment (promotes Data Security and Privacy) Consumers can see data change Events generated by others Ideal for supporting mobile device based Consumer Applications Tablet Dashboards, Smart Phone Queries Any Direct Consumer Application can be seamlessly re-deployed in a Brokered Environment

SIF 3 infrastructure (2) Brokered Environment (optional) Consumer Management Information System Message Broker Middleware Adapter Application

SIF 3 - Brokered Environment Contains a Central Message Broker Connects multiple Service Consumers and Providers Enterprise Service Bus + other middleware components Modular (Queues, Connectors, Registries) vs. ZIS Provides a More Mature API including a Superset of Direct Environment All Direct Consumers can also run in Brokered Environments Supports Administrative Interfaces Centralised Security Policy & Workflow Registries & Service Governance Preventative Maintenance & Diagnostics Provides a highly scalable solution

SIF 3 infrastructure (3) Hybrid Environment (Any of 20,000 RESTful Mobile Client Applications) Student Contact System MIS Data Provider Message Broker Middleware Adapter Application Direct Environment Student Portal LMS Data Provider Brokered Environment

SIF 3 Solutions: Summary Direct Environment Single source for all data Brokered Environment Multiple Data Sources Hybrid Environment Connects Consumers to Providers with no middleware - can be multiple connections. Low End: MIS accessed by students via mobile devices High End: Data warehouse seeded by multiple end-user MIS and accessed by data analytic and reporting applications Connects Consumers to multiple Providers leveraging existing IT middleware (ESB) All Direct Consumers can run in brokered environments External applications register to provide their data highly scalable Standardised Consumer interface to MIS, data warehouse Centralised enforcement of site data security and privacy policies Direct School Portal utilises Direct Environment for 10000 students accessing via a mobile device. The mobile device software sees standard RESTful Service to display Identical information whether School Portal is local or in the cloud Brokered School Portal is also Consumer in a Brokered Environment accessing MIS, Assessment and VLE services School MIS Provider is also a Consumer of a Authority/Trust-wide Brokered Environment where it seeds Learner Data into the Data warehouse.

How SIF could work for you?

Incorporating SIF 2.x and SIF 3.x Library Automation MIS Identity Management VLM Assessment Operational Data Store H.R. & Finance Meals

Incorporating SIF 2.x and SIF 3.x SIF 2 (SIF Classic) Library Automation MIS Identity Management VLM Assessment Operational Data Store H.R. & Finance Meals

Incorporating SIF 2.x and SIF 3.x SIF 2 (SIF Classic) Library Automation Student Contact System Identity Management VLM Assessment Meals Operational Data Store H.R. & Finance

Incorporating SIF 2.x and SIF 3.x SIF 2 (SIF Classic) Library Automation SIF 3 (Direct and/or Brokered) Student Contact System Identity Management VLM Assessment Meals Operational Data Store H.R. & Finance

Incorporating SIF 2.x and SIF 3.x SIF 2 (SIF Classic) Library Automation Work-based learning SIF 3 (Direct and/or Brokered) Student Contact System Identity Management VLM Assessment Meals Operational Data Store H.R. & Finance

Choice Choice of data model SIF Implementation Data Model (UK) 2.0 OR X Data Model OR Y Data Model SIF Implementation Infrastructure Specification 3.0.1

Scalability (1) Performance Enhancement Description Multiple Consumer Sessions for same application Multiple FIFO Message Queues for same Consumer Multiple concurrent Consumer connections option to same Queue Get Next and Pop Queue Service Query Request Long Polling Site Administrators can enable the same application instance, using multiple identifier values (ex: Johnson JHS and Miller Elementary) to register itself as multiple completely independent and parallel Consumers. Any Consumer can set up separate Message Queues and assign different Event types to them. This allows the less important Events (ex: Attendance) to sit in Queue B until the messages in Queue A have all been retrieved and processed. A Consumer may use (where the Queue Service supports it) more than one multiple simultaneous concurrent connection by having multiple HTTP Requests open with a single Queue. This replaces the SIF 2.x double handshake between Consumer and Polling Queue with a single handshake, reducing the number of message exchanges necessary to retrieve an incoming Event during periods of high message traffic by a factor of two. This option eliminates the message retrieval latency time previously associated with repeated polling for an arriving message, in periods of light traffic. Reference The Environments Service create Environment section in Infrastructure Services. The Environment Scalability section in Infrastructure Services. The Queue Service section in Infrastructure Services. The Queue Service section in Infrastructure Services. The Queue Service section in Infrastructure Services.

Scalability (2) Performance Enhancement Description Multiple Objects conveyed in single Request and single Event Immediate Service Responses The ability to pack a single Request to or Event with multiple affected objects from a specific Service Provider offers huge performance benefits for several important use cases (ex: a thousand small Attendance Objects created at the end of a Reporting Period result in the generation of 10 multi-object Events rather than 1000 single object Events). In another major change from SIF 2.x, a Consumer may now request that the Service Response to any given Request be returned synchronously (i.e. immediately) on the HTTP Response to its issued HTTP Request, rather than appear asynchronously at some later time, intermixed with arriving Events. Reference The Service Operation section in the Base Architecture. The Basic Infrastructure Framework section in the Base Architecture. This will likely be the common case for most Requests, it conforms to the standard REST design pattern, it maximises the effectiveness of Direct Environments and it is expected to improve overall Request / Response processing throughput significantly.

Scalability (3) Performance Enhancement Description etag An etag is equivalent to a checksum on all the objects of the type being queried which are maintained by the Service Provider, and it is optionally returned in all Query Responses. Reference The Service Operation (Query) section in the Base Architecture. If a valid etag is included in a Query Request, and if there were no object data changes since the etag was created, no objects are returned. This offers an efficient way for Consumer applications executing on mobile devices (which are unlikely to subscribe to Events) to determine whether they already have the latest data on one or more objects without comparing all the elements.

Use Cases & examples from the real-world

Norfolk County Council (SIF 2) Identity Management Capita Open Hive esafety (Google Apps) Schools People Net (HR) Norfolk Cloud Portal Capita Learning Gateway echalk MIS PupilAsset (Assessment) Capita Discover (Assessment) eschools (register) Different Class Classroom Monitor Show My Homework Pearson Fronter Google Classroom

Norfolk County Council (SIF 3) Identity Management Capita Open Hive esafety (Google Apps) Schools People Net (HR) Norfolk Cloud Portal Capita Learning Gateway echalk Government reporting Best of Breed / mobile applications Data Warehouse MIS PupilAsset (Assessment) Capita Discover (Assessment) eschools (register) Different Class Classroom Monitor Show My Homework Pearson Fronter The example shown above is for illustration purposes only to show the flexibility of SIF 3. It is not meant to represent Norfolk County Council s proposed/future plans. Google Classroom

Open Discussion

Next Steps? Now in terms of providing an Adapter which will integrate your application into a SIF Environment (allowing it to seamlessly and securely exchange data with other applications), basically your options, which determine the kind of tasks your consultant will do and how long it will take, are: Do it yourself: SIF 3 is a standard of standards. Another way of looking at this is it has a solid core of web technologies and patterns. So your favourite languages, libraries, and frameworks for your favourite environment should serve you very well. For the application requiring only simple interoperability, this may be the best option. Leverage Open Source: Say you want to kick it up a notch and leverage a lot of SIF 3 s functionality or just build your own software without becoming an expert on SIF. There exist some excellent Open Source tools specifically for SIF 3. For the sophisticated application that needs to be completely under the owner s control, this is a top option. AU examples: https://github.com/nsip/sif3-framework-java / https://github.com/nsip/sif3framework-dotnet Find a Partner: It is fair to say that SIF would not be where it is today without our infrastructure vendors. Beyond providing middleware, they combine database expertise and SIF know-how to quickly produce quality products for others. When quality and timeline are your primary concerns, bringing in the experts is the finest option.

Contact For further information, please contact: Penny Murray, Community Development Manager pmurray@sifassociation.org Or visit the SIF Association website: www.sifassociation.org