Pittsurgh, PA 15213-3890 Mesuring Systems Interoperility Conferene on the Aquisition of Softwre-Intensive Systems Jnury 28-30, 2003 Mrk Ksuni Softwre Engineering Mesurement nd Anlysis [SEMA] Softwre Engineering Institute mksuni@sei.mu.edu Sponsored y the U.S. Deprtment of Defense Presenttion Outline Introdution to the Prolem SEI s Reserh ols Typil Questions Some Definitions Approhes to Mesuring Interoperility Sorerd Approh s of Informtion Systems Interoperility (LISI) Mngement Performne Mesures Next Steps 2
The Chllenge Interoperility is the numer one prolem in joint fore & omined opertions. It is the CINC s top issue *. The prolem my e getting worse Rel-world opertions, evlutions nd exerises ontinue to highlight joint/omined wrfighting pility shortflls As new olition prtners develop, omplex systems re quired, nd fixes to pst prolems re pplied in stove-piped fshion Joint Vision 2010 nd 2020 ll for inresingly network-entri wrfre, dependent upon fully interoperle systems * As stted y Ms. Roin Quinln, Deputy Diretor, Systems Interoperility, Offie of the Seretry of Defense [Quinln, 2000]. 3 Common Mesurement Questions Are we le to identify the root uses of interoperility prolems? Are new system quisitions eoming more effetive t voiding the sme types of interoperility prolems tht ourred yesterdy? How does one quntify interoperility in n tionle wy? How do we mesure the trdeoffs etween systems interoperility nd other fundmentl ttriutes of C4I systems inluding Seurity Survivility Performne Avilility Flexiility 4
SEI Reserh ols In This Are Understnd the stte of the prtie for mesuring systems interoperility Typil pprohes for ssessing nd mesuring interoperility Innovtions tht re urrently eing explored or piloted for improving the stte of the prtie Identify potentil mesures nd vlidte their usefulness through ollortive field-sed investigtions Shre the reserh results with the ommunity 5 Stte of the Prtie Report Bsed on previously pulished reports Defines interoperility nd surveys the issues involved with hieving interoperility Reviews urrent pprohes to the interoperility prolem Highlights promising new pproh to ssessing nd mesuring interoperility the s of Systems Interoperility (LISI) Model Reviews other potentil mesures for mesuring vrious dimensions of interoperility Reommends n initil set of mesures for improving interoperility White pper is ville t http://www.psms.om/ 6
Interoperility Defined Tehnil interoperility The ility of systems, units or fores to provide servies to nd ept servies from other systems, units, or fores nd to use the servies to enle them to operte effetively together. Opertionl interoperility Joint Pulition 1-02 7 Tehnil Vs. Opertionl Interop User Interfe User Interfe User Opertionl Interoperility [Semnti Understnding] User Tehnil Interoperility [Syntti Understnding] System A System A Interfe System B Interfe System B 8
Interoperility Three s Opertionl Identifies Wrfighter Reltionships nd Informtion Needs Proess nd s of Informtion Exhnge Requirements Bsi Tehnology Supportility New Cpilities Tehnil Presries Stndrds nd Conventions Proessing nd inter -Nodl s of Informtion Exhnge Requirements Systems, Assoitions to Nodes, Ativities, Needs nd Requirements Systems Speifi Cpilities Identified to stisfy informtion -exhnge levels nd other opertionl requirements Reltes Cpilities nd Chrteristis to Opertionl Requirements Tehnil Criteri overning Interoperle Implementtion & Prourement of Seleted System Cpility Adpted from [Chtfield 98] 9 Presenttion Outline Introdution to the Prolem SEI s Reserh ols Typil Questions Some Definitions Approhes to Mesuring Interoperility Sorerd Approh s of Informtion Systems Interoperility (LISI) Mngement Performne Mesures Next Steps 10
Tehnil Interoperility Sorerd Opertionl Tehnil Systems System Compline S 1 S 2 S 3 S 4 S 5 S 6 R Y R Y S n R Adpted from [Committee 99] The entries rte s pss/mrginl/fil (green, yellow, or red) the ompline of systems S 1, S 2 S n with the relevnt stndrds nd guidne. 11 Systems Interoperility Sorerd Opertionl Tehnil Systems S1 S2 S1 S2 S3 S4 S5 S n S3 Y R S4 Y N/A S5 R Y..... Sn Y The entries rte s pss/mrginl/fil (green, yellow, or red) the pirwise interoperility of the systems indited in the row nd olumn hedings. 12
Opertionl Interoperility Sorerd Opertionl Tehnil Systems Mission Slie #3 Mission Slie #2 Mission Slie #1 S11 S1 R S6 S9 S3 Y The dimonds rte s pss/mrginl/fil (green, yellow, or red) the ility of the systems (indited s irles) to provide the required informtion flows (indited y rrows) for prtiulr mission slie. 13 Presenttion Outline Introdution to the Prolem SEI s Reserh ols Typil Questions Some Definitions Approhes to Mesuring Interoperility Sorerd Approh s of Informtion Systems Interoperility (LISI) Mngement Performne Mesures Next Steps 14
s of Informtion Systems Interoperility [LISI] Projet initited y MITRE, The C4ISR Integrtion Tsk Fore, nd the C4ISR Arhiteture Working roup LISI is referene model nd proess for ssessing informtion systems interoperility. It provides disipline for defining, mesuring, ssessing, nd ertifying the degree of interoperility required or hieved etween systems. 0 1 2 3 4 Enterprise Domin Funtionl Conneted Isolted 15 0: Isolted Mnul gtewy 16
1: Funtionl Emil, FM voie, ttil dt links, text files 17 2: Funtionl Annotted imgery, mps w/ overlys 18
3: Domin Common Opertionl Piture Wrfighter #3 Wrfighter #1 Wrfighter #2 Bttle Mnger 19 4: Enterprise Event-triggered glol dtse updte Pentgon Bttle roup Homelnd Seurity Europen NATO HQ 20
LISI Mturity s - Summry 4 3 2 1 0 Enterprise Domin Funtionl Conneted Isolted Cross-domin informtion & dvned ollortion Intertive mnipultion of shred dt & pplitions Shred dt ut seprte pplitions Sophistited ollortion Miniml ommon funtions; seprte dt & pplitions Heterogeneous produt exhnge Bsi ollortion Eletroni onneted; seprte dt & pplitions Homogeneous produt exhnge Non-onneted Homogeneous produt exhnge 21 LISI Cpilities Model Interoperility Attriutes Desription Computing P A I D environment PA ID Enterprise Universl Enterprise level Intertive Multiple Enterprise model 4 dimensionl topologies Domin Integrted 3 Domin level roupwre World wide Domin model network Funtionl Distriuted 2 Progrm level Desktop Lol networks Progrm model utomtion Conneted Peer-to- 1 Lol/site level Stndrd system Simple Lol Peer drivers onnetion Isolted Mnul 0 Aess ontrol N/A Independent Privte 22
The PAID Attriutes P A I Poliies nd proedures tht enle systems to exhnge informtion pilities nd servies Stndrds Mngement Seurity Poliy Opertions The set of pplitions tht enle informtion exhnge, proessing, or mnipultion (sed on user requirements). The infrstruture required to support the systems opertions Communitions nd Networks Hrdwre System Servies Seurity Equipment D The dt nd informtion strutures used to support oth the funtionl pplitions nd system infrstruture 23 LISI Cpilities Model Interoperility Attriutes Desription Computing P A I D environment PA ID Enterprise Universl Enterprise level Intertive Multiple Enterprise model 4 dimensionl topologies Domin Integrted 3 Domin level roupwre World wide Domin model network Funtionl Distriuted 2 Progrm level Desktop Lol networks Progrm model utomtion Conneted Peer-to- 1 Lol/site level Stndrd system Simple Lol Peer drivers onnetion Isolted Mnul 0 Aess ontrol N/A Independent Privte 24
LISI Cpilities Model Interoperility Attriutes (Environment) P A I D Enterprise (Universl) 4 Domin (Integrted) 3 Funtionl (Distriuted) 2 Conneted (Peer-to-Peer) 1 Isolted (Mnul) 0 Multi-Ntionl Enterprises Cross overnment Enterprises Intertive (Cross pplitions) DoD Enter prises Full Ojet Cut nd Pste d d Domin (Servie/Ageny Dotrine, Proedures, Trining) Common Operting Environment (e.g., DII-COE 5 Compline) Progrm (Stndrd Proedures, Trining, et.) Stndrds Complint (e.g., JTA) Seurity Profile Medi Exhnge Proedures Mnul Aess Controls Shred Dt (e.g., Simultion Displys, Diret DB exhnge) roup Collortion Full Text Cut & Pste We Browser Bsi Opertions (Douments, Spredsheets, Pitures, et.) Advned Messging (Messge Prsers, Emil w/ tthments) Bsi Messging (e.g., Simple Text)) Dt File Trnsfer Simple Intertion (e.g., Telemetry, Remote Aess, Voie, Fx) Multi-Dimensionl Topologies 0 No known interoperility N/A WAN LAN NET Two-Wy One-Wy Removle Medi Mnul Re-Entry Cross -Enterprise Models Enterprise Models DBMS Domin Models Progrm Models nd Advned Dt Formts Bsi Dt Formts Medi Formts Privte Dt Exmple Implementtion Options Tle WAN SIPRNET JWICS NIPRNET (Internet) DISN LES VSAT DISN NET Link 16 Link 22 UHF Rdio VHF Nets Ethernet Token Ring Other Nets 25 The LISI Metri Opertionl Metri Type eneri Expeted Speifi Code E S Tehnil Tehnil Systems Opertionl Enterprise Domin Funtionl Conneted Isolted 4 3 2 1 0 Sulevel Vries y levels; -z defined s Systems through z 26
Exmple LISI Profile & Resulting Metri for Single System Tehnil Interoperility Attriutes (Environment) P A I D Enterprise (Universl) Domin (Integrted) 3 Funtionl (Distriuted) 2 Conneted (Peer-to-Peer) 1 Isolted (Mnul) 0 4 Servie-pproved MNS & ORD, WAN ddressing sheme IE 4.0 DII COE Complint. Windows-std file nme extensions MS Offie, Aess CMTK, 5D, MPE er d d On line Doumenttion Windows Interfe Design uide (JTA) ITU-T Re X.509. Mil Std 2045-28500 Seurity L els Login proedures Eudor FTP Cht 2.0 Win32 API.PPS TCP/IP WAN, NFS, SNMP, ISDN rd IPLAN NES NTP.X.500 TBS, LINK 16 & 22 HF Dt Modem, Kermit, STU III, SM Cellulr BS 0 No known interoperility MIDB, SQL NIFT,2 USMTF, x.400,.wks,.xls, DTED, DBDB,.ppt,.do, RPF, CM, JBI, JPE, HTML, VPF MPE 1.2 KS, wmf System generi metri is 2 27 When Compring Two Systems The LISI metri is simply the lesser of the two systems generi levels. This is the expeted LISI metri. Systems System A System B LISI Metri eneri 4 3 Expeted (A&B) E3 28
Two Systems in Opertionl Use This LISI metri tkes into ount the environmentl ftors nd speifi mission requirements. This is the speifi LISI metri. Mission Slie n Mission Slie #3 Mission Slie #2 Mission Slie #1 S11 S1 R S6 S9 S3 Y LISI Metri Et. S3p S4d S2 29 Opertionl Detiled Mesures of Interest - Exmples Connetivity Cpity System Overlod Underutiliztion Underpity Dt lteny Qeff = ( Qmx Qoh) ( t f tp) Where: Q eff = Effetive system pity (dt rte) Q mx = Mximum dt rte Q oh = System overhed dt rte Perentge t f = Time of initil slot durtion trnsmission (unit trnsmission) messges reeived t p = orretly Unit propgtion y shooters time Perentge of onsisteny/disprity of redundnt dt soures Numer of tries needed to estlish onnetions Dely in sending ritil ommnd messges nd time to reeive knowledge messges Informtion interprettion & utiliztion 30
Enterprise Domin Funtionl Conneted Isolted d d Multi -Ntionl Enterprises Cross overnment Enterprises DoD Enterprises Domin (Servie/Ageny Dotrine, Proedures, Trining) Common Operting Environment (e.g., DII -COE 5 Compline) Progrm (Stndrd Proedures, Trining, et.) Stndrds Complint (e.g., JTA) Seurity Profile Medi Exhnge Proedures Mnul Aess Controls Interoperility Attriutes Intertive (Cross pplitions) FullOjet CutndPste Shred Dt (e.g., Simultion Displys, Diret DB exhnge) roup Collortion Full Text Cut & Pste We Browser Bsi Opertions (Douments, Spredsheets, Pitures, et.) Advned Messging (Messge Prsers, Emil w/ tthments) Bsi Messging (e.g., Simple Text)) Dt File Tr nsfer SimpleIntertion (e.g., Telemetry, Remote Aess, Voie,Fx) N/A Multi- Dimensionl Topologies WAN LAN NET Two-Wy One-Wy Cross- Enterprise Models Enterprise Models DBMS Domin Models Progrm Models nd Advned Dt Formts Bsi Dt Formts RemovleMedi Medi Formts Mnul Re-Entry 0 No known interoperility Privte Dt Enterprise (Universl) Domin (Integrted) Funtionl Conneted Isolted Servie-pproved MNS & ORD, WAN ddressing sheme IE 4.0 DII COE Complint. MS Offie, Aess Windows-std file CMTK,5D, nme extensions MPE er d d On line Doumenttion WindowsInterfe Design uide (JTA) ITU-T Re X.509. Mil Std 2045-28500 SeurityL els Login proedures Interoperility Attriutes Eudor FTP Cht 2.0 Win32 API.PPS TCP/IP WAN, NFS, SNMP, ISDN rd IPLAN NES NTP.X.500 TBS, LINK 16 & 22 HF Dt Modem, Kermit, STU III, SM Cellulr BS 0 No known interoperility MIDB, SQL NIFT,2 USMTF, x.400,.wks,.xls, DTED, DBDB,.ppt,.do, RPF, CM, JBI, JPE, HTML, VPF MPE 1.2 KS,wmf Cpilities Model P A I D (Environment) (Integrted) 3 (Distriuted) 2 (Peer-to -Peer) 1 (Mnul) 0 1 DoD uidne Poliy DII COE JTA Et. enerte system profile using LISI questionnire 3 S1 System profile nd Interoperility metri (Environment) P A I D 4 3 2 (Distriuted) 1 (Peer-to-Peer) 0 (Mnul) R S6 S3 S11 LISI overly of opertionl rhiteture Y S9 System metri is 2 2 Other system profiles with orresponding LISI metri S1 S2 LISI Dt Repository S1 S2 S3 S4 S5 Sn S3 Y R S4 Y N/A S5 R Y..... S n Y enerte System interoperility mtrix Tehnology Insertion Forums 31 Presenttion Outline Introdution to the Prolem SEI s Reserh ols Typil Questions Some Definitions Approhes to Mesuring Interoperility Sorerd Approh s of Informtion Systems Interoperility (LISI) Mngement Performne Mesures Next Steps 32
Mesuring Mngement Commitment to Interoperility The Committee to Review DoD C4I Plns nd Progrms found tht: hieving C4I interoperility is more mtter of orgniztionl ommitment nd mngement thn one of tehnology Potentil mngement mesures Numer of systems ertified to e interoperle Time (or personnel required to develop time-phsed fore nd deployment dt Time need to stnd up ttil network for joint tsk fore Numer of individuls trined in the use of speifi C4I systems 33 Next Steps 1 2 3 4 5 Estlish ollortive reltionships with stkeholders who re onduting interoperility ssessments. Provide guidne for mesurement spets of the ssessment proess(es). Pilot the proess using mesures developed in stge 2. Condut lessons lerned to evlute the utility of the mesures tht were piloted. Assess the results of the pilot study, develop reommendtions nd pulish the results for the ommunity. 34
Referenes [C4ISR 98] [Chtfield 98] [Committee 99] [Quinln 00] C4ISR Arhiteture Working roup, s of Informtion Systems Interoperility (LISI). 1998. Aville online t <http://www.3i.osd.mil/org/io/i3/aw_digitl_lirry/>. Chtfield, J., Enyert, C., nd Fiks, W. New Arhiteture Diretions. The Edge Newsletter. Jnury,1998. Aville online t <http://www.mitre.org/pus/edge/jnury_98/fifth.htm>. Committee to Review DoD C4I Plns nd Progrms. Relizing the Potentil of C4I. Ntionl Ademy Press. Wshington, D.C. 1999. Quinln, Roin. Wepon Systems Interoperility: Evolving Cpility to Support the Wrfighter. April, 2000. Aville online t <http://jit.fhu.dis.mil/>. 35