1 Twards Supprting the Adptin f Sftware Reference Architectures: An Empirically-Grunded Framewrk Silveri Martínez-Fernández Universitat Plitècnica de Catalunya Jrdi Girna, , Barcelna (Spain) ABSTRACT A Sftware Reference Architecture (SRA) allws rganizatins t reuse architectural knwledge and sftware cmpnents in a systematic way and, therefre, t reduce csts. SRAs mainly appear in rganizatins in which the multiplicity f sftware systems (i.e., sftware systems develped at multiple lcatins, by multiple vendrs and acrss multiple rganizatins) triggers a need fr life-cycle supprt fr all systems. Thus, SRAs are very attractive when rganizatins becme large and distributed in rder t develp new systems r new versins f systems. In return, rganizatins face the need t analyze the return-ninvestment (ROI) in adpting SRAs, and t review these SRAs in rder t ensure their quality and incremental imprvement. The gal f this research is t envisage an empirically-grunded framewrk that supprts rganizatins t decide n the adptin f SRAs and its subsequent design and suitability fr the rganizatin purpses. It helps rganizatins t harvest and arrange relevant evidence frm the wide spectrum f invlved stakehlders and available infrmatin and dcumentatin in SRA prjects. Such a framewrk is being shaped thrugh an actinresearch apprach between ur research grup and everis, an IT cnsulting firm. Categries and Subject Descriptrs D.2.9 [Sftware Engineering]: Management cst estimatin, life cycle, prductivity D.2.11 [Sftware Engineering]: Sftware Architectures dmain-specific architectures Keywrds Sftware reference architecture, empirical sftware engineering 1. INTRODUCTION 1.1 Backgrund Every sftware system has a [cncrete] sftware architecture : the sftware architecture f a system is the set f structures needed t reasn abut the system, which cmprise sftware elements, relatins amng them, and prperties f bth . Nwadays, the size and cmplexity f sftware systems, tgether with critical time-t-market needs, demand new sftware engineering appraches t design sftware architectures . One f these appraches is the use f a Sftware Reference IDESE'13 Baltimre, Maryland USA. Cpyright wned by authr. Architecture (SRA) that allws t systematically reuse knwledge and elements when develping a cncrete sftware architecture  . A SRA becmes, then, the baseline fr many sftware systems, as depicted in Figure 1. The purpse f SRAs is t serve as guidance fr the develpment, standardizatin, and evlutin f diverse sftware systems . This is pssible because SRAs are abstract enugh t allw its usage in differing cntexts . Figure 1. SRAs are the baseline f many sftware systems. 1.2 Mtivatin It has been claimed that SRAs reduce the cmplexity f hardware and sftware architecture by systematically reducing envirnmental diversity [...], enables greatly increased speed and reduce peratinal expenses as well as quality imprvements due t lwered cmplexity, greater investment and greater reuse . Thus, IT rganizatins that lack architecture and cnfiguratin standards [...] have higher csts and less agility that thse with enfrced standards . Accrding t their expected benefits, SRAs have becme widely studied and used by researchers and practitiners. There are many examples f SRAs f different types. On the ne hand, there are SRAs that target a technlgical dmain (als called platfrmriented SRAs ) such as The Open Grup Standard fr SOA Reference Architecture, which is a blueprint that prvides guidelines and ptins fr making architecture, design, and implementatin decisins when adpting a service-riented apprach t infrmatin technlgy . There are als SRAs frm academia t slve well-knwn technlgical prblems (e.g., sftware testing tls ). On the ther hand, there is anther type f SRAs that fcus n a specific business dmain. These SRAs can either target many rganizatins (whse sftware systems share the business dmain), r target a specific single rganizatin (which aims t standardize r facilitate the develpment and maintenance f its wn sftware systems). An example f a SRA that targets many rganizatins is AUTOSAR , which fcus n the autmtive dmain and is being used fr many car manufacturers, and suppliers in rder t standardize the sftware in mdern vehicles.
2 An example f SRAs fr a single rganizatin is the SRA fr NASA s Earth Science Data Systems, which facilitates and hmgenizes the develpment f this type f systems . In this thesis wrk, we pay special attentin t this last type f SRAs, thse that are designed fr and adpted in a single rganizatin. They appear in rganizatins where the multiplicity f sftware systems (i.e., systems develped at multiple lcatins, by multiple vendrs and acrss multiple rganizatins) triggers a need fr lifecycle supprt fr all systems . In this cntext, SRAs are very attractive when rganizatins becme large and distributed  in rder t develp new systems r new versins f systems. 1.3 Prblem Statement Despite SRAs are cnsidered a highly relevant strategic asset by industry and academia, there are still sme gaps that hamper its successful adptin. While cmprehensive methds have been defined fr the analysis f cst and benefits (e.g., CBAM ) and the review (e.g., ATAM ) f (cncrete) sftware architectures, SRAs have received relatively less attentin in literature. The reasn fr this can be prbably traced in an assumptin that these methds fr sftware architectures are directly applicable t SRAs. But in practice, as Angelv et al. pint ut , practitiners face additinal difficulties in wrking with SRAs. It is due t the specific features f SRAs with respect t sftware architectures , such as the need f a high up-frnt investment, their generic nature and high level f abstractin, the wide grup f stakehlders that they invlve, r the risks frm the instantiatin in the rganizatin s prtfli f sftware systems. This situatin triggers specific questins that have nt been addressed yet. Specifically, tw majr issues that need further research have been identified: 1. Lack f supprt fr deciding n the adptin f SRA. There is a shrtage f appraches t precisely evaluate the benefit f architecture prjects  in rder t take infrmed decisins abut adpting a SRA in an rganizatin. Thus, managers and executives lack f supprt t analyze whether it is wrth t invest n the adptin f a SRA in their rganizatin and t calculate its return-n-investment (ROI). 2. Lack f supprt fr SRA design and review. On the ne hand, in spite f research n the elements that shuld cmpse a SRA  and methds abut hw t design them , there is little evidence and supprt abut hw they are actually put frward in practice. On the ther hand, althugh there are several evaluatin methds fr SRA review , they have been hardly applied in the industrial practice. One ptential reasn culd be that the rganizatins find them expensive t apply and hard t be custmized and selected fr their specific needs and practices ; especially because there is n supprt fr identifying the real factrs that might apply in their specific rganizatins (e.g., sme methds suggest t cnsider the imprtant quality attributes fr the rganizatin, but which are such attributes fr the rganizatin?). Therefre, in such situatins, practitiners face difficulties that jepardize the success f the SRA prject. In this cntext, we greatly believe that the availability f evidence abut cst and benefits f real SRA acquisitin prgrams and SRA design and review industrial experiences wuld serve as a basis fr articulating a framewrk t supprt rganizatins and practitiners t face bth prblems. The paper is structured as fllws. Sectin 2 describes previus wrk t cpe with these tw issues. Sectin 3 shws the gal f this thesis. Sectin 4 presents the empirically-grunded framewrk that aims t accmplish such gal. Sectin 5 explains the actin-research initiative in which the framewrk is being shaped and validated, and Sectin 6 the threats t validity. Sectin 7 shws the current status f this research and future wrk. Finally, Sectin 8 expses the pints in which advice wuld be mre valuable. 2. RELATED WORK This sectin presents firstly related wrk fr calculating the ROI f architecture-centric appraches; and secndly related wrk abut reviewing SRAs. 2.1 SRA Ecnmics Althugh there is a lack f research in evaluating the ecnmic viability f SRA adptin, there is a strng base f research in related areas that culd be adapted with this purpse. One the ne hand, given the reusable nature f Sftware Prduct Lines (SPL), the ecnmic mdels that have been prpsed in this area fr identifying its csts and benefits can be adapted t SRA ecnmics. Hwever, ur analysis f these mdels led us t identify that they mainly fall shrt in: Validatin in industry. Very few [ecnmic mdels fr prduct line architecture] actually have been used as a basis fr further develpment r adpted in industry . Thus, there is a clear need fr many mre empirical studies t validate existing mdels . Easy adptin f mdels in industry by identifying realistic metrics t cllect and reprt. It is difficult fr the practitiners t evaluate the usability and usefulness f a prpsed slutin [ecnmic mdels fr prduct line architecture] fr applicatin in industry . N guidelines exist t fully peratinalize the mdels in practice . On the ther hand, there is als related wrk in ther areas. Fr instance, ecnmics-driven sftware architecture analysis methds (e.g., CBAM ). Hwever, existing prpsals d nt specifically aim at making an investment analysis f the adptin f an architecture-centric prgram. SRA adptin is actually a sub area inside their generic decisin-making cntext. Furthermre, ther wrks have addressed aspects as the quantificatin f the benefit frm architecture prjects that imprve quality attributes , and the uncertainties assciated with early lifecycle cst estimatin . At a lwer level, generic sftware metrics like design structure matrix (DSM), culd als be adequate fr calculating the cst and benefit factrs f SRA adptin and make mre cmplete mdels. As a result, the intended thesis uses and tailrs knwledge in these mentined areas fr its wn purpses. 2.2 Review f SRAs The sftware architecture f a sftware system is an early result f the develpment cycle that helps t identify and address imprtant quality aspects, such as system s perfrmance, security, reliability, and maintainability, befre the implementatin f the system is started . Therefre, it is a strategic apprach fr reducing cmplexity, csts and risks. As a result, t ensure architectural quality is an elusive gal. Thus, the practice f evaluating sftware architectures (i.e., the prcess f evaluating whether suitable aspects have been addressed) has matured, with well-
3 knwn methds such as ATAM , which helps stakehlders understand the cnsequences f architectural decisins with respect t the sftware system s quality attribute requirements. Existing methds fr sftware architecture evaluatin have been previusly applied fr SRAs, such as in . Hwever, the sftware engineering cmmunity rarely adpts the methds and techniques available t supprt disciplined architecture review prcesses . Fur pssible reasns fr this are: As Ali Babar et al. pint ut, we think that there remains a need fr systematically accumulating and widely disseminating evidence abut the factrs that may influence the selectin and use f different methds, techniques, and tls fr architecture evaluatin . Evaluatin methds d nt include evidence abut relevant aspects f SRAs, such as which quality attributes are relevant in this type f prjects. Evaluatin teams need t have the visin frm all stakehlders (e.g., prject managers, sftware architects, develpers, etc.). This is nt always supprted by evaluatin methds, leading t prblems while cnducting architectural reviews. Each f these stakehlders has a vested interest in different architectural aspects, which are imprtant t analyze and reasn abut the apprpriateness and the quality f the reference architecture . There is a lack f recent research that prpses means t evaluate reference architectures . These fur issues can be addressed by cllecting evidence abut the factrs that influence the selectin f evaluatin methds, studying the imprtant review criteria and the interest f essential stakehlders. Fr this reasn, mre and mre empirical studies t supprt SRAs thery are starting t be cnducted, as . 2.3 What is it needed? This state-f-the-art drve us t: The frmulatin f an ecnmic mdel fr SRAs built upn: Cst and benefit factrs frm prduct line architecture mdels that are easy-t-apply by the industry. The gal is t prvide guidelines t fully peratinalize the mdel in practice. The gap f SRA ecnmics inside the sftware architecture decisin-making cntext. Generic sftware metrics that can quantify new cst and benefit factrs. The study and cllectin f evidence abut relevant aspects t supprt the design f SRA and the use f architecture evaluatin methds. In , we identified six qualitative relevant aspects (verview, requirements, architectural decisins, business qualities, methdlgy and technlgies) which we take as a primary input fr their further refinement based n the evidence frm rganizatins. 3. RESEARCH OBJECTIVES This sectin expses the gal f this thesis: t supprt rganizatins t decide n the adptin f SRAs and its subsequent design and suitability fr the rganizatin purpses. As stated abve, such a gal will be dealt with by expliting real evidence. It is divided int tw Research Questins (RQ), which respectively deal with the tw prblems stated in Sectin Research Questin 1 The creatin and maintenance f cmplex sftware systems invlves making a series f business-critical architecture design decisins. Imagine that yu are the CIO f an rganizatin with a wide prtfli f sftware systems. Yu have read abut the expected benefits that a SRA may bring t yur rganizatin, e.g., standardizatin f cncrete sftware architecture f systems, greater reuse, shrter time-t-market, reduced csts, reduced risk, supprt fr system develpment at multiple lcatins, by multiple vendrs and acrss multiple rganizatins, and s n. Therefre, yu are cnsidering adpting an existing r new SRA t create and maintain yur rganizatin s sftware systems. Hwever, hw d yu knw if it is wrth fr yur rganizatin t invest n the adptin f a SRA? This questin culd be answered by making a business case with the help f an ecnmic mdel fr SRAs. In the SRA cntext, an ecnmic mdel is needed t help making business cases. An ecnmic mdel shuld take int accunt csts, benefits, risks, and schedule implicatins. An ecnmic mdel t perfrm cst-benefit analysis n the adptin f SRA is a key asset fr ptimizing architectural decisin-making. Reifer defines a business case as the materials prepared fr decisins makers t shw that the idea being cnsidered is a gd ne and that the numbers that surrund it make financial sense . That is, business cases enable t justify investments in technlgy. Spending in the adptin f a SRA withut a previus and trustwrthy analysis seems t be reckless and can lead t a disaster. This triggers the statement f the RQ 1: RQ 1. Hw can rganizatins be supprted t quantitatively analyze the up-frnt investment n the adptin f a SRA? The bjective f the RQ 1 is t prvide guidelines t supprt rganizatins t quantitatively analyze if it is wrth t adpt a SRA. Such an bjective cnsists f cnstructing an ecnmic mdel that supprts the decisin f adpting a SRA. This analysis ptimizes the decisin-making prcess when studying whether t make the strategic mve t SRA in an rganizatin. This research questin is divided int fur sub-research questins. The mtivatin f each sub-research questin is as fllws. First f all it is needed t understand the cntext in which SRAs are adpted (RQ 1.1). Secnd, t identify the data that can be easily cllected in the industry t quantify the csts and benefits f SRAs (RQ 1.2). Third, t define the actual csts and benefits implied by SRA adptin (RQ 1.3). Finally, t make the business case fr the adptin f SRAs (RQ 1.4). RQ 1.1: Hw are SRAs used in practice? RQ 1.2: Which available data d rganizatins have t quantitatively calculate the csts and benefits f adpting a SRA in an rganizatin? RQ 1.3: Which are the cst and benefit factrs f acquiring a SRA in an rganizatin? RQ 1.4: Hw is it pssible t calculate the ROI f the adptin f a SRA in an rganizatin? 3.2 Research Questin 2 Intrducing a SRA int an rganizatin nt nly invlves making a decisin cnsidering the afrementined prductivity issues, but als invlves the analysis f risks, nn-risks, benefits and tradeffs. Whereas prductivity is actually measured in terms f effrt/cst and ecnmic benefits, architectural quality is usually estimated in relatin t eliciting implicit and explicit requirements
4 f the different stakehlders affected by the develpment f the system. Nevertheless, bth views are necessary t achieve a cmprehensive analysis f the system. T help rganizatins t cpe with architectural quality during the design and review prcesses f SRAs, we prpse t accumulate real evidence abut the relevant aspects fr SRAs frm key stakehlders (e.g., which quality attributes they mainly enfrce). This triggers the statement f the RQ 2: RQ 2. Hw can rganizatins be supprted t deal with architectural quality frm its wn industrial evidence? The bjective f the RQ 2 is t supprt practitiners t deal with architectural quality by prviding them a way t gather its wn industrial evidence abut the aspects that matter fr their SRA. Such gathered aspects are aimed t be used t feed any existing architectural evaluatin methd. Specifically, this research questin is divided int five sub-research questins. These RQs address relevant aspects fr the design and review f SRAs: business qualities (RQ 2.1), elements (RQ 2.2), requirements (RQ 2.3), architectural decisins (RQ 2.4), and supprtive technlgies (RQ 2.5) respectively. RQ 2.1: Hw different stakehlders perceive the ptential benefits and drawbacks f SRAs? RQ 2.2: Which are the elements that cmpse a SRA and what is their ptential reuse acrss dmains? RQ 2.3: Which quality attributes des a SRA enfrce? RQ 2.4: Hw are architectural decisins taken and dcumented in SRA prjects? RQ 2.5: Which supprtive technlgies (i.e., methdlgies, tls) are currently being used in SRA prjects? The next sectin presents an empirically-grunded framewrk that aims t prvide means t answer these RQs. 4. RESEARCH APPROACH Empirical research is a way f gaining knwledge by means f direct and indirect bservatin r experience. One f the bjectives f Empirical Sftware Engineering is t gather and utilize evidence t advance sftware engineering methds, prcesses, techniques, and tls. This thesis prpsal fsters the cnductin f empirical studies as a way t incrementally build up SRA thery. The next sectin describes the expected cntributin f this thesis: a Framewrk fr SRA Analysis and Review. 4.1 An Empirically-Grunded Framewrk T accmplish the gal f this research, we plan t devise a framewrk by prviding prcedural guidelines fr setting up and carrying ut empirical studies. The framewrk is cmpsed f an assrtment f empirical studies that wuld help rganizatins t deal with RQ1 and RQ2. Each empirical study fits int ne f the three steps fr empirical research suggested by Whlin et al. : understand, evaluate and imprve. The main idea is that it is in mst cases impssible t start imprving directly and that empirical studies can be cmplementary and supprt each ther (e.g., results frm a preceding study can be used t crrbrate r develp further these results). The framewrk explicitly deals with the understanding and evaluatin steps. The imprving step is achieved by iteratively applying the evaluatin step and cnsidering the lessns learned. The studies shuld be cnducted sequentially. Figure 2 describes the studies that cmpse the framewrk. The rws indicate the step in which the study is being applied whereas the clumns shw the RQ that the study appraches. As shwn by Figure 2, the framewrk is cmpsed f fur studies, tw fr each RQ: RQ 1 is supprted by tw studies: A Survey t check existing value-driven data in rganizatins. It aims t prvide supprt r guidelines t check existing value-driven data in the rganizatin in rder t perfrm a quantitative evaluatin. An Ecnmic mdel t calculate the ROI f adpting a SRA. It aims t prvide an ecnmic mdel t calculate the ROI f adpting a SRA. RQ 2 is supprted by tw studies: A Survey t understand the impact f using a SRA. It aims t prvide supprt r guidelines t understand the impact f using a SRA in the rganizatin in rder t perfrm a qualitative evaluatin. An architectural evaluatin methd specific fr SRA. The abve survey helps t prvide supprt fr the selectin f an architectural evaluatin methd fr SRA and easy its cnductin with infrmatin frm key stakehlders. As explained in Sectin 2.2, currently, there exist evaluatin methds. Therefre, the framewrk just enables practitiners fr the smth applicatin f such existing methds. Figure 2. Empirically-grunded framewrk t supprt rganizatins n SRA adptin, design and review.
5 5. ACTION-RESEARCH PROJECT This research has its rigin in an nging actin-research initiative amng ur research grup and everis, a cnsulting cmpany. Actin research is learning by ding - a grup f peple identify a prblem, d smething t reslve it, see hw successful their effrts were, and if nt satisfied, try again . As a result, the afrementined framewrk is being devised by applying the actin-research cycle in everis: 1) diagnsis f a prblem, 2) examinatin f ptins t slve the prblem, 3) selectin f ptins and executin, 4) analysis f the results, and, 5) repeat fr imprvement. As a cnsulting cmpany, everis ffers slutins fr big businesses (e.g., banks, insurance cmpanies, public administratin, utilities, and industrial rganizatins) that need t manage a wide prtfli f sftware systems that share a specific-dmain. Given the cmplexity f the resulting infrmatin systems, which integrate bespke applicatins with cmmercial packages, these systems need high-quality sftware architecture. This is the service that rganizatins hire t everis. The slutin prvided by everis is based n the adptin f a SRA in the rganizatin, frm which cncrete sftware architectures are derived and used in a wide spectrum f sftware systems. In this cntext, everis cmmissined ur research grup tw main tasks (respectively aligned with ur RQs): t calculate the ROI that rganizatins get after adpting a SRA. and t gather evidence t supprt the design and review f SRAs fr their clients. Precisely, the architecture grup f everis experienced difficulties t cpe with the tw issues that we are cping with in the RQs. As a cnsequence, we are applying the studies envisaged in RQ 1 in rder t calculate the ROI derived frm SRAs that everis created (r plan t create) fr rganizatins. On the ther hand, t supprt them t achieve architectural quality, we plan t cnduct the studies f RQ 2. T d s, it is necessary t cntact SRA s stakehlders . In everis, three essential rles are distinguished: sftware architects that cperatively wrk t figure ut a SRA t accmplish the desired quality attributes and architecturally-significant requirements f the client rganizatin; architecture develpers that are respnsible fr cding, maintaining, integrating, testing and dcumenting the SRA s sftware cmpnents; and applicatin builders that take reusable cmpnents frm the SRA and instantiate them t build cncrete sftware architectures fr sftware systems. 5.1 Framewrk Shaping and Validatin The expected result f this thesis is the empirically-grunded framewrk afrementined (see Figure 2). The framewrk will be incrementally cnstructed based n the actin-research apprach in everis. everis' results will be suitably packaged with the aim f being applied in similar rganizatins. Furthermre, fr its shaping and validatin, the research is divided in tw stages: the frmative and the summative stages. During the frmative stage, we will cnduct empirical studies 1, 2 and 3 (detailed belw) in everis. As their cnductin advances, their feedback will cntribute t incrementally shape and package btained results. The summative stage will take place nce the framewrk has been adequately imprved, shaped and packaged. The primary rle f this stage will be t validate the final versin f the framewrk with practitiners. The next three sub-sectins respectively describe hw the studies f the framewrk have been designed and are being cnducted in the actin-research initiative with everis. 5.2 Empirical Study 1 Objectives f this study. The bjective f this survey is t identify the quantitative infrmatin that can be retrieved frm past prjects in rder t feed the ecnmic mdel (see Empirical Study 2 belw). The main perceived ecnmic benefits n the use f SRAs are the cst avidance in the develpment and maintenance f systems due t the reuse f sftware elements and the adptin f best practices f sftware develpment that increase the prductivity f develpers. The ecnmic mdel needs this data t define and calibrate the parameters t calculate the ROI f adpting an SRA in an rganizatin. Methd. Explratry surveys with persnalized questinnaires applied t relevant stakehlders t find ut the quantitative data that has been cllected in SRA prjects and applicatin prjects. Sampling. A sample f 5 everis SRA prjects and 5 applicatins built upn such SRAs have been selected, and their respective sftware architects and applicatin builders. Apprach fr data cllectin. We use nline questinnaires t ask sftware architects and applicatin builders abut existing infrmatin in past prjects fr calculating cst avidance frm SRA reuse in applicatins. The questinnaire is cmpsed f yes-n questins asking if specific metrics are available fr SRA prjects. Questinnaires enable the additin f cmments and metrics if desired by the interviewees. Data Analysis Methds and Techniques. The data analysis cnsists f cunting in hw many prjects a specific metric is available. Further details f this apprach have been reprted in . 5.3 Empirical Study 2 Objectives f this study. T remain cmpetitive, rganizatins are challenged t make infrmed and feasible value-driven design decisins. Hwever, there is a lack f supprt fr evaluating the ecnmic impact f these decisins with regard t SRAs. This damages the cmmunicatin amng architects and management, which can result in pr decisins. This empirical study analyze whether it is wrth investing in a SRA with the help f an ecnmic mdel. Methd. A case study in which REARM , which is an ecnmic mdel fr SRA adptin, is applied. Sampling. A sample f 2 everis SRA prjects and 2 applicatins built upn such SRAs have been selected. Apprach fr data cllectin. Results frm the Empirical Study 1 revealed that the data available in rder t calculate csts and benefits are effrt and sftware metrics . We cllect these metrics, which are presented in , frm tw types f tls. On the ne hand, a time tracking tl (e.g., JIRA , Redmine ) t cllect the invested effrt frm training, develpment and maintenance activities. Keeping track f activities is cmmn in practice fr prject management and auditing.
6 On the ther hand, tls that calculate sftware metrics t analyze the benefits that can be fund in the surce cde. Fr instance, SnarSurce  ffers tl supprt fr btaining general sftware metrics such as LOC, dependencies between mdules, technical debt, and percentages f tests and rules cmpliance. Data Analysis Methds and Techniques. Fr analyzing the utput f the ecnmic mdel we apply analysis techniques fr business case, such as breakeven analysis, cst-benefit analysis, payback analysis and sensitivity analysis . We have cnducted this empirical study in a public administratin. Results have been published in . 5.4 Empirical Study 3 Objectives f this survey. The purpse f this survey is t understand the impact f using SRAs fr designing the cncrete sftware architecture f the applicatins f a client rganizatin. This is a descriptive survey that measures what ccurred while using SRAs rather than why. With this survey we want t incrementally increase the evidence abut key aspects that really matter t the SRA being analyzed. These key aspects crrespnd t the five sub-rqs f RQ 2. Methd. Descriptive surveys with persnalized questinnaires applied t relevant stakehlders t gather their perceptins and needs. Sampling. The target ppulatins f this survey are SRA prjects and SRA-based applicatins executed by everis. A sample f 9 representative everis SRA prjects in client rganizatins was selected. In these prjects we plan t cntact three essential stakehlders: sftware architects, SRA develpers and applicatin builders. All these prjects were frm Eurpe (seven frm Spain). Apprach fr data cllectin. On the ne hand, semi-structured interviews are used fr sftware architects. The reasn f using interviews is that these rles have higher knwledge than the ther rles abut SRA, s we want t cllect as much infrmatin as pssible frm them. Prir t the interviews, questinnaires are delivered t cllect persnal infrmatin abut the interviewee and t infrm him/her abut the interview. On the ther hand, nline questinnaires are used fr SRA develpers and applicatin builders, since mst f their questins are abut supprtive technlgies and their respnses can be previusly listed, simplifying the data cllectin prcess. The cmplete versin f the prtcl and the questinnaires is available at survey-prtcl.pdf. Data Analysis Methds and Techniques. T perfrm data analysis, we apply qualitative analysis methds . Our research team held several discussin meetings during and after data cllectin. Fr aviding bias, the everis team did nt participated n these meetings (we cntacted them just fr serving as intermediaries fr appraching t the respndents whenever we needed clarificatins). We incrementally prcessed the manual transcriptins f all interviews and autmatically gt the data frm the nline questinnaires. We based ur analysis n grunded thery techniques  such as cnstant cmparisn and crss-case analysis . These techniques are well-fitting in situatins where the researcher des nt want t use pre-cnceived ideas, and instead is driven by the desire t capture all facets f the cllected data and t allw the prpsitins t emerge frm the data. The data analysis cnsisted f tw steps. First, analysis is driven by cding pieces f data as the cnstant cmparisn methd requires . We first read the interview transcripts and the data frm the questinnaires and attach a cding wrd t a prtin f the text a phrase r a paragraph. The cdes are selected t reflect the meaning f the respective prtin f the interview text t a specific RQ. Secnd, we perfrm crss-case analysis  t see the different views frm multiple stakehlders ver the answers with the same cde. We cluster all pieces f text that related t the same cde t analyze it in a cnsistent and systematic way. Finally, t interpret the results, we plan t hld a meeting with the everis team in rder t discuss and imprve ur understanding f the results. By the time f writing this paper, preliminary results abut the RQ 2.1 have been reprted in . 6. THREATS TO VALIDITY This sectin discusses pssible threats t validity f the design f the framewrk in the actin-research prject with everis. It is presented in terms f cnstruct, internal and external validity as well as reliability, as prpsed in . It als emphasizes the mitigatin actins used. 6.1 Cnstruct Validity T strengthen this aspect we have perfrmed a rigrus planning f the study and established a slid prtcl fr data cllectin and data analysis. First f all, t start devising the framewrk, it became necessary t previusly identify relevant aspects t assess SRAs. Hwever, a cmmnly accepted set f criteria t assess SRAs des nt exist . Thus, we identified imprtant aspects t assess SRAs ut f practice and ut f the literature in . The framewrk envisages these aspects as a primary input fr their further refinement during its frmative and summative stages. The clse invlvement f the everis team in the research planning and design is being vital t the suitable cnstructin and develpment f the data cllectin instruments (i.e., the ecnmic mdel s metrics, the interview guides and the questinnaires). In additin, these instruments have been pilted and enhanced t ensure their effectiveness. Given the invlvement f the everis team n the study, we were aware f the imprtance f including specific mitigatin actins fr evaluatin apprehensin by ensuring the aggregated presentatin f the respnses and their cnfidentiality. 6.2 Internal Validity One f the main relevant decisins that directly affected the sampling apprach is that we decided t first chse everis prjects and then participants that cvered the rles we were lking fr. In this way, we ensured that each participant wuld fcus his/her answers n the cntext f the defined prject. This wuld allw a better interpretatin and assessment f cntextual infrmatin. It wuld therwise have been very difficult t interpret certain SRA influential factrs related t the nature f the prjects. We are aware that sme pssible biases may be related t this strategy, fr instance that the everis team chses the mst successful prjects as sampling. T minimize this, we explained them the imprtance f having a representative
7 sampling f the prjects they perfrm in rder t btain reliable data. Regarding the individuals, there is always the pssibility that they frget smething r d nt explicitly state it when they are asked abut it. T reduce this risk: 1) in the case f the interviews, we discussed sme ptential tpics that might be mitted by the respndents, and paid particular attentin t them during the interviews in rder t ask fr clarificatins if necessary; 2) in the case f the nline questinnaires, we designed them in such a way that the respndent culd add additinal cmments and has t answer all the crrespnding questins while he/she culd cmplete the questinnaire at any time, s it gives them the pssibility f cnsulting registries and dcumentatin in case he/she needs t remember smething; 3) in all cases we perfrmed triangulatin by adding questins aimed t cnfirm the crrectness f the answers. We put frward several mitigatin strategies. First, recrding and transcribing all interviews cntributed t a better understanding and assessment f the data gathered. Secnd, t reduce the ptential researcher bias, several meetings were held amng the researchers in rder t discuss the results. Third, althugh the access t the tls, surce cde and dcumentatin is prvided by the everis team, the cllectin f metrics and data frm them is dne by the researchers. 6.3 External Validity As it was mentined in Sectin 1.2, SRAs are widely recgnized in the industry, and ther rganizatins present a very similar cntext t everis. As a cnsequence, we think that it culd be pssible t bserve similar experiences in prjects and cmpanies with similar cntexts. As Seddn et al. suggests: if the frces within an rganizatin that drve bserved behavir are likely t exist in ther rganizatins, it is likely that thse ther rganizatins, t, will exhibit similar behavir . As future wrk, as part f the summative stage, we plan t replicate the framewrk in similar cntexts t everis. We present the results frm the studies with a detailed explanatin f SRA prjects cntext as well as the methds and materials used. This is essential t allw the replicatin f the empirical studies f the framewrk fr ther SRA vendrs and acquisitin cmpanies with similar cntexts in rder t crrbrate the results and being able t generalize the results. Mrever, we acknwledge that several ther factrs may influence SRA prjects and their stakehlder s perceptins (as rganizatinal prcesses and plicies, resurces, cultural issues, etc.). We, therefre plan t design the framewrk in a way that ther factrs can be included at the cnvenience f the rganizatin that is applying it. 6.4 Reliability In rder t strengthen this aspect we addressed the validity f the study. Besides the strategies abve, we maintained a detailed prtcl, cnducted the survey tasks (data cllectin and analysis) by at least tw researchers, spent sufficient time with the study, and gave sufficient cncern t the analysis f all respnses. Such analysis was subsequently discussed with the everis team t imprve the understanding and cntextualizatin f the cnclusins. Finally, in case f the existence f related thery papers r empirical studies, we study hw the results frm ur actin-research apprach supprt r refute previus hypthesis and add new empirical-based prpsitins. 7. CURRENT STATUS AND FUTURE OF THIS RESEARCH S far we have cmpleted the fllwing activities f the frmative stage t shape the framewrk: Preliminary design f the framewrk  based n the state-f-the-art and the state-f-the-practice. Cnductin f the Empirical Study 1 . Cnductin f the Empirical Study 2 . Executin f the Empirical Study 3 and analysis f ne grup f questins (related t benefits and drawbacks ). We plan in the immediate future t carry ut the analysis f the remaining grups f questins frm the Empirical Study 3. Finally, we will package the final research results and validate them in the summative stage f the framewrk. This evaluatin cnsists f the replicatin f the Empirical Studies 1, 2 and 3. Organizatins analyzing whether t make the strategic mve t SRA adptin and rganizatins that face the design and review f SRAs will benefit frm this framewrk. 8. ISSUES OF DISSERTATION There are several issues f which I wuld like t get feedback at the dctral sympsium, althugh any feedback wuld be welcme: Are there any ther relevant aspects t SRAs that culd be addressed in future studies? Are there any mre ways t imprve the data cllectin and analysis? Hw t ensure external validity fr similar cntexts? Hw is it pssible t generalize/package the results withut bias? 9. ACKNOWLEDGMENTS This wrk has been supprted by Cátedra everis and the Spanish prject TIN C REFERENCES  Ali, M.S., Babar, M.A., Schmid, K.: A cmparative survey f ecnmic mdels fr sftware prduct lines. In: Sftware Engineering and Advanced Applicatins, SEAA 09. pp IEEE (2009)  Angelv, S., Grefen, P., Greefhrst, D.: A framewrk fr analysis and design f sftware reference architectures. Infrmatin and Sftware Technlgy 54(4), (2012)  Angelv, S., Trienekens, J.J., Grefen, P.: Twards a methd fr the evaluatin f reference architectures: Experiences frm a case. In: ECSA, pp Springer (2008)  Angelv, S., Trienekens, J.J., Kusters, R.: Sftware reference architectures - explring their usage and design in practice. In: Sftware Architecture. Springer (2013)  AUTOSAR: (2013) [Online; accessed 10-Jul-2013]  Babar, M.A., Bass, L., Grtn, I.: Factrs influencing industrial practices f sftware architecture evaluatin: an empirical investigatin. In: QSA, pp Springer (2007)
8  Babar, M.A., Grtn, I.: Sftware architecture review: The state f practice. Cmputer 42(7), (2009)  Bass, L., Clements, P., Kazman, R.: Sftware architecture in practice, 3 rd Editin. Addisn-Wesley Prfessinal (2012)  Carriere, J., Kazman, R., Ozkaya, I.: A cst-benefit framewrk fr making architectural decisins in a business cntext. In: Sftware Engineering, 2010 ACM/IEEE 32nd Internatinal Cnference n. vl. 2, pp IEEE (2010)  Clements, P., Kazman, R., Klein, M.: Evaluating sftware architectures. Addisn-Wesley Reading (2001)  Clutier, R., Muller, G., Verma, D., Nilchiani, R., Hle, E., Bne, M.: The cncept f reference architectures. Systems Engineering 13(1), (2010)  Crbin, J., & Strauss, A. (Eds.). (2008). Basics f qualitative research: Techniques and prcedures fr develping grunded thery. Sage.  Fergusn, R. et al: "Quantifying uncertainty in early lifecycle cst estimatin (QUELCE)." SEI Technical Reprt, paper 706 (2011).  Gallagher, B.P.: Using the architecture tradeff analysis methd t evaluate a reference architecture: A case study. Tech. rep., DTIC Dcument (2000)  Galster, M., Avgeriu, P.: Empirically-grunded reference architectures: a prpsal. In: Prceedings f QSA. pp ACM (2011)  Galster, M., Avgeriu, P., Tfan, D.: Cnstraints fr the design f variability-intensive service-riented reference architectures an industrial case study. Infrmatin and Sftware Technlgy (2012)  Graaf, B., Van Dijk, H., van Deursen, A.: Evaluating an embedded sftware reference architecture -industrial experience reprt-. In: CSMR Ninth Eurpean Cnference n. pp IEEE (2005)  Glass, R. L. (1994). The sftware-research crisis. Sftware, IEEE, 11(6),  JIRA. (2013) [Online; accessed 14-Jul-2013]  Khurum, M., Grschek, T., Peterssn, K.: Systematic review f slutins prpsed fr prduct line ecnmics. Strategic Decisin Supprt fr Sftware Intensive Prduct Management p. 64 (2008)  Martínez-Fernández, S., Ayala, C., Franch, X., Martins, H.: Benefits and drawbacks f reference architectures. In: 7th Eurpean Cnference n Sftware Architecture (ECSA) (2013)  Martínez-Fernández, S., Ayala, C., Franch, X., Martins, H.: A framewrk fr sftware reference architecture analysis and review. In: 10 th Wrkshp n Experimental Sftware Engineering (ESELAW) (2013)  Martínez-Fernández, S., Ayala, C., Franch, X., Martins, H.: REARM: A reuse-based ecnmic mdel fr sftware reference architectures. In: 13th Internatinal Cnference n Sftware Reuse (ICSR) (2013)  Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded surcebk. Sage.  Muller, G., van de Laar, P.: Researching reference architectures. In: Views n Evlvability f Embedded Systems, pp Springer (2011)  Nakagawa, E.Y., Antnin, P.O., Becker, M.: Reference architecture and prduct line architecture: A subtle but critical difference. In: Sftware Architecture, pp Springer (2011)  Nakagawa, E.Y., Oquend, F., Becker, M.: RAMdel: A reference mdel fr reference architectures. In: WICSA. pp IEEE (2012)  NASA: ESDS Reference Architecture fr the Decadal Survey Era. ESDS%20Reference%20Architecture%20v1.1.pdf (2012). [Online; accessed;10-jul-2013]  Oliveira, L.B.R., Nakagawa, E.Y.: A service-riented reference architecture fr sftware testing tls. In: Sftware Architecture, pp Springer (2011)  Redmine. (2013) [Online; accessed 14-Jul-2013]  Reifer, D.J.: Making the sftware business case: Imprvement by the numbers. Addisn-Wesley Prfessinal (2001)  Rbsn, C. (2002). Real wrld research: A resurce fr scial scientists and practitiner-researchers (Vl. 2). Oxfrd: Blackwell.  Schmid, K.: An initial mdel f prduct line ecnmics. In: Sftware Prduct-Family Engineering, pp Springer (2002)  Sctt, D.: Gartner hype cycle fr real-time infrastructure. (2012), [Online; accessed 1-Mar- 2013]  Seaman, C.B.: Qualitative methds in empirical studies f sftware engineering. IEEE Transactins n Sftware Engineering, vl. 25(4), pp , (1999).  Seddn, P., Scheepers, P.: Twards the imprved treatment f generalizatin f knwledge claims in IS research: drawing general cnclusins frm samples. EJIS, pp. 1-16, (2011).  Snar Surce. (2013) [Online; accessed 14-Jul-2013]  The Open Grup: SOA Reference Architecture. https://www2.pengrup.rg/gsys/jsp/publicatins/public atindetails.jsp?publicatinid=12490 (2013).  Whlin, C., Höst, M., Henningssn, K.: Empirical research methds in sftware engineering. In: Empirical Methds and Studies in Sftware Engineering, pp Springer (2003)  Yin, R. K. (2009). Case study research: Design and methds (Vl. 5). Sage.
ITIL Release Cntrl & Validatin (RCV) Certificatin Prgram - 5 Days Prgram Overview ITIL is a set f best practices guidance that has becme a wrldwide-adpted framewrk fr Infrmatin Technlgy Services Management
Psitin Prfile Psitin Lcatin Reprting t Jb family Band BI/Infrmatin Manager Wellingtn Prfessinal Leaders/Specialists Band I Date February 2013 1. POSITION PURPOSE The purpse f this psitin is t: Lead and
A Walk n the Human Perfrmance Side Part I Perfrmance Architects have a license t snp. We are in the business f supprting ur client rganizatins in their quest fr results that meet r exceed gals. We accmplish
Develping a Cmmunicatin Plan- Aim Aim The aim f a cmmunicatin plan states the verall gal f the cmmunicatin effrt. Determining the Aim Ask yurself r yur team what the verall gal f the cmmunicatin plan is.
POLISH STANDARDS ON HEALTH AND SAFETY AS A TOOL FOR IMPLEMENTING REQUIREMENTS OF THE EUROPEAN DIRECTIVES INTO THE PRACTICE OF ENTERPRISES M. PĘCIŁŁO Central Institute fr Labur Prtectin ul. Czerniakwska
ITIL Service Offerings & Agreement (SOA) Certificatin Prgram - 5 Days Prgram Overview ITIL is a set f best practices guidance that has becme a wrldwide-adpted framewrk fr Infrmatin Technlgy Services Management
ONGOING FEEDBACK AND PERFORMANCE MANAGEMENT A. Principles and Benefits f Onging Feedback While it may seem like an added respnsibility t managers already "full plate," managers that prvide nging feedback
The Imprtance f Market Research 1. What is market research? Successful businesses have extensive knwledge f their custmers and their cmpetitrs. Market research is the prcess f gathering infrmatin which
Change Management Prcess B1.10 Change Management Prcess 1. Intrductin This plicy utlines [Yur Cmpany] s apprach t managing change within the rganisatin. All changes in strategy, activities and prcesses
Systems Lad Testing Appendix 1 Overview As usage f the Blackbard Academic Suite grws and its availability requirements increase, many custmers lk t understand the capability f its infrastructure. As part
Imprtance and Cntributin f Sftware Engineering t the Educatin f Infrmatics Prfessinals Dr. Tick, József Budapest Plytechnic, Hungary, firstname.lastname@example.org Abstract: As a result f the Blgna prcess a new frm f higher
Research Reprt Abstract: The Emerging Intersectin Between Big Data and Security Analytics By Jn Oltsik, Senir Principal Analyst With Jennifer Gahm Nvember 2012 2012 by The Enterprise Strategy Grup, Inc.
Successin Planning & Leadership Develpment: Yur Utility s Bridge t the Future Richard L. Gerstberger, P.E. TAP Resurce Develpment Grup, Inc. 4625 West 32 nd Ave Denver, CO 80212 ABSTRACT A few years ag,
What is Sftware Risk Management? (And why shuld I care?) Peter Kulik, KLCI, Inc. 1 st Editin, Octber 1996 Risks are schedule delays and cst verruns waiting t happen. As industry practices have imprved,
ITIL V3 Planning, Prtectin and Optimizatin (PPO) Certificatin Prgram - 5 Days Prgram Overview The ITIL Intermediate Qualificatin: Planning, Prtectin and Optimizatin (PPO) Certificate is a free-standing
Sftware Quality Assurance Is it the same as Testing? Teji Chpra, Senir Test Cnsultant Planit Sftware Testing Abstract This paper attempts t dispel sme cmmn miscnceptins regarding the rles f Testing and
Dcument Purpse The purpse f this dcument is t prvide guidance n the practice f Business Case and t describe the practice verview, requirements, best practices, activities, and key terms related t these
Kaiser Permanente Gal Setting Overview and Template Gal Setting Overview Each and every emplyee, manager, physician, and leader plays a crucial rle in helping Kaiser Permanente t achieve its gals. This
Sample Rle Descriptin Immunizatin Infrmatin System (IIS) Testing Analyst Nte: This rle descriptin is meant t ffer sample language and a cmprehensive list f ptential desired respnsibilities with crrespnding
NC3A SOA Techwatch Day Call fr Presentatins 1 February 2012 Hsted at NATO C3 Agency, The Hague, The Netherlands By NC3A Chief Technlgy Office (CTO) David Burtn Chief Technlgy fficer Versin 1, 1 December
95-730 E-business Technlgy and Management Curse Descriptin The Internet, and assciated technlgies, are nw an established element f the IT prtfli f rganizatins in bth the public and private sectrs. Experiments
The University f Kent Crprate Standards fr data quality and the cllatin f data fr external presentatin This paper intrduces a set f standards with the aim f safeguarding the University s psitin in published
CCHIIM ICD-10 Cntinuing Educatin Requirements fr AHIMA Certified Prfessinals (& Frequently Asked Questins fr Recertificatin) The transitin t ICD-10-CM and ICD-10-PCS is anticipated t imprve the capture
Key Steps fr Organizatins in Respnding t Privacy Breaches Purpse The purpse f this dcument is t prvide guidance t private sectr rganizatins, bth small and large, when a privacy breach ccurs. Organizatins
ARE YOU INTERESTED IN THE PRIOR LEARNING ASSESSMENT (PLA) PROGRAM? City University f Seattle recgnizes that learning ccurs in many different ways and under varied circumstances. As a result, the University
ISO Management Systems Guidance n understanding the benefits f an ISO Management System Welcme & Intrductins 4031 University Drive, 206, Fairfax, VA 22030 3 Grant Square, 243, Hinsdale, IL 60521 www.radiancmpliance.cm
UNIVERSITY OF LETHBRIDGE Mgt 4390 Z Leading Organizatinal Change Instructr: Rb Gawreluck, PhD. Term: Spring, 2012 Class hurs` 9:00 11:50 Classrm 1301 Office Hurs: By appintment Telephne: 780-907-7294 Email:
SECTION 5: EVALUATION METHODOLOGY The State f Oklahma will cnduct a cmprehensive, fair, and impartial evaluatin f bids received in respnse t this ITB. Technical and Cst Bids will be evaluated and scred
The Imprtance Advanced Data Cllectin System Maintenance Berry Drijsen Glbal Service Business Manager WHITE PAPER knwledge t shape yur future The Imprtance Advanced Data Cllectin System Maintenance Cntents
Audit Manual Sectin J SECTION J QUALITY ASSURANCE AND IMPROVEMENT PROGRAM Ref. Plicy and Practice Requirements IIA Standards and Other references J 1 Plicy: The Head f Internal Audit shall develp and maintain
Recgnitin f Prir Learning (RPL) TAE40110 Certificate IV in Training and Assessment What is RPL? RPL recgnises that yu may already have the skills and knwledge needed t meet natinal cmpetency standards.
CS 360 Sftware Develpment Spring 2008 Tuesdays and Thursdays 3:30 p.m. 4:45 p.m. Instructr: Ingrid Russell Office: Dana 343 email: email@example.com http://uhaweb.hartfrd.edu/irussell Curse Descriptin:
Grant Applicatin Writing Tips and Tricks Grants are prvided by gvernment (lcal, state and natinal), charitable trusts, and by cmmunity rganisatins (eg Ltteries, Rtary, etc). Each grant has a specific purpse,
CCHIIM ICD-10 Cntinuing Educatin Requirements fr AHIMA Certified Prfessinals (& Frequently Asked Questins fr Recertificatin) The transitin t ICD-10-CM and ICD-10-PCS is anticipated t imprve the capture
Oakland Unified Schl District Impact Assessment Perfrmance Management in Actin The perfrmance management system that has been built in this district prvides the systems that supprt ur cmmitment t scial
TOWARDS OF AN INFORMATION SERVICE TO EDUCATIONAL LEADERSHIPS: BUSINESS INTELLIGENCE AS ANALYTICAL ENGINE OF SERVICE A N D R E I A F E R R E I R A, A N T Ó N I O C A S T R O, D E L F I N A S Á S O A R E
Hw t Address Key Selectin Criteria Yu've seen an jb pprtunity that yu're interested in, n a jbs bard r in the press and want t apply, but where d yu start? A key requirement fr jbs in Gvernment is t respnd
Prcess Imprvement Center f Excellence Service Prpsal Recmmendatin Operatinal Oversight Cmmittee Reprt Submissin INTRODUCTION This Prpsal prvides initial infrmatin regarding a pssible additin t a service.
Internal Audit Charter and perating standards 2 1 verview This dcument sets ut the basis fr internal audit: (i) the Internal Audit charter, which establishes the framewrk fr Internal Audit; and (ii) hw
Service - Framewrk 2013 Getting Started Right with Service System Netwrk Firewall Sftware Service App With the right framewrk, enterprises f almst any size small t large can implement effective functinal
Research Reprt Abstract: Security Management and Operatins: Changes n the Hrizn By Jn Oltsik, Senir Principal Analyst With Kristine Ka and Jennifer Gahm July 2012 2012, The Enterprise Strategy Grup, Inc.
Research Findings frm the West Virginia Virtual Schl Spanish Prgram Funded by the U.S. Department f Educatin Cnducted by R0cKMAN ETAL San Francisc, CA, Chicag, IL, and Blmingtn, IN Octber 4, 2006 R0cKMAN
INTERNATIONAL LABOUR ORGANISATION ACT/EMP PUBLICATIONS [Tp] HUMAN RESOURCE DEVELOPMENT FOR ADJUSTMENT AT THE ENTERPRISE LEVEL Training Prgramme (Edited by C.S. Venkata Ratnam) [Next] Table f Cntents Intrductin
FEEDBACK FROM THE VICTORIA QUALITY COUNCIL INTERHOSPITAL PATIENT TRANSFER WORKSHOP Results arising frm the survey f Participants at the Victrian Quality Cuncil (VQC) Interhspital Patient Transfer Wrkshp
Army DCIPS Emplyee Self-Reprt f Accmplishments Overview Revised July 2012 Table f Cntents Self-Reprt f Accmplishments Overview... 3 Understanding the Emplyee Self-Reprt f Accmplishments... 3 Thinking Abut
TRAINING REPORT Fr FARM Africa Extensin Staff Data Analysis, Technical Reprt Writing, and Grant Prpsal Writing Curse held at FARM Africa Meru Office frm 5 9th December 2005 By 1. Prf. Jhn Gwland Mwangi,
Sftware and Hardware Change Management Plicy fr CDes Cmputer Labs Overview The cmputer labs in the Cllege f Design are clsely integrated with the academic needs f faculty and students. Cmputer lab resurces
MANITOBA SECURITIES COMMISSION STRATEGIC PLAN 2013-2016 The Manitba Securities Cmmissin (the Cmmissin) is a divisin f the Manitba Financial Services Agency (MFSA). The ther divisin is the Financial Institutins
EEC Single Sign In (SSI) Applicatin The EEC Single Sign In (SSI) Single Sign In (SSI) is the secure, nline applicatin that cntrls access t all f the Department f Early Educatin and Care (EEC) web applicatins.
Research Reprt Abstract: Netwrk Security Trends in the Era f Clud and Mbile Cmputing By Jn Oltsik, Senir Principal Analyst and Bill Lundell, Senir Research Analyst With Jennifer Gahm, Senir Prject Manager
Jb Prfile Data & Reprting Analyst (Grant Fund) Directrate Lcatin Reprts t Hurs Finance Slihull Finance Directr Nminally 37 hurs but peratinally available at all times t meet Cmpany requirements Cntract
Hw t Reduce Prject Lead Times Thrugh Imprved Scheduling PROBABILISTIC SCHEDULING & BUFFER MANAGEMENT Cnventinal Prject Scheduling ften results in plans that cannt be executed and t many surprises. In many
The Champlain Valley APICS Chapter is a premier prfessinal assciatin fr supply chain and peratins management and wrking tgether with the APICS rganizatin the leading prvider f research, educatin and certificatin
CDE Data Gvernance Prgram - CDE-Specific and SLDS (P20+) Prgrams On September 27 th and 28 th, State Supprt Team (SST) Members Crey Chatis and Jeff Sellers visited Clrad t help CDE begin a Data Gvernance
Research Reprt Abstract: Advanced Malware Detectin and Prtectin Trends By Jn Oltsik, Senir Principal Analyst With Jennifer Gahm, Senir Prject Manager September 2013 2013 by The Enterprise Strategy Grup,
Dcument Purpse The purpse f this dcument is t prvide guidance n the practice f Risk Management and t describe the practice verview, requirements, best practices, activities, and key terms related t these
TOOLKIT fr Making Written Material Clear and Effective SECTION 3: Methds fr testing written material with readers PART 6 Hw t cllect and use feedback frm readers Chapter 12 Shuld yu d audi r vide recrding
Accident Investigatin APPLICABLE STANDARD: 1960.29 EMPLOYEES AFFECTED: All emplyees WHAT IS IT? Accident investigatin is the prcess f determining the rt causes f accidents, n-the-jb injuries, prperty damage,
Sectin 1 General Infrmatin RFR Number: (Reference BPO Number) Functinal Area (Enter One Only) F50B3400026 7 Infrmatin System Security Labr Categry A single supprt resurce may be engaged fr a perid nt t
BRISTOL CITY COUNCIL ROLE AND EMPLOYEE PROFILE: Architect (Practitiner Level) Specific Rle Data Architect Grade Directrate Managed by BG13 (TBC) Business Change Senir Infrmatin Systems & Technlgy Architect
Imprved Data Center Pwer Cnsumptin and Streamlining Management in Windws Server 2008 R2 with SP1 Disclaimer The infrmatin cntained in this dcument represents the current view f Micrsft Crpratin n the issues
Plicy n Free and Open-surce Sftware Gvernment Plicy f Iceland Prime Minister s Office December 2007 Intrductin Free and pen-surce sftware is sftware based n a surce cde which the authrs chse t make public
Internet and E-Mail Plicy User s Guide Versin 2.2 supprting partnership in mental health Internet and E-Mail Plicy User s Guide Ver. 2.2-1/5 Intrductin Health and Scial Care requires a great deal f cmmunicatin
The Department f Industrial and Organisatinal Psychlgy ffers prspective master s students tw ptins twards btaining a master s degree in Industrial and Organisatinal Psychlgy, namely, a cursewrk ptin and
Cmmunal Prperty Institutin Capacity Assessment Tl Intrductin t cmmunal prperty institutins Cmmunal prperty institutins (CPIs) Participants in the land refrm prgramme can hld prperty thrugh different frms
1.0 Backgrund and Overview Dcument Management Versining Strategy Versining is an imprtant cmpnent f cntent creatin and management. Versin management is a key cmpnent f enterprise cntent management. The
Organizatin Design Specialist Suthern Africa Regin BACKGROUND One f the key challenges t implementing the new strategic directin and especially the cmmitment t face the custmer (the child and the cmmunity)
Business Plan Overview Organizatin and Cntent Summary A business plan is a descriptin f yur business, including yur prduct yur market, yur peple and yur financing needs. Yu shuld cnsider that a well prepared
1 Title Fstering Cperative Learning, Discussin, and Critical Thinking in Elementary Math (Grades 1-5) Target Audience This curse is intended fr pre-service and in-service grades 1-5 teachers. Curse Descriptin
Cmparisn f Enterprise Architecture Cncepts between the framewrk and Macrscpe Versin 1.0 June 2014 Macrscpe is a registered trademark f Fujitsu Cnsulting (Canada) Inc. Fujitsu and the Fujitsu lg are registered
HIPAA 5010 Implementatin FAQs fr Health Care Prfessinals Updated September 27, 2012 Key Messages In January 2009, the Department f Health and Human Services published the final rule cntaining the requirements
Advising and Mentring IMPROVING ADVISING AND MENTORING OF GRADUATE AND PROFESSIONAL STUDENTS Advising is a key cmpnent in the successful cmpletin f a graduate degree. A gd advising relatinship crrelates
Das Knw-hw. Wrk- and Prcess Organisatin Seminars and Training Prgrams Fr Effective Prductin Management - REFA Basic Training in Wrk Organisatin - REFA Germany s leading assciatin in Wrk Design Industrial
CAPITALISATION PLAN PLAN FOR CAPITALISATION OF PROJECT RESULTS INTRODUCTION Easy Finance Capitalisatin Plan identifies the strategy t ensure explitatin and sustainability f the prject beynd prject s develpment
BSc (Hns) Civil Engineering Curse cntent The BSc (Hns) Civil Engineering curse is designed arund the cre themes f the prfessinal accrediting bdies and the mdules yu study are designed t develp yur general
Chapter 7 Business Cntinuity and Risk Management Sectin 01 Business Cntinuity Management 070101 Initiating the Business Cntinuity Plan (BCP) Purpse: T establish the apprpriate level f business cntinuity
Your consent to our cookies if you continue to use this website.