An Enterprise Architect s Approach to Assessment Development

Transcription

1 An Enterprise Architect s Approach to Assessment Development How to Architect, Design and Implement an Efficient Assessment-Building Process 2012 Users Conference New Orleans March 20-23

2 Topics 1. TIBCO Who we are. 2. TIBCO s Knowledge Assessment Program 3. What is Architecture? 4. The Value Proposition of architecting a solution 5. How We Applied and Continue to Apply Architecture 6. Discovering & Building Reusable Components within the Questionmark Perception Repository Q&A Appendix & Glossary Slide 2

3 An Introduction to TIBCO Software, Inc. TIBCO Software Inc. (NASDAQ: TIBX) is a provider of infrastructure software for companies to use on-premise or as part of cloud computing environments. Slide 3

4 TIBCO Certification Program Overview TCA TCP TCA TIBCO Certified Architect Attended formal TIBCO training Passed knowledge assessments and certification exams Passed 5-day performance-based test, followed by evaluation by TIBCO Global Architects TCP TIBCO Certified Professional Attended TIBCO training (or has equivalent knowledge & experience) Passed formal certification exam Available for core technologies TTP TTP TIBCO Trained Professional Actually attended TIBCO training Passed on-line Knowledge Assessment Available for most TIBCO courses Slide 4

5 TIBCO s Knowledge Assessment Our program supports a battery of tests for a rapidly changing high-tech environment. The subject matter is exceptionally volatile. A single technical product enhancement can have a significant impact on an existing item bank. The subject domain ranges from general software development concepts through computer science architecture principles. The stakes are not life-threatening but they can be career-busting. Slide 5

6 What is Architecture? It is planning the business process from conception through maintenance using proven Architecture Best Practices and experienced architects. Formal or semi-formal information gathering sessions Applying standards, specifications, and best practices Modeling a good infrastructure architecture (the foundation) Modeling a good business process architecture (the building) Slide 6

7 Why Did We Architect our Knowledge The business drivers: Assessment System? To improve the Knowledge Assessment Development Business Process To improve the information provided to SMEs To document our business processes To avoid creating a spaghetti architecture in the repository This is a technical term to describe systems built without architecture planning. Slide 7

8 The Value Proposition Higher item quality Reduced time-to-market Improved business process Improved information provided to SMEs Enhanced and automated item-level audit trail Slide 8

9 How We Apply Architecture We Use Architecture Best Practices to: 1. Gather Information 2. Establish an Architecture Vision 3. Architect the Infrastructure 4. Architect the Business Processes For more about modeling see: Appendix A: Architects and Architecture Slide 9

10 The Architecture VISION on 15 June 2011 (1/2) Slide 10

11 The Architecture Vision as Implemented (2/2) Enhanced Quality Assurance Automation Slide 11

12 Defining Reusable Components in the Context of the Questionmark Repository (1/3) Definition: Reusable components are preconfigured frequently-used well-defined functional units within a repository. It is important to formalize their development To maximize and ensure an attractive ROI, apply standards, specifications, and conventions Slide 12

13 Defining Reusable Components in the Context of the Questionmark Repository (2/3) Major Characteristics of Reusable Components: 1. Significant ROI 2. Highly stable. 3. Highly reusable 4. A crisp boundary. Slide 13

14 Defining Reusable Components in the Context of the Questionmark Repository (3/3) Will my reusable components be the same as yours? Some, like templates, yes. Others, probably. Some will be unique to the way business is conducted It is the concept of actively discovering candidate components then selectively implementing them as reusable components in a planned manner that is important. Slide 14

15 Types of Reusable Components for the Questionmark Repository Types of reusable components: Complete Assessment templates (common) Participant screen templates (common) Outcomes (sometimes overlooked) Demographics (can be powerful) Dummy items (placeholders) Mini item banks (scored or un-scored items used across multiple assessments) Others Slide 15

16 Discovering Reusable Components in Our Questionmark Repository (1/3) 1. First, it is not simply looking for common pieces of repository stuff and reusing them. 2. Second, it is important to select and implement only those candidate components that will ensure an attractive ROI. 3. Third, think outside the box: Example: During the discovery phase, it became logical that a quality assessment template would be of considerable value. At the time this was unexpected; in hindsight it is now obvious. Slide 16

17 Discovering Reusable Components in Our Questionmark Repository (2/3) Make a list of the big things in your repository that have the most value to the way you do business. Topics Assessments Templates Global Resources Global Content The above statement seems obvious, even trivial. But is it? Slide 17

18 Discovering Reusable Components in Our Questionmark Repository (3/3) Discovering candidate reusable components The most important big things in our repository for the way we at TIBCO do business are: Topics Assessments Templates Global Resources Global Content Topics Assessments Demographics Intro Screens Now we know the most productive places to look for discovering reusable components. Slide 18

19 DEMO: Selecting Reusable Components to Build for the Questionmark Repository 20% of your candidate components will provide 80% of the value. Identify them: 1. Make a list of candidate reusable components 2. Make a list of the most important aspects of reusable components to your business process 3. Use a multiplicative weighted matrix to winnow the list 4. Apply Pareto s Principle to the results Slide 19

20 Building Reusable Components for the Questionmark Repository (1/2) Build components that have the -abilities: flexibility, scalability, reliability, When possible, design components that do not require versioning within the production repository Increases ROI by simplifying maintenance Slide 20

21 Building Reusable Components for the Questionmark Repository (2/2) Questions to ask before building a component Can it be used in more than one context? Will its function be needed over time? Where in the repository should it be stored? The repository structure for custom components should be organized to reflect their relationships and to facilitate the way they will be used. Slide 21

22 Overview of TIBCO s Reusable Assessments Templates Components Intro Screens Quality Assurance Demographics Topics Slide 22

23 DEMO: TIBCO Reusable Components Slide 23

24 DEMO: Auto-Enhancing the QM Printout Slide 24

25 Item-level auditing in the QM Printout (3/3) Demo: Generation of docs Item-level auditing For more information: Appendix C: Enhancing the QM Printout Slide 25

26 Future Enhancements??? Slide 26

27 Questions & Answers Slide 27

28 THANK YOU! Slide 28

29 Appendix A: Architects and Architecture Types of Architecture & Kinds of Architects Architecture Modeling Initial Architecture Models Selecting Components to Implement Slide 29

30 * Types of Architecture and Kinds of Architects Types of Architecture: Business Process Architecture Infrastructure Architecture Kinds of Architects Global Architects Enterprise Architects Project Architects Slide 30

31 * Architecture Modeling Architecture modeling starts with capturing and modeling the essence of a domain, often a business process. The most effective models used in the software industry for capturing the essence of a business process are: Yourdon s context level diagram UML s Collaboration diagrams UML = Unified Modeling Language Slide 31

32 * Initial Architecture Models Slide 32

33 Selecting Components to Implement Slide 33

34 Appendix B: Best Practices Document the business process (a living document) Create a simple workflow vision Study the repository s structure and content to discover candidate reusable components Name and list the candidates then do a weighted analysis Design and build the 20% that yield the most benefit Determine a repository structure that best supports how the components are related and how they will be used Test. Test. Test. Deploy Monitor the usage Slide 34

35 Appendix C: Enhancing the QM Printout Creating a Custom default QM print template MS Macros Slide 35

36 *Customizing the default QM print template (1/2) Slide 36

37 Item-level auditing in the QM Printout (3/3) Item-level auditing enhancements Slide 37

38 Appendix D: Business Process Documentation Yes, we thoroughly documented our business processes and continue to keep it current. No, we do not regret it. The ROI exceeded expectations. Slide 38

39 Glossary Architecture Terms Architecture Vision Infrastructure Architecture Business Architecture Spaghetti Architecture Complexity for spaghetti software architecture can be calculated as multiply for all systems, SOA implementation complexity can be calculated as simple addition. IT complexity can be calculated as a Session's Summation of Bird's Formulation of Glass's Law (Sessions, The IT Complexity Crisis: Danger and Opportunity, 2009). In simple terms if we have 3 systems each with 10 states, complexity for spaghetti one will be 10x10x10 = 1000, if we partition this SOA style complexity will be = 30. Standards industry-wide specifications that are adhered to within an organization. Specifications standards in-the-making. Best practices Sometime called conventions, are those things experience teaches us we should or should not do. ROI Return On Investment Spaghetti repository see Spaghetti Architecture above and use your imagination Slide 39

40 THE END Slide 40