Project Close-Out Discussion

Transcription

1 PROJECT LESSONS LEARNED REPORT Project Name: Business Intelligence (BI) Prepared by: Diane Kleinman Date: June 15, 2009 Project Close-Out Discussion A Lessons Learned meeting was held on 6/12/09. The summarized lessons learned survey results are attached to this document. Attendees: Vel Angamthu Wendy Berkowitz Wayne Bowker Aaron Demenge Michael Garza Jim Green George Hardgrove Janet Heller Bill Kanfield List this project s biggest successes. Description We have a more organized reporting structure. We were able to remove a lot of reports that weren t being used and we have reports that actually work. The tool will allow users to write their own reports and these can be modified on an ongoing basis Janet Heller Bill Kanfield Ann Lundholm Ron Mapston Peggy McCarthy Tammy Nelson Bill Paulus Jennifer Pierson Shari Zeise Factors that Promoted this Success The capabilities of BI allow for a more organized reporting structure A benefit of this project was time was taken to examine existing reports and to remove those reports that no longer were needed or did not work. BI allows more than just a few people to have the capability to create reports. There is no bottleneck like there was before when only one person knew and was able to write reports that are needed. Now users can write their own reports and reports can be modified on an ongoing basis. List areas of potential improvement along with high-impact improvement strategies: Category Project Shortcoming Lesson Learned Project Management There are still questions around whether or not to still treat this as a project and let the team make decisions on how to move forward with operational monthly reports. FM Leadership needs to determine if this project should continue with a project structure in place. There needs to be a focus on completing operational Project Communication The right people were not always included on project teams and sometimes the teams needed to change mid-stream as the project requirements changed. monthly reports. Continually monitor the makeup of project teams to make sure that the right people are included when requirements change. LESSONS_LEARNED_REPORT BI Project Page 1

2 Project Close-Out Discussion Project Roles & Responsibilities Requirements Testing Weekly project status reports did not always accurately reflect the real-time status of the project. These reports became more valuable towards the end of the project because they more accurately reflected what was happening. Part of the reason for this is that there were sometimes conflicting reports from project teams and there were day-to-day changes in the status of issues. There were other projects besides BI that also had priority and resources were pulled in multiple directions (e.g., 8iR2, PeopleSoft, Pillar projects). Team member did not lack interest or commitment; there were just competing priorities that made it difficult to dedicate the time needed to meet all scheduled deadlines. As an organization, we need to figure out what our project capacity is. Key decision makers were not involved early enough. Requirements from top level management came in too late. The end that we had in mind was different from the boss s end. Need to formalize the report request process. Some vetting is required to do any monthly and ad hoc reports. Sending an with a report request is not enough. A design document that clearly defines business rules needs to be done for any new universe data (e.g., parts universe). As new reports/needs were defined, then the design changed and that impacted previously created reports. I felt like I was working on the beach and every so often the tide would come in and wash my work away. Status reports should be more direct and reflect what the real project issues are. This can be challenging when things are moving quickly on a project and the status report is just a snapshot of what is happening at the time. FM Leadership needs to determine project priorities across the organization and this should be clearly communicated to management so resources are not pulled in multiple directions. Project direction needs to be a top-down approach with input from top level management earlier in the project rather than later after requirements have already been defined. Consider formalizing the request process for monthly and ad hoc reports. A design document needs to be done before bringing in someone to do the design work. This will help eliminate any confusion and will save time in the end. A test environment needs to be set up early in the project. LESSONS_LEARNED_REPORT BI Project Page 2

3 Project Close-Out Discussion Training FM needs to determine how reports will be used by FM users. Training plan should be developed for different user roles which includes: - How to navigate in BI - How to read and understand data in reports - What actions should be taken with reports Enter other comments: Project Lessons-Learned Document / Signatures Project Manager: Vel Angamthu I have reviewed the information contained in this Project Lessons-Learned Document and agree: Name Project Title Signature Date Bill Paulus Business Sponsor approval 6/16/09 Wayne Bowker Technical Sponsor approval 6/16/09 Vel Angamthu Project Manager The signatures above indicate an understanding of the purpose and content of this document by those signing it. By signing this document, they agree to this as the formal Project Lessons-Learned Document. LESSONS_LEARNED_REPORT BI Project Page 3

4 Business Intelligence (BI) - Lessons Learned Survey Summary 1. Are you satisfied with the finished deliverable? Very 15.4% 2 Somewhat 38.5% 5 Not Very 38.5% 5 Not at All 7.7% 1 If not satisfied, what could have been done differently? 7 answered question 13 skipped question 0 This is difficult to answer since I don't think the deliverable is finished. I started out as a co-lead of a team but the other co-lead began heading in a different direction working directly with the AVP. Complete lack of up front planning. There no attempt to apply what was learned from the mistakes of Crystal 10. It is not finished. Specs and code were changed up until the last minute. There were no deliverable or spec freezes so it seemed nothing got done. Also some key players were brought in at the final hour with a different vision which caused tons of work to be discarded. The project is nearing completion, but there is a lot of work ahead of us to operationalize this solution. Because of the nature of the project, potential issues continued to surface up to the end of the project. I am not sure what could have been done differently An initial design (or requirements documents) should have been created before development started. 2. How efficient and effective were project team meetings Very 25.0% 3 Somewhat 50.0% 6 Not Very 33.3% 4 Not at All 0.0% 0 What would you change? 6 I really wasn't a part of these so I can't answer this one. Most times not everyone was at meetings. Several times, no one showed. Not clear agendas or deliverables The project dropped from the sky. There was zero research of what they had and what they needed. LESSONS_LEARNED_REPORT BI Project Page 4

5 Each meeting seemed to be a recap of the last meeting. The org structure chart was handed out so many times even when there were no new players in the room. Project team meetings were not necessarily held regularly with one core team. I think the effectiveness of the meetings varied by group. Too many meetings at the beginning of the project and too few at the end. I felt I lost touch with the project somewhat near the end. 3. Was the entire team committed to the project schedule? Very 23.1% 3 Somewhat 46.2% 6 Not Very 23.1% 3 Not at all 7.7% 1 If not, what could have been done differently? 7 answered question 13 skipped question 0 As much as we could be. This was a high priority but management did nothing to help us with the rest of our workload. See #2 above. What project schedule? I feel many competing projects caused people to not be as committed as they could have been. I don't think that team members lacked interest or commitment, but there were competing priorities that made it difficult to dedicate the time needed to meet all schedule deadlines. I think the team was committed to the project schedule, but did not understand all the implications and resulting time requirements until too close to the end of the project. I thought we accomplished a lot near the end. More resource time for internal staff was available than at the beginning. LESSONS_LEARNED_REPORT BI Project Page 5

6 4. How involved did you feel in project decisions? Very 30.8% 4 Somewhat 38.5% 5 Not Very 23.1% 3 Not at all 7.7% 1 If you did not feel involved, what decisions did you feel left out of? 4 answered question 13 skipped question 0 Decisions affecting data were made and not communicated to the entire group. Some of us knew things and others didn't -- made it very difficult to test, make recommendations, etc. It seemed like they wanted to put everything in BI, but didn't stop for a second to consider what was already working and what was not. The initial design. Report Design. Ability to present options to Operations. 5. How efficient and effective was communication between the project sponsor, project manager and team members? Very 15.4% 2 Somewhat 53.8% 7 Not Very 15.4% 2 Not at all 15.4% 2 What could have been done differently? 7 answered question 13 skipped question 0 I did not hear much from the sponsors and when I did it was too late to do anything about it. Vel did his best to keep everyone moving toward the same goal, but it feels like we haven't reached a logical conclusion. Goals and planning would have been a good start. The project sponsor that represented my area did not communicate results from those meetings to our organization. Weekly status meetings and formal communications were very helpful. Sometimes communication was done at a personal level, rather than a team level and this made it difficult to ensure all team members were equally informed. I didn't do very well at first but I put more emphasis on this towards the end. Needed to have more input towards items that were assigned to me at the end of the project. Time for completion was sometimes unreasonable and didn't felt I got enough advanced notice. LESSONS_LEARNED_REPORT BI Project Page 6

7 6. How clearly defined were the objectives for this project? Very 16.7% 2 Somewhat 41.7% 5 Not Very 33.3% 4 Not at All 8.3% 1 If not well defined, what could have been done differently? 5 I'm still not sure what is in and out of scope. Never saw a single document. At a high level, the business objectives and benefits for this project were clear and well documented. This project was different than many technical projects because the objectives became more defined as the project progressed. Requirements were defined incrementally which made the project more difficult to manage There did not seem to be a very clear scope for this project. 7. How clear were you on your role in the project? Very 41.7% 5 Somewhat 25.0% 3 Not Very 16.7% 2 Not at all 16.7% 2 If your role was not clear, what could have been done differently? 3 Part of this is due to my manager not communicating this information. They could have planned resource needs in advance. I felt my role (or my perception of my role) changed throughout the project. LESSONS_LEARNED_REPORT BI Project Page 7

8 8. How effective was the requirements identification process? Very 0.0% 0 Somewhat 41.7% 5 Not Very 41.7% 5 Not at all 0.0% 0 I was not involved in this step 16.7% 2 If not effective, what could have been done differently? 5 The fact that we have very different customers who need different things from a report was never validated. We haven't completed this yet! The desired infrastructure was clear, but specific reporting needs were developed throughout the project by business owners. Requirements sometimes seemed to be a moving target and some key decision makers were not involved early enough in the project to ensure all requirements were clear. Once again, sometimes requirements were identified incrementally, which is different from a more traditional project and that makes it more difficult to assess the identification process. Not enough of this was done up front. Requirements were fairly well known for this phase of the project. Still due to the nature of the project, new potential uses continued to come up which sometimes diverted attention. LESSONS_LEARNED_REPORT BI Project Page 8

9 9. How effective was the design (or implementation specifications)? Very 25.0% 3 Somewhat 25.0% 3 Not Very 16.7% 2 Not at all 8.3% 1 I was not involved in this step 25.0% 3 If not effective, what could have been done differently? 3 We were told BI is a design and build as you go along. But as new reports/needs were defined, then the design changed and that impacted previously created reports. I felt like I was working on the beach and every so often the tide would come in and wash my work away. More University employee input may have been beneficial earlier in the project. Globus employees were sometimes left to use their judgment instead of getting clear direction. Not enough of this was done up front. 10. How effective were project team specification or design reviews? Very 16.7% 2 Somewhat 16.7% 2 Not Very 25.0% 3 Not at all 8.3% 1 I was not involved in this step 33.3% 4 If not effective, what could have been done differently? 3 I did not see design reviews. I saw finished product and was asked to review it. Changes were made without notifying the correct people. So people were using the universe or trying to test data and were not getting the result expected (because they did not know a change was made). Liked the folder structure meeting(s). LESSONS_LEARNED_REPORT BI Project Page 9

10 11. How effective was the functional specifications? Very 0.0% 0 Somewhat 33.3% 4 Not Very 16.7% 2 Not at all 8.3% 1 I was not involved in this step 41.7% 5 If not effective, what could have been done differently? 3 Everything seemingly went into the Universe. I don't think many people will be able to navigate it. And that will be worse as people move around. Where are the written specs? Since this project did not follow a traditional waterfall approach, the iterative nature of the specdesign-test- implement makes it more difficult to assess the steps individually. Not sure we really had any for the most part. 12. How effective was the test plan? Very 0.0% 0 Somewhat 63.6% 7 Not Very 9.1% 1 Not at all 0.0% 0 I was not involved in this step 27.3% 3 If not effective, what could have been done differently? 5 answered question 11 skipped question 2 Data validation continues to be a challenge. Test plans were not clearly defined and were different depending on the type of data being analyzed. Depends on the area. Not all areas had the same level of detail in their test plans. Plan was ok but I only knew my part. Didn't have time to fully complete. Will continue to find issues as the system is used. LESSONS_LEARNED_REPORT BI Project Page 10

11 13. How effective was the training plan? Very 33.3% 4 Somewhat 16.7% 2 Not Very 16.7% 2 Not at all 8.3% 1 I was not involved in this step 25.0% 3 If not effective, what could have been done differently? 6 Is there a training plan? There was much discussion about a need for one, but to date, I haven't seen one. A number of people went to two days of training. But what about the many people that will be accessing the reports going forward? Key members are trained, but there is a remaining activity to train and deploy for a wider FM audience to be effective users. The training plan was very clear, but the implementation steps were not completed exactly as defined due to time constraints and staff turnover Didn't really have a training plan. I believe this is still being worked out? 14. How effective was the testing process? Very 8.3% 1 Somewhat 50.0% 6 Not Very 8.3% 1 Not at all 8.3% 1 I was not involved in this step 25.0% 3 If not effective, what could have been done differently? 4 It was effective, but too short and too much crammed into the last two weeks. It was frustrating because you could test a report and be happy, then something would change some where unrelated and your report change and you would have to re-validate it. Detailed and defined testing should have started early in the project to provide adequate time for changes and retesting Depends on the area being tested. Not all areas were tested the same. LESSONS_LEARNED_REPORT BI Project Page 11

12 15. How effective was the interaction/cooperation between technical subteams? Very 25.0% 3 Somewhat 41.7% 5 Not Very 0.0% 0 Not at all 8.3% 1 I was not involved in this step 25.0% 3 If not effective, what could have been done differently? 4 Again, Vel tried but attendance was scarce. Seemed everyone had their own universe and there was little cooperation between them. And at times a change in one universe could negatively impact reports in another group. Changes to the universes needed to be better communicated and agreed to before the change. There is always a balance between the time invested by team members, and the benefit of increased coordination. At the beginning this was not very good. At the end of the project this was much better. 16. How effective was the deployment process? Very 9.1% 1 Somewhat 45.5% 5 Not Very 27.3% 3 Not at all 0.0% 0 I was not involved in this step 18.2% 2 If not efffective, what could have been done differently? 5 answered question 11 skipped question 2 It hasn't been deployed yet. I think efforts to date have been effective but there is more to do. There was not a traditional deployment process in this project It hasn't really been officially deployed. It is currently in this half deployed half dev status. System was deployed as it was developed. LESSONS_LEARNED_REPORT BI Project Page 12

13 17. For the next project, how or what could we improve on in the way the project was conducted? 8 answered question 8 skipped question 5 Management needs to do a better job of communicating to their staff and adjusting workloads to accommodate the project. Define the scope and budget before you start. A code and spec freeze well before the final week of the project would be a good start. But there were many decisions that were delayed early on that caused delays along the way...a software license, defining what Tree to use, are two such examples. More formal requirements gathering, project plan, and agreed upon deliverables at the onset would of helped. As well as commitment by the entire team. I thought Vel and his team did the best they could with what they we given. Even though there could of been things done better, we still accomplished a lot and have a foundation for future growth, if we use it correctly. For a future project of this type, I would like to see USIT take ownership of the application support sooner in the project. If it is possible, to more completely define the scope of this type of project early in the project that should be done. In addition, input for executive decision makers should be received throughout the project Have a defined scope, functional specifications and design specifications. Also make sure the roles are more clearly defined. Have better communication between all of the teams. Do a better job defining requirements. Get sign off from more people at more levels of the organization. Schedule the system down time more in the off-hours. LESSONS_LEARNED_REPORT BI Project Page 13