New Trends in IT Industry Why Worry? Giuliano (Giulio) Antoniol, Canada Research Chair Tier I in Software Change and Evolution The Software Cost-effective Change and Evolution Laboratory Ecole Polytechnique de Montreal antoniol@ieee ieee.org Page 1
Well we all agree that almost every company depends on software technologies to function, the challenge is that the technologies and software applications are constantly changing and adapting to the needs of users this process of change is risky, since unplanned and undisciplined changes in any software system of realistic size risk degrading the quality of the software and producing unexpected side effects Read: evolution covers up to 60 % of TCO add to this that bugs are politically correct do you agree? Page 2
bugs politically correct Are they really politically correct? Page 3
well Page 4
Indeed they are. Page 5
But Help me please if software is everywhere and software may not be 100 % bug free, if indeed evolution is a driving factor of software TCO Page 6
So Why Why are we struggling to get a research project funded to find PhD students to create a new teaching programs or to introduce software evolution teaching elements in software courses to open new faculty positions Why people from industry is not lining up out of our offices Software engineering is not part of NSERC strategic grants EU FP7 has not evolution or maintenance as a keyword Service and Software Architectures, Infrastructures and Engineering Secure, dependable and trusted Infrastructures Page 7
Let s s do a step back. What are we talking about We deal with an immaterial product an intangible artifact produced and maintained by highly skilled people, people with high technology occupations Page 8
OK but what is a high technology occupation I mean where is it produced or changed and by whom. Page 9
High Technology Occupations - U.S. Department of Labor Bureau of Labor Statistics High-technology occupations are scientific, engineering, and technician occupations They include the following occupational groups and detailed occupations: computer and mathematical scientists; engineers; drafters, engineering, and mapping technicians; life scientists; physical scientists; life, physical, and social science technicians; computer and information systems managers; engineering managers; and natural sciences managers. Workers in these occupations need an in-depth knowledge of the theories and principles of science, engineering, and mathematics underlying technology, a knowledge generally acquired through specialized post-high school education in some field of technology leading up to an award ranging from a vocational certificate or an associate s degree to a doctorate. Page 10
I do not feel comfortable software is not there software engineering is like civil or mechanical engineering amazing Page 11
OK so developers may have a degree in math, computer science, software engineering or they are recycled tourist guide tell me who won ACM ICPC (The ACM International Collegiate Programming Contest) Page 12
Who won ACM-ICPC? (1700 Universities (1700 Universities 82 countries) Rank 2007 2006 2005 1 Warsaw University Saratov State University Shanghai Jiao Tong University 2 Tsinghua University Jagiellonian University - Krakow Moscow State University 3 St. Petersburg University Altai State Technical University St. Petersburg University 4 MIT Twente University University of Waterloo 5 Novosibirsk State University Shanghai Jiao Tong University University of Wroclaw 6 Saratov State University St. Petersburg University Fudan University 7 Twente University Warsaw University KTH - Royal Inst. of Technology 8 Shanghai Jiao Tong University MIT Norwegian University of Science & Tech. 9 University of Waterloo Moscow State University Izhevsk State Technical University 10 Moscow State University Ufa State Technical University of Aviation POLITEHNICA University Bucharest The ACM International Collegiate Programming Contest sponsored by IBM Page 13
We are not there and I feel depress where is our product made or evolved that s not easy you don t need a track to move it Page 14
Who are these guys? Solution Centre Account Subcontract That s easy isn t it? Page 15
More difficult That s not that easy plus Page 16
But there were a lot of spots These spots are related to a new vocabulary: Outsourcing Offshoring Homeshoring Nearshoring Page 17
Do you need a good reason Not to open a Swiss account well of course if you are not living there. or you don t have a pile of money Page 18
Cost US $ per delivered (maintained) LOC average 1997-2000 [H Rubin 2000] Switzerland 27 (1.6) Germany Japan Denmark USA France England Israel Italy Canada Ireland Greece 22 (1.3) 21 (1.2) 19 (1.1) 18 (1.1) 13 (0.8) 11 (0.7) 11 (0.6) 10 (0.6) 10 (0.6) 10 (0.6) 6 (0.4) India 5 (0.3) Page 19
Defects per KLOC [H. Rubin 2000] USA Japan England Germany Israel Italy France Switzerland India Greece Norway Ireland Canada 1.6 1.8 2.1 2.2 2.3 2.5 2.5 2.5 2.6 2.8 3.6 3.7 3.9 I feel bad you tell me: why should I pay 6 times more to go from 2.5 to 2.6 Of course don t ask me an Italian/Canadian to write code Page 20
More Recent Data [neoit[ 2004 cost in US $] Programmer 2-3 yrs. Experience Min, Max India China Philippines Russia Ireland Mexico Malaysia Brazil Vietnam Canada USA 6000 5500 6500 7000 21000 18000 8700 9000 2850 25000 45000 9000 9600 10900 13000 28000 23000 12800 16000 4100 50000 85000 Page 21
don t t touch the phone wages are just 45 50 % of costs! Page 22
But despite hidden costs Communication Systems redundant communication links Physical Infrastructure and Support building, power,. Governance manager cost a lot all over the world Resource Redeployment Management Redeployment Resource Redundancy it is often needed to retain intellectual property and capacity Training and productivity Disaster Recovery and Business Continuity Capabilities Offshore Knowledge Development/Advisory Services Travel costs Exchange Rate Fluctuations Page 23
Do they always gain Nearly 10% of the companies surveyed had negative cost savings. [Aelera Analytics, December 2004] Page 24
A negative cost saving? that s fun but why well do you remember the old and famous mantra of CMM levels We say: Publish or perish well Page 25
Innovate or Perish [C.K. Prahalad Wall Street Journal June 8, 2005] If global firms have to compete effectively for global markets, they must have the ability to improve their cost,, quality, time-to to-market, and capacity to innovate. This requires a continual search for talent and a willingness to change the internal processes of managing to keep ahead of the competition. Because remote development and delivery demands clear documentation, the process capabilities of both the customer and the vendor improve. India has more than 40 software houses with a quality ranking of CMM 4 or 5 Speed of reaction is yet another source of advantage. Page 26
Offshoring Ed Yourdon Remark (2004) in order to make offshoring work, you need a much more formal management structure, and a set of formal, rigorous, well-defined processes that can be carried out by knowledge workers in some other part of the world. In the software business, this means that companies whose IT organizations are operating at SEI-CMM level 1 or 2 will have a tough time sending their development projects to India isn t this adding insult to the injury they are forced to improve the process! Page 27
Offshore projects any problem? [ Outsourcing IT: Over Here vs. Over There - The Promises and the Perils, Aelera Analytics, December 2004]. Page 28
Offshoring [MkKinsey June 2006] Cost advantages is the critical factor for 85% of the respondents plus speed to market, productivity gains, and growth strategy carry. Nearly 65% of the respondents either currently or plan to offshore software development Nearly 50% of respondents are currently using an offshore provider 30% use a subsidiary or a captive unit. 60% of the respondents were targeting over 25% savings by going offshore for software development over 75% reporting that they have achieved at least 75% of their cost saving goal Page 29
Lesson I: adapt and improve processes to go offshore a more controlled set of processes is needed This can lead us to help company offshoring, nearshoring or homeshoring Page 30
What do we have in common? Page 31
We are Off-shored, near-shored, home-shored and relocated researchers be prepared it s your turn now Page 32
R&D are they safe? Up to half of global US software employment could theoretically be conducted offshore. Up to 76% R&D could potentially be sent to offshore locations, along with 65% of support and services. By 2008, it is expected that about 18% of this employment will be send to offshore locations [McKinsey study "The" Emerging Global Labor Market, June 2006 ], Page 33
Remember ACM-ICPC Rank 2007 2006 2005 1 Warsaw University Saratov State University Shanghai Jiao Tong University 2 Tsinghua University Jagiellonian University - Krakow Moscow State University 3 St. Petersburg University Altai State Technical University St. Petersburg University 4 MIT Twente University University of Waterloo 5 Novosibirsk State University Shanghai Jiao Tong University University of Wroclaw 6 Saratov State University St. Petersburg University Fudan University 7 Twente University Warsaw University KTH - Royal Inst. of Technology 8 Shanghai Jiao Tong University MIT Norwegian University of Science & Tech. 9 University of Waterloo Moscow State University Izhevsk State Technical University 10 Moscow State University Ufa State Technical University of Aviation POLITEHNICA University Bucharest The ACM International Collegiate Programming Contest sponsored by IBM Page 34
Lesson II: highly skilled and cheaper personnel can be found at a much lower cost then in EU, Canada or USA they can also do R&D. Page 35
Quoting C. Colombo: there are also opportunities! and the sea will bring new hope to each man [C. Colombo] Page 36
Society changes [R. Florida: the rise of the creative class] People is more inclined to a better quality of life Salary no longer the sole reason to get hired.. What s matter most; Information Week: salary survey 2001 80 70 60 50 40 30 20 10 0 Challenge Flexibility Stability Base pay Vacation Opinions Valued Benefits Technology Job Athmosphere Casual Attire Training Innov. Work Page 37
perhaps We also have a communication problem why students are moving to non technical disciplines? Is there a real reason? Page 38
US High Tech Employment - US BLS [D. Hecker July 2005] Industry % Change 1992 2002 % Change 2002 2012 Pharmaceutical and medicine manufacturing Computer and peripheral equipment manufacturing Software publishers Internet publishing and broadcasting 30.2-24 125 116 23.2-27 70 41 Internet service providers and Web search Portals Data processing, hosting, and related services Computer systems design and related services Scientific research-and-development services 265 39 161 9.7 64 40 54 6.7 Page 39
OK well but where is the market going excellent question unfortunately I don t t have an answer but Page 40
Market increases Software and services still increase more then hardware Companies with Revenues > 1B$ Gartner 2005 100% 80% 60% 40% 20% 0% 15 15 13 11 10 42 37 36 33 34 19 16 11 15 20 17 17 19 20 16 15 15 15 18 21 2002 2003 2004 2005 2006 Year Hardware Software Outsourcing Personnel Other Page 41
SOA a new opportunity [The Economist Teo, Pian Information & Management Volume 41, Issue 4, March 2004, Pages 457-468 468 Aberdeen Group, July 2006, said 90 % of firms have at least one service project under way Page 42
Autonomic New opportunity to move toward Autonomic and self healing systems [H. Muller 2005] The IRST project of an intelligent room a device with the homeostasis ability Page 43
Lesson III: opportunity are there OK! They must go offshore, they are migrating to services integrating business intelligence move to autonomic systems there is the need to integrate and evolve complex system people struggle to collect log and nail down bugs Page 44
Linux ready IBM and HP offer Linux certified and Linux ready laptops Red Hat Enterprise Linux v.4 spans IBM eserver Dell offers PowerEdge servers with RedHat application stack e.g., JBoss Linux desktop with OpenOffice is largely compatible with Microsoft file formats China market Q1 2007: Linux 2.5 %, Windows 41.8 %, Unix 53.9 % Linux expected to increase 30 % year-to to-year Page 45
and beside that According to a recent Forrester report about 80 % of respondent are using open source software for mission critical applications Page 46
OK but where is my customer base Page 47
Global Software Development EDS, Tata are just two examples Open source traditionally adopted distributed, around-the the-clock development and evolution strategies SVN, CVS, Bugzilla,, Mantis, Jikes are the backbones Company specific problems (e.g., environment sync) versus open source challenges (e.g., turnover and no documentation) Page 48
Eclipse an opportunity Plugins can be developed tested and distributed We can shorten the cycle: idea, implement, test distribute Technology transfer cycle can be dramatically shortened Open source, free and de-regulated development projects the final user is the sole judge really around the clock Page 49
Open Source Around the Clock Berlin time zone! Page 50
Open Source Development or Evolution Page 51
Lesson IV: do not forget that company that are going offshore are targeting anything between 15 % to 25 % cost reduction prove the same with your research! How? Page 52
And don t t forget your Mom If you re not familiar with The Mom Test, it s basically a quick way to see if your site is so simple your mom can use it. A variant that can also reveal results fast for you is the Five second usability test, which is kind of similar, but gives you other results. [Jesper Rønn-Jensen, front-end web developer, usability specialist at Capgemini Denmark] Page 53
Lesson V: do not forget that managers, programmers are not researchers when I was a manager my time was spent and planned quite in a different way my time was money... no one will waste time if the benefits are unclear and substantially greater of the pain Page 54
Challenges I How can we effectively quantify pros and cons of large evolution projects? How can we manage and optimize around-the the- clock distributed evolution projects? what are the evolution processes? How can we track, monitor and make programmers aware of project status? Can we design communication schemas to effectively provide information How can we help in building a community, a social network of around-the-clock developers? Page 55
Challenges II Economical viability how can we model it? global development mixed open source, closed source, distributed projects Is there a software evolution peculiarity? Should we just adapt traditional financial models? Open source projects are not driven by revenues how can we get open source people more involved co-design and co-develop technologies and approaches? Page 56
My two really big challenges Remember that we must prove that our ideas work, scale up to sizable systems and basically is economically viable: 1. How do I prove say a 25 % productivity gain? 2. How do I compare my result with yours? Page 57
Quoting Charles Darwin It is not the strongest of the species that survives, nor the most intelligent, but the one that is most responsive to change Page 58
Finally Page 59
Why moving to Montreal A very good reason to move to Canada. Page 60
Any question? Page 61