Invited Paper at Software Process Workshop (SPW) Bejing, Mai 25-27 2005. Software are Processes Too Jacky Estublier LSR-IMAG, 220 rue de la Chimie BP53 38041 Grenoble Cedex 9, France Jacky.Estublier@imag.fr Abstract. A process defines the way activities are organized, managed, measured, supported and improved to reach a goal. It has been shown, 15 years ago [1] that processes are software too; more precisely that their description can also be software. We hypothesize that a system can be characterized by its goal and by answering the questions: why, what and how. We show that software process work investigated only a tiny subset of processes, where only the how have been addressed. Meta-process research tried to address the why to change a process model, but was largely unfruitful. This paper first relates processes, software production and humans in the framework of the meta pyramid proposed by the OMG MDA. We show that programs and process models are fully similar, but not at the same level in the meta pyramids. Therefore the claim: software are processes too. The meta pyramid framework is used to show and contrast new and original potential uses of process technology. It is shown in particular that strategic software management requires a kind of process support where the what is not humans, but the software itself. Finally it is shown that autonomic computing will soon require process support where the why, the what and the how will have to be fully formalized and the process models automatically executed. We believe that this new and demanding context will foster new research on process modeling and support. 1. Introduction A process is the way activities are organized, managed, measured, supported and improved to reach a goal. For example, in the software process, the activity is software development (design, implementation, test, ) and the goal is to deliver the planned software product. A process model is the formal expression of a part of the process, with the goal to understand, communicate, improve, support or automate the process. Process technology supports a process in order to consistently reach the goal within predefined time, budget, and quality constraints. The process, therefore, encodes a part of the company know-how which otherwise is only in the performer's head. It is acknowledged that this (partial) transfer of knowhow allows the company to master its assets, to transfer more easily the know-how to newcomers, to increase repeatability, and to allow process analysis and improvement.
From this perspective, in the software process community, the goal is the repeatability of the process with optimal human resource consumption, focusing on how the job (designing, developing and maintaining a software) should be done. The job is performed by humans on documents (what) on the time scale of a software project (weeks and months) [2] [3] [4]. It is interesting to see that any interactive software system also has a goal to support (and constrain) human activities in order to reach a goal in a repeatable and predictable way, optimizing the human resource. From this perspective, interactive software is a process support system. How, is often defined by the functions the system supports and what is often the values of some data managed by the software system. Time scale is minutes. The software process line of work, and most interactive programs, hypothesize that the context is constant (technology, market, goals). But the context permanently evolves, on a larger time scale (years), and successful software (and models) must evolve to adapt to its changing context. This long-term evolution is also a process, often carefully planned (strategic view of the product). The goal is to adapt the software product (what) to new markets and technologies, changing (how) the software product content. The time scale is years. This is why software evolves. So far, this kind of process has not been formally described and supported. Today, in autonomic computing for example, a software must evolve continually at execution, changing its architecture and its features, under the constraints of its environment. This is typically what a process is. The goal is to adapt to evolving contextual conditions, why is to maintain an optimal software service, what is the software system, how is architectural and feature changes; in the time frame of seconds. Because of the very short time frame, these changes must be performed fully automatically, and therefore the why, how and what must be formalized. We discuss these new dimensions of processes, at the light of the MDA framework, and we show what are the differences and synergies. 2. Meta-pyramids The meta prefix is used very often in research, but it is a relative notion. A metalanguage is a language that describe languages; meta-data is data that describes data, and a meta-model is a model that models model. The meta notion occurs at different levels, in different domains and under different terms. 2.1. The software meta-pyramid The meta notion can be composed. For instance a meta-meta-model is a model that describes meta-models. Clearly each composition step leads to a level that is increasingly difficult to grasp. In practice after a few levels of abstraction, adding more levels is not useful. Figure 1 presents the classical meta-pyramid.
The language technology provides a good example of meta-tower. The Backus Normal Form (BNF) is a meta-language that is useful for describing languages. One might use the BNF grammar to describe the Java or C++ grammar. The Java grammar describes how to write valid Java programs [5] [6][7]. M+3 Meta - Meta - Model MOF M+2 Meta - Model UML MM C# MM Cobol MM Java MM EJB MM Z MM M+1 Model M+0 Instance ACME Banking Apps FOO Banking Apps Z Bookstore Apps FOO Banking State 10am ACME Banking State 3pm Z Bookstore state 9pm alternatives # of entities Figure 1 The MDA Software pyramid [8] The relationship between two levels is the conformity relationship. Conformity can be defined in different ways. It can be the instantiation conformity (an instance conforms to its class); it can be linguistic conformity (a sentence conforms to the language in which it is written), ontological conformity (an instance conforms to the definition of its ontological class) and so on. MDA uses instantiation conformity between M+0 and M+1, linguistic conformity otherwise. At the lowest level (M+0), instances (or objects) represent abstractions of real world entities (e.g., tom is a client of the ACME bank). Each state of the program execution is a model (an abstraction) of a state in the real world. The program is a model of the evolution of the real world. M+1 is the level where programs reside. Program are models describing the application concepts (e.g., client and account ) and the rules that govern all possible states (e.g., balance>0 ). The meta-model level (M+2) describes the concepts that are used to describe models. For instance the concepts of class and association are used in UML class diagrams, the concepts of class, method, field, and inheritance are used in Java programs, etc. At the meta-meta-model level (M+3), there is the MOF. The MOF is used to describe itself, and to describe arbitrary meta-models including meta-models for UML, Cobol, Java, C, etc. Roughly speaking the MOF is the equivalent of the BNF in the context of UML. 2.2. The Software process Meta-Pyramid Aside from the MDA pyramid, called here the technology pyramid, we can find the human pyramid, representing the humans working at each level. In this pyramid, the
conformity relationship is informal: the persons at one level define the tools and methods to be used at the next lower level [9]. At the model level are software developers; they produce the software. One level higher, the software experts are designing and developing the technologies needed for the developers, including programming languages (compilers), middleware and other piece of enabling technology. These persons are much less numerous than the software developers. At the meta-meta-level are a few specialists that develop (meta) tools for the software specialists (compiler compilers, meta parsers and so on). As claimed long ago, a software process is itself software [1], therefore, the process pyramid is also a technology pyramid. But process execution supports the work performed by the software developers, which are the process' end users. The process model, developed by the software experts, specifies how the software developers' job has to be done. The Process Support System (PSS) is the language (and associated support tools) in which models are written; it is created by metaware experts. We find the usual technology pyramid, one level higher than the traditional one, showing clearly that the claim software processes are software too is justified, but one step higher. In software processes, software production is supported, the goal of the processes is support and automation. The software process community has considered the most critical issue to be the modeling and support of activities in a non-deterministic world; non-deterministic because the humans that perform the job (who) are nondeterministic. In these software processes, human are producing, consuming and changing products, which are the final product, the targeted software (what). To increase efficiency, predictability, and reduce errors, the process, when supported, prohibits the user from doing anything, and instead lets him perform only some predefined task, in a predefined order; usually the process engine handles the products on behalf of the user. Software Humans Software Process Enabling technology Program Metaware experts Soft. experts Programmers PSS Process Model Process execution Program execution Software Engineering End users Build Support Process Enginering Figure 2 Software engineering and process engineering
Now consider an interactive program. The program simply manages some data (product) on behalf of its users, it prohibits the user from doing anything with the data and instead proposes only predefined operations (activities) in a predefined order; it handles the issues around concurrent work on the same data. The program is very similar to a process model, and the machine to a process engine. We can say the program supports its users in pretty much the same way as a process supports its users. A model (program) expresses how humans are supposed to interact with software artifacts (data); it expresses the sequence of actions (how) to be performed by these humans of these artifacts (what) to reach a goal. In both cases the goal is the repeatability of the process, the consistency of the resulting data and the optimization of human time. This shows that our claim software are processes too is fully justified. The relationships between the human and technology pyramids are build and support. At a given level, the humans are building the tools at the same level in the technology pyramid. As an exception, at the base level, the end users are not (necessarily) developing any software. The execution of software succeeds one level down in the technology pyramid, using the instantiation conformity (see fig 1). Therefore, these tools, when executed (1 level down) are supporting the work performed by the human at that level. It is the typical relationship between a process and the humans it manages or supports. This consideration is true for any interactive software; this is true for the Process Support System (PSS), that supports the software expert in defining the process model, and it is also true for most of the enabling technology found at the M2 level of the technology pyramid: IDEs and other tools are supporting the software engineers work. This is why we claim that software are processes too. This analogy, once established, makes it interesting to revisit process technology and its use. 3. New uses of processes 3.1. Program coordination Orchestration and choreography are nice names for the coordination of independent programs. Indeed, it means that the what are programs and that the goal is to provide control to these program in order to reach a goal. These program can be seen as performing an activity, in which case orchestration and choreography can be seen as a process, with a goal to implement a complex application, acting on other programs (the what), providing control to these program (the how) [18]. In contrast with software processes, the job is not performed by a human, but by programs. Programs are usually deterministic, hence, coordination languages are seen as programming languages, with the same characteristics and constraints. But communication often occur through the internet, which is not fully reliable, introducing another reason for non-determinism. This is why exception handling is an
important issue. The point of program coordination, since it is fully automatic, is to get clear an unambiguous semantics; it is a microprocess line of work [10] [12][14]. It is interesting to see that this kind of automated coordination is recognized by the software engineering community as being a process (under name workflow), even if no humans are involved in their execution. Indeed, process (or workflow) driven XX is becoming a common expression to indicate that process technology is used to coordinate XX [15]. 3.2. Meta process In most work on meta-processes, the meta-process is not the level above the process in the meta pyramid, but a completely different process, which (tries to) support the software experts when updating/changing a process model, in order to solve the discrepancies found during process execution (process instance evolution) [11], to adapt the model to new methods and tools (process evolution), or to improve the model (process improvement). Indeed the work on meta-processes were among the first attempts to formalize the know-how of software experts when deciding to react to contextual changes during process execution. This know-how is the analysis of the changes, and the decision of what changes are to be performed in order to maintain some fundamental properties of the process (the why). Considering that the process model is also a program, the traditional software process, defining how to change the model, can be applied [17] [3]. Meta Process Humans Software Process PSS Process Model Process execution Metaware experts Soft. experts Programmers PSS Process Model Process execution End users Figure 3 The Meta-process pyramid It is interesting to see that virtually all the work performed on meta-processes addressed why to change the model, and what to change; leaving how to change to the traditional software process. This is an important difference. Unfortunately, the least we can say, is that meta-processes have not been successful so far. This is not surprising, since modeling the why (modeling
motivations, choices in non deterministic space, behavior in non planned context..) is intrinsically more complex than modeling the how. The time scale of such meta-processes is weeks for process instance evolution, but years for process adaptation and software improvement. 3.3. Process for strategic software evolution The long process by which a software product evolves from its current state to a future desired state is often carefully planned. For example, a development plan may state that product XX, currently on Linux must be available also on Windows XP, must support multi-threading and must provide support for YY and ZZ features in the next 2 years. For a professional product, it is likely that the long list of changes to be performed to reach the goal will be analyzed and the process will be identified (steps, duration, resources ). Companies would like to have support during this long period of time, for example, for each change to be performed (e.g., normal maintenance) to assess if it is consistent with the long term goal or not. This is similar to civil engineering: before performing work is an area, checks are performed to see if there are conflicts with other planned future work (highway, ). This is typical software process: it formalizes an evolution to reach a goal (the future desired state of the software), it contains a lot of know-how, and has to be performed in a highly non-deterministic context (technology and market evolve permanently at that time scale). This long-term process is different from the usual one in the sense that it focuses on the structure and technical content of the software (the what) while traditional processes focuses the humans and documents (semantically neutral from the process perspective). It defines the changes (what) that must be applied to the content and structure of the software, in order for the software evolve as planned. The entities the process acts upon are not humans, but the software itself, not files, but its content and structure. Strategic Process Software PSS Process Model Process execution Years Technology, needs and market Evolution Enabling technology A CME Banking Apps Program A CME Banking State 10am Program execution Figure 4 Strategic software evolution It means the strategic process should control and coordinate the different production processes (governing how the changes are to be performed). In general, it
involves the coordination of two or more different processes, possibly handled by different process engines, in different geographical locations. These issues have been addresses by the Workflow Management Coalition (WfMC), but with many restrictions [20][21]. The problem still needs to be addressed in a more general context [19]. 3.4. Process for dynamic software evolution The fact modern software applications are gradually supported by a network of computers, has dramatic consequences. Provided the number of machines potentially involved in such an application (up to thousands), the variability of the network bandwidth and availability, and even the mobility of some machines (laptops, PDA, phones), the administration of these complex software application cannot be done manually. It gives birth to so called autonomic computing systems. These systems are commonly defined as exhibiting self-monitoring, self-configuration, selfoptimization, self-healing, and/or self-protection [22][23]. Software Autonomic Process Enabling technology ACME Banking Apps Program ACME Banking State 10am Program execution Model of Evolution Seconds PSS Process Model Process execution Figure 5 Dynamic software evolution This means that autonomic systems are permanently changing their architecture and their features in response to the changing context. The goal of these changes is to maintain a service in an evolving and non-deterministic world. The why of these changes is that something occurred in the environment which hampers the software application to provide an optimal service (or no service at all). An autonomic software system must analyze the new context, and decide what to change in order to restore optimal service (the why). Since the autonomic system performs the change, it must therefore know how to perform the change. The needs and ambitions of autonomic systems is very similar to those of metaprocesses, but with a major difference: meta-processes have a time scale of years, and therefore are at least partially performed by humans. In autonomic computing, these changes occur in the time frame of a few seconds and are performed on many machine simultaneously: decisions must be made and performed fully automatically. It is very interesting to consider that autonomic computing will require processes that fully formalize the why, the what and the how.
4. Conclusion Process technology has explored a number of issues and technologies, but with implicit hypothesis : humans are doing the job, the process expresses how to perform the job, in a rather stable world. The difficult issue is the non-deterministic of humans. From the process point of view, the software product (the what) is a set of documents whose content is not really of concern. Work on meta-processes tried to address other issues, the product (the process model) being known and changed to adapt to changes that have occurred in the environment. The meta-process work tried to address the what and the why. Unfortunately, this work was not so successful. In recent years, process technology has been used for a number of other purposes, in which the what became the content and structure of programs (or models), where the major issue is no longer human non-determinism, but the non-determinism of context evolution, either over very long periods of time (market, needs and technology) or very short periods of time for widely distributed and dynamic applications executing in non-predictable environments (mobility, bandwidth, availability, services and so on). This evolution considers software as both a process and the subject of a process, blurring even more the traditional distinction between processes and software. We believe that this new and demanding context should resurrect the old work performed on meta-processes and foster a new trend of research activity on process support in general, and on how to formalize the why and what in processes. Acknowledgement. We are grateful to Jean-Marie Favre for the discussion and initial ideas leading to that short paper. I hope an extended version will be produce in common soon. References [1] L. Osterweil. Processes are software too. Proceedings of the 9th international conference on Software Engineering. Monterey, California, United States, Pages: 2 13. 1987 [2] Conradi, R., Fernstrom, C., Fuggetta, A., Snowdown, B. Towards a Reference Framework for Process Concepts. In Proc. EW SPT 92, 2nd European Workshop on Software Process Technology, Trondheim, Norway (Sept. 1992) [3] J.C. Derniame (ed). «Software Process : Principles, Methodology and Technology». LNCS 1500. Springer Verlag 1999. [4] B. Curtis, M.. I. Kellner, J. Over. "Process Modeling", Communications of the ACM, Volume 35, No. 9, September 1992. [5] Dsouza, D. Model-Driven Architecture and Integration: Opportunities and Challenges Version 1.1, document available at www.kinetiuym.com, February 2001. [6] OMG Model Driven Architecture A Technical Perspective Architecture Board MDA Drafting Team Draft 21st February 2001
[7] John Poole. Model-Driven Architecture: Vision, Standards And Emerging Technologies. ECOOP 2002. [8] J.M. Favre. CacOphoNy: Metamodel-Driven Architecture Reconstruction. Working Conference on Reverse Engineering (WCRE 2004). November 2004, Delft, The Netherlands [9] J.M.Favre, S. Ducasse "Using Meta-Model Transformation to Model Software Evolution". ATEM2004, Workshop with WCRE 2004) Delft, The Netherlands, November 8th-12, 2004. And in Electronic Notes in Theoritical Computer Science [10] C. Godart, F. Charoy, O.Perrin and H. Skaf-Molli: Cooperative Workflows to Coordinate Asynchronous Cooperative Applications in a Simple Way.. 7th International Conference on Parallel and Distributed Systems (ICPADS 00). Iwate, Japan, July 2000 [11] P.Y. Cunin, R.M. Greenwood, L. Francou, I. Robertson and B. Warboys. The PIE Methodology - Concept and Application. 8th European Workshop on Software Process Technology (EWSPT). Dortmunt, Germany. June 19, 2001. [12] G. Valetto, G. Kaiser. Using Process Technology to Control and coordinate Software Adaptation. ICSE, Portland May 2003. [13] Wise, A., Cass, G.A., Staudt Lerner, B., McCall, E.K., Osterweil, L.J., Sutton Jr., S.M.: Using Little-JIL to Coordinate Agents in Software Engineering. Proceedings of the Automated Software Engineering Conference ASE 2000, Grenoble, France (September 2000) 155-163. [14] Estublier, J., Sanlaville, S.: Business Processes and Workflow Coordination of Web Services. IEEE International Conference on e-technology, e-commerce and e-service (EEE-05), Hong Kong (March 2005). [15] Estublier, J., Villalobos, J., Le, A.T., Sanlaville, S., Vega, G.: An Approach and Framework for Extensible Process Support System. 9th European Workshop on Software Process Technology EWSPT 2003, Helsinki, Finland (September 2003). [16] Leymann, F.: Web Services Flow Language (WSFL 1.0). IBM Software Group (May 2001). [17] Minh Ngoc Nguyen, Reidar Conradi: "The Software Meta-process: Taxonomy and Assessment", p. 167-175, Proc. 3rd Int'l Conference on Software Process (ICSP'3), IEEE- CS Press, 10-11 Oct. 1994 Washington. [18] Leymann, F., Roller, D.: Workflow-based applications. IBM System Journal. Vol. 36, No.1 (1997) 102-123. [19] Jacky Estublier, Pierre-Yves Cunin, Noureddine Belkhatir. «An architecture for process support interoperability. ICSP 5, Pages 137-147. 15-17 June 1998 Chicago, Illinois, USA. [20] Workflow Management Coalition: Interface 1: Process Definition Interface. WfMC TC- 1016 (August 1996) [21] Workflow Management Coalition "Workflow Standard - Interoperability - Abstract Specification", Document Number WFMC-TC-1012, Version 2.0b, 30 November 1999. [22] IBM Systems Journal: Special Issue on Autonomic Computing, Vol. 42, No. 1, Jan 2003. [23] J.O. Kephart and D.M. Chess, The Vision of Autonomic Computing, IEEE Computer, Vol. 36, No. 1, pp. 41-50, Jan 2003.