Efficient Designing Through Interactions Between Two Subscriptions

Transcription

1 Hierarchy and Misalignments in Complex New Product Development Projects Mohsen Jafari Songhori Department of Mechanical Engineering, University of Melbourne, Vic 3010 Australia. Javad Nasiry Information Systems, Business Statistics and Operations Management Department, Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong. Michael Kirley Department of Computing and Information Systems, University of Melbourne, Vic 3010 Australia. Developing complex new products requires firms to break down the product into subsystems and create an organizational structure which ideally mirrors the product architecture. However, empirical evidence on the mirroring hypothesis is mixed and misalignments occur in the product and the corresponding organizational architectures. Misalignments take two general forms: (1) a missing link between two teams responsible for two interacting subsystems results in an unmatched interface and (2) two teams interacting without a link between their respective subsystems cause an unmatched interaction. In a model of product design as a search on a rugged landscape, we model misalignments as design teams searching on a perceived rather than real landscape. As a consequence, type-i or type-ii errors are likely whereby the former causes the teams to reject superior designs and the latter to accept inferior designs. We study the performance deterioration by two measures: the magnitude and frequency of errors. We show that unmatched interactions cause a higher type-i error both in magnitude and frequency. Unmatched interactions and interfaces cause the same magnitude of type-ii error but unmatched interfaces cause a higher frequency of type-ii error. We further study how misalignments affect the convergence behavior of the search process, i.e., the time to converge and the quality of the final design. We find that misalignments affect, though not necessarily increase, the convergence time significantly but they are not a critical factor in the final design quality. We discuss the managerial implications of our results for the new product development projects. Key words : Complexity, Misalignments, Unmatched interactions, Unmatched interfaces, Hierarchy, NK(C) simulation 1

2 2 1. Introduction A380, the world s largest passenger jetliner, began its flights in Due to delays in its development, however, Airbus had to reschedule orders, pay out penalties to customers and lay off part of its workforce. Wiring of the plane was the source of most of the problems. About 500 Kilometers of wiring is needed for the A380. This task was allocated to two groups in Germany and France who independently made slight changes to its design without always informing the other team. Some incompatibilities in wiring were only discovered in the final assembly stage resulting in delays. As a senior Airbus manager put it We perhaps underestimated the complexity of the aircraft (International Herald Tribune 2006). Developing complex new products such as cars and airplanes requires firms to break down the product into physical subsystems that must interact properly to make the product functional. These subsystems and their interactions constitute the product or technical architecture. A team of designers is responsible to develop each subsystem. These product development (PD) teams interact within an organizational architecture with the goal of developing the product by a given deadline and budget. Ideally, the product and organizational architectures should mirror one another, i.e., the network structure of one should correspond to that of the other (Colfer and Baldwin 2010). This hypothesis implies that two PD teams should interact if the subsystems they are responsible to develop interact and vice versa. Though intuitive, the empirical findings do not generally support the mirroring hypothesis: misalignments occur when two subsystems interact while their corresponding teams do not (an unmatched interface) and when two teams interact while the subsystems they are developing do not (an unmatched interaction) (Sosa et al. 2004, Gokpinar et al. 2010). We study the consequences of such misalignments on the performance of PD teams in a model of product development as a search process by a number of teams on rugged interdependent landscapes. We hypothesize that misalignments create perceived rather than real landscapes on which teams search for design alternatives; see Figure 1. As a result, the teams are prone to reject superior designs (type-i error) or accept inferior designs (type-ii error). The errors have two different aspects: magnitude and frequency, i.e., how large the errors are when they occur and how frequently they occur. The choice between the two measures depends on the managers priorities. If the goal is to reduce the overall product development time (and costs), managers should focus on error frequency. However, if the quality of the subsystem design is the priority, size of the errors, whenever they happen, is more important. We conceptualize a PD project as PD teams searching on a landscape to find the best possible design. We simulate the search process by an NK(C) fitness landscape model. In particular, we assume that the subsystem assigned to a team consists of N elements each of which has an average of

3 3 (a) PD teams without misalignments searching on real landscapes. (b) PD teams with misalignments searching on perceived landscapes. Figure 1 Conceptualizing search on a set of interdependent (real/perceived) landscapes. Red lines in panel (b) indicate misalignments. K interactions with other elements. The parameter C represents the average number of interactions between the elements of one subsystem with those of other subsystems. In other words, changing the status of one element in the design of one subsystem affects C other elements in the design of other subsystems. Furthermore, the system is assumed to adapt by doing either incremental or long-jump searches to find a (possibly local) optimal point (Kauffman 1993). We assume that misalignments occur across the boundaries of teams and subsystems rather than within (Sosa et al. 2004). In each stage of the search, we assume that an interface between two subsystems is overlooked with a probability. This means that an interface may be known to teams but teams fail to update themselves on the current status or ignore the interdependency due to overconfidence or workload. We further assume that unnecessary communication between teams with independent subsystems anchors the teams on irrelevant design ideas, or leads to information overload. We then model an unmatched interaction between two teams as one team limiting its feasible local search by imitating the state of an element of the other team. In this setup, we seek to understand (1) whether misalignments degrade the PD project performance and if so which type causes larger or more frequent errors? Unmatched interactions may facilitate coordination among homogeneous teams and increase performance (O Reilly et al. 1989, Zenger and Lawrence 1989). It may have the opposite effect too because teams limited attention is dedicated to unnecessary communication and information (Ancona and Caldwell 1992, Bantel and Jackson 1989). Our results show that unmatched interactions cause larger and more frequent type-i errors. In other words, communications between designers of unrelated subsystems is likely to

4 4 lengthen the development process, cost more and result in a lower quality design. This result implies a boundary condition on the wisdom that unnecessary interactions might be beneficial as they push the design off inferior local peaks (Knudsen and Levinthal 2007). We argue that the benefits accrue only if the interactions are among the teams responsible to develop interacting subsystems. Otherwise, these social ties will lead to time and budget overruns. For type-ii errors, our results do not distinguish between unmatched interactions and unmatched interfaces in the magnitude of errors. However, errors are more frequent with unmatched interfaces. This means that overlooked interfaces between the subsystems lead to accepting more inferior designs by teams and lower the quality of the overall product. Therefore, a dense product architecture requires a dense organizational design to lower the frequency of type-ii errors. Overall, our results highlight the interplay between the organizational and product architectures in new product development projects and argue that the allocation of scarce resources depends on the management priorities. If designing high quality products is a priority, resources should focus on eliminating unmatched interactions. If, on the other hand, the goal is to avoid accepting too many inferior designs, it is more effective to curb the unmatched interfaces, i.e., identifying the connections among the subsystems or using teams with dense organizational interactions and communications. Subsystems in a complex product have various levels of interdependencies. While some subsystems affect the performance of many other subsystems or are affected by them, some others are relatively independent. Therefore, it is possible to place subsystems and the teams developing them in a hierarchy where a subsystem with dense interdependencies takes a high hierarchy position and a subsystem with sparse interdependencies a low hierarchy position. Further, a subsystem with relatively dense interconnectedness takes a medium hierarchy position. We apply a simple rule to develop this hierarchy in our model and answer a second research question (2) misalignments at which hierarchy level affect the performance of product development teams the most? We show that misalignments in the middle cause a higher magnitude of type-i error while misalignments at the bottom lead to larger type-ii errors. These results are counterintuitive as one might expect that resources should be allocated to eliminate the misalignments at the top. Our results show that overlooking the interdependencies at the medium and low levels, such as the wiring system in A380, can lead to low quality designs resulting in costly rework. With respect to the frequency of errors, our simulation results conclusively show that misalignments at the medium hierarchy lead to more frequent type-i errors. However, the frequency of type-ii errors depends on the intensity of interaction among the elements of subsystems. Misalignments at the low level cause more frequent type-ii errors when the interactions among the elements of a subsystem is low (K = 1). As the interaction intensity among the elements increases (K = 4), misalignments at the medium level tend to cause more type-ii errors.

5 5 Overall, we find that misalignments at lower hierarchy levels result in both larger and more frequent errors. Therefore, to reduce the likelihood of errors happening and their magnitude once they happen, it is most effective to eliminate the mismatches at low or medium hierarchy levels. Though how often errors occur and their magnitude provide a perspective to understand how misalignments affect product development processes, the convergence behavior of the search process has important managerial implications too. Convergence occurs whenever no team can increase its fitness value by further local search, i.e., a local optimum design is reached (Mihm et al. 2003). Therefore the third research question we address in this paper is (3) how do misalignments affect the convergence of the search process? Due to computational constraints in simulating NK(C) search processes, we take a slightly different approach and define the convergence as a situation where the majority of PD teams have obtained a better design alternative than the original alternative. In practice, product design teams are sensitive to the time it takes to find a solution. If the design process does not converge in a reasonable amount of time, project managers take remedies such as reverting back to solid base designs or decreasing the number of iterations by freezing the design of some subsystems and changing the design of the remaining until a solution is reached (Mihm et al. 2003). We study two aspects in convergence: (1) the time to converge and (2) the quality of the final design. The two aspects are of managerial importance as time to market and quality of the design are two critical issues in developing a new product specially in competitive markets. We find that misalignments at the medium level lengthen the convergence time the most but, in general, the quality of the final design is not affected by the level at which the misalignments occur. In summary, our results provide a few managerial insights in allocating scarce resources in developing new products. Both types of misalignments degrade the performance of the design process and the managerial resources (cognitive, financial,... ) should focus on limiting unnecessary communications among design teams to avoid rejecting superior designs by teams and on recognizing the interfaces among subsystems to avoid accepting inferior designs. Moreover, we find that misalignments at lower hierarchy levels cause more and larger errors and also lengthen the time to converge to a final design. This finding puts emphasis on managing misalignments at these levels to avoid costly mistakes and time overruns. The rest of the paper is organized as follows. Section 2 provides a review of the relevant work. In Section 3, we describe the mathematical model and how we conceptualize the search process and unmatched interactions and interfaces. We further define the errors and the convergence characteristics that we investigate. In Section 4 we detail the experiments and report the results. Section 5 provides a discussion on the limitations of our work and concludes the paper.

6 6 2. Literature Review Complex systems are made up of a large number of parts that interact in a nonsimple way (Simon 1962). To manage these systems organizations divide them into a number of subsystems that are handled by individuals or teams. Inevitably, boundedly rational decision makers overlook some relevant variables and their interactions (Schrader et al. 1993, Sommer and Loch 2004). Further, individuals and teams may lack full coordination and use obsolete information about other subsystems when solving their own subproblems (Mihm et al. 2003, 2010). Not surprisingly, the empirical findings on mirroring hypothesis are mixed. Colfer and Baldwin (2010) review 102 empirical studies published between 2000 and 2009 and find that about two-thirds of the sample supports the mirroring hypothesis. The hypothesis, however, is rejected in other cases as either collocated, richly communicating groups developed designs made up of largely independent subsystems or independent and dispersed contributors collaborated on highly interdependent designs. As one may expect, the hypothesis finds support mainly in projects run within a firm or across few firms and fails in open collaborative projects. Empirical studies in the complex product development literature also report misaligned product and organizational architectures. Sosa et al. (2004) investigate product and organizational architectures of a large commercial aircraft engine development and find that both critical and noncritical interactions may be unknown to the PD teams. They argue that although the performance effects of these unknown interactions might be low, they can result in a large amount of extra expenditure during the airplane product life of 20 to 30 years. Gokpinar et al. (2010) quantify the mismatches between product and organizational architectures in an auto manufacturer by a coordination deficit metric. Their results indicate that centrality of a subsystem in a product architecture affects the product quality (measured by the number of warranty claims) in an inverted-u shape. That is subsystems of intermediate complexity cause more quality problems. They conjecture that low-centrality subsystems are relatively simple and their interactions are easily managed by organizations. Moreover, organizations allocate more resources to high-centrality subsystems and manage their interactions more intensely which decreases the likelihood of quality problems caused by these subsystems. Sosa et al. (2007) propose two different types of misalignments in the design and development of complex products. The unmatched interfaces occur when the designers of two subsystems do not have organizational ties, e.g. do not communicate, despite the two subsystems being functionally interactive. The unmatched interactions, on the other hand, occur when PD teams of two unrelated subsystems have interactions. With unmatched interfaces, the lack of communication between teams can result in two problems. First, because of interdependencies among subsystems, the overall performance depends on how

7 7 effectively the teams can assess the effects of their decisions on the other interdependent teams decisions. This is the notion of the teams payoff function dependence on the trans-specialty understanding (Postrel 2002). This understanding helps members of one-specialty assess the role of other-specialties in solving a problem and increases the possibility of one team s decisions being aligned with those of the others in the benefit of the overall project. Second, glitches occur often in product development projects (Hoopes and Postrel 1999, Hoopes 2001). A glitch is a costly mistake that occurs in a multi-agent project due to lack of shared knowledge about problem constraints. Glitches however are not limited to highly complex projects. Hoopes and Postrel (1999) provide an example of a new executive information system (Hoopes and Postrel 1999) where system designers spend months designing a system in which reports for different categories of customers, products, and years may generated. However, once the design is to be coded by programmers, they realize that the databases can be searched only on customers and years. The time spent by both groups (designers and programmers) on solving this problem makes this mistake a costly glitch. These arguments and examples suggest that unmatched interfaces are likely to degrade the performance of PD teams through either decreases in trans-specialty understanding or glitches. With unmatched interactions, unnecessary communication among PD teams increases their workload which in turn increases the error rate and causes unexpected problems (Rahmandad and Repenning 2008). The extra workload alters the dynamics and expected performance of PD projects through fire fighting phenomenon, i.e., the allocation of scarce resources to unexpected problems or fires (Repenning et al. 2001, Repenning 2001). Operating in a fire fighting mode causes rework by engineers and managers and budget and cost overruns follow. Unmatched interactions increase the PD teams workload and, consequently, are likely to generate more fires or unexpected problems negatively affecting the performance of product development projects. Although misalignments in product and organizational architectures are expected to degrade complex PD performance, the extent of the effect remains unclear in the literature. The literature on both distributed design (Mihm et al. 2003, Braha and Bar-Yam 2007, Mihm et al. 2010) and distributed search in complex systems (Lazer and Friedman 2007, Baumann 2013) has implicitly assumed aligned product and organization architectures. Recent literature acknowledges their existence and negative consequences (Sosa et al. 2004, 2007, Gokpinar et al. 2010). To the best of our knowledge, we appear to be the first to model misalignments causing PD teams to search on a perceived rather than real landscape and explicitly study the consequences on three dimensions: (1) type-i error or likelihood of rejecting superior design alternatives (2) type-ii error or likelihood of accepting inferior designs, and (3) the convergence characteristics.

8 8 We further contribute to the literature by studying misalignments in conjunction with a subsystem s hierarchy in the architecture of the product. The concept of a subsystem s hierarchy has been studied under different terms in the literature. Sosa et al. (2011) develop a methodology of identifying the hubs in the product architecture, i.e., the subsystems with a disproportional number of linkages. MacCormack et al. (2010) study the core-periphery structure in a sample of software development projects. Tushman and Murmann (1998) observe that products are composed of hierarchically ordered subsystems and categorize some as core and some others as peripheral. In this paper, we adhere to the terminology proposed by Simon (1962) in conceptualizing complex systems as a hierarchy of subsystems. The role of hierarchy, a feature of complex systems and organizations, in performance of complex organizations has been previously examined (Ethiraj and Levinthal 2004a, Mihm et al. 2010). We study whether the hierarchy of a subsystem where a misalignment occurs has a significant impact on the performance of the PD team developing the subsystem. We model the product development process as search on a rugged landscape by teams conducting local search until a local peak has been obtained. An early paper using this approach in operations management is Mihm et al. (2003) where the authors built a model of a distributed design project with interdependent subsystems. Their results illustrate that the nonlinearity and complexity increased with the size of the system. Mihm et al. (2010) employ a similar model to investigate the effects of the organizational hierarchy on solution quality, stability, and speed in distributed search projects. Baumann (2013) identifies the contingency factors that influence the value of integration among decentralized searchers in a complex system. These studies implicitly assume aligned product and organizational architectures. The result of the search process at any time is represented by a fitness value. This concept was first developed by Kauffman (1993) in the biology literature where a fitness landscape function is conceived for a set of complex interactive elements governed by a number of agents such as distributed design teams. The idea was introduced into the engineering design and management literature by Levinthal and Warglien (1999), Gavetti and Levinthal (2000), Rivkin (2000), and Rivkin and Siggelkow (2003). 3. Model In this section, we setup the mathematical model we use to simulate the search process. The model consists of three parts: 1) the characterization of the real and perceived landscapes over which the teams search; 2) a conceptualization of misalignment forms; and 3) defining a subsystem s hierarchy in the product architecture. We describe each part and define the measures we use to evaluate the performance of the search teams. We then formalize the type-i and type-ii errors incurred by teams searching on a perceived vs real landscape. We finally elaborate on how we model convergence and its characteristics.

9 The Landscape Model Consider a product with n subsystems s {1, 2,, n}. Team i {1, 2,, n} is responsible to develop subsystem s i. We model the performance of team i at time t as the outcome of a search process over the landscape of subsystem s i. In the NK(C) terminology, the landscape of subsystem s i consists of n e interacting binary elements that are 0 or 1 at any given time. Therefore, the total number of elements for n subsystems is N = n e n. This means we assume that all subsystems have the same number of elements n e. Denote the state of s i at time t by s i t = {e ij t e ij t {0, 1}, j = 1, 2,, n e }. In other words, the state of the subsystem is known when the states of all its n e elements are known. Therefore, each team has 2 ne s i t = (00110). different design states. For instance, the state of team i at time t when n e = 5 may be Each state s i t corresponds to a fitness value or performance levelf i t for subsystem s i which is the average of the contribution of n e elements of the subsystem. Denote the contribution of element j of team i at time t by f i t (e ij t ). Then: f i t = ne j=1 f i t (e ij t ) n e. (1) We need three ingredients to define ft i (e ij t ): (1) the status of the element e ij t itself; (2) the status of K other elements of team i that interact with element e ij t ; (3) the status of all the other elements of other teams which interact with element e ij t. The second ingredient captures the idea that the elements under the control of a team are interdependent. In particular, the contribution of one element depends on the status of K other elements. The third ingredient is the idea that subsystems are interdependent and the fitness value of element e ij t depends on C elements of each of the subsystems that interact with subsystem i. Denote the set of subsystems interacting with subsystem i as DE i1. For instance, if subsystem i interacts with three other subsystems and C = 2, then the contribution of e ij t depends on 3 2 = 6 elements under the control of those three teams. The second stage of the NK(C) model is to generate the landscape function. Because element e ij interacts with K + DE i C other elements, it follows that there are 2 K+ DEi C possible contribution values. The contribution of the element e ij t is drawn from a uniform [0, 1] distribution. The properties of the fitness landscape are not sensitive to the distribution applied to generate the landscape (Weinberger 1991). The third stage in the NK(C) model is the characterization of the search process on the landscape. At time t = 0, each team is randomly assigned a state and the fitness value of the team is calculated. Thereafter, at each time period t, the teams involve in local search by changing one (or more) of 1 DE stands for dependent.

10 10 their elements status from 0 to 1 or vise-versa. We denote the state of the elements of all teams except team i at time t by s i t. Team i changes the status of some of its elements generates a new alternative s i t+1. Team i accepts the new alternative if it yields a higher fitness value, i.e., s i t+1 = s i t+1, if ft+1( s i i t+1, s i t ) > f i t (s i t, s i t ) (2) Otherwise, it retains the previous alternative and s i t+1 = s i t. At each time period, all teams conduct the search process described above and continue until the simulation time ends (or the search process converges; see Section 3.5) Misalignment forms in the NK(C) search model In absence of misalignments, team i is aware of the state of the elements of the interdependent teams and conducts an informed local search. This case corresponds to the ideal case of instantaneous decision broadcasting among design teams optimizing the overall project performance (Mihm et al. 2003). With unmatched interfaces, a design team makes its design decisions without knowledge of the current status of some of its interdependent teams (Mihm et al. 2003). The team might be unaware of the interdependencies, use outdated information, or simply ignore the connections. We posit that an interface between two subsystems is overlooked by the corresponding teams with probability p ii uf. In other words, the teams are aware of the interface but fail to discuss the interdependency with a given probability. We assume that p ii uf is time-invariant in the simulation model. In Section 4, we describe the process to generate these probabilities. Because each team has n e elements and each element interacts with C elements of team i, there are n e C interactions between elements of team i and team i. Consider an element j of team i. At time t, we categorize the C elements of team i that interact with element j into two sets: A set M i their interdependency is accounted for and a set U i are matched with element j, i.e., are unmatched, i.e., their interdependency is overlooked. We allocate an element from C into one of these sets as follows: at any time t and for each element in C, a random number in [0, 1] is drawn. If the draw is at least as large as p ii uf, we assume the element belongs to M i and otherwise to U i. By overlooking the elements in U i, team i conducts search over a landscape different from the real one. We refer to this landscape as the perceived landscape and denote the fitness value achieved by team i at time t with unmatched interfaces as f i,uf t. The search on the perceived landscape introduces errors in the search process which we will formalize later in this section. Unmatched interactions occur when two teams interact while there is no interface between their corresponding subsystems. The communication between the two teams might anchor the teams on irrelevant design ideas, or lead to information overload. We model an unmatched interaction

11 11 between teams i and i as the former limiting its feasible local search by imitating the state of one element of team i. More formally, at any time t, and corresponding to each team i, we choose one team i DE i according to roulette wheel selection algorithm (Goldberg 1989, Ethiraj and Levinthal 2004b) and define p ii ua as the probability that team i has an unmatched interaction with team i. We generate these probabilities through a process described in Section 4 and assume they are timeinvariant in the simulation. The roulette wheel selection algorithm selects a team i DE i with probability p ii ua i DE i p i,i ua. Once a team i is selected at time t, it will have one unmatched interaction with team i. In other words, team i will copy the state of one (randomly chosen) element of team i in generating a new alternative. The new state s i t+1 has the following properties: s i t+1 = {ẽ i,j t }, s i t = {e i,j t }, s i t = {e i,j t } for j {1,, n e } s i t s i t+1 = {e i j t } = {ẽ ij t } for some j {1,, n e }. We denote the fitness value of team i in the presence of unmatched interactions as f i,ua t 3.3. Misaligned PD Teams in the NK(C) Search Model To address our second research question that misalignments at which hierarchy level affect the performance of product development teams the most, we take an abstract viewpoint to misalignments and do not differentiate between the forms. In particular, we model the misalignments in a rather similar way to that of unmatched interfaces, i.e., misalignments are randomly overlooked interactions. However, once the interaction between an element of a team i and an element of team i is categorized as matched or unmatched, it stays so for the entire simulation time. In other words, the sets M i and U i are time invariant. To generate these sets, we define p ii as the probability that the interdependency between elements of team i and i DE i is ignored and then use the same approach as in Section 3.2. This approach fixes the perceived search landscape for each team and allows us to focus on the performance degradation due to misalignments in different hierarchy levels which we define next Hierarchy in the NK(C) Search Model We assign a team i to a high, medium or high hierarchy level by a measure based on how many teams affect the fitness value of team i, i.e., DE i and how many teams fitness values are affected by team i, i.e., SI i. Therefore, m i = DE i + SI i captures the overall influence of team i in the space of the complex PD project. We can then put all teams in a descending order based on their hierarchy measure such that team i has a higher hierarchy level than team i + 1 if m i > m i+1. We define the three hierarchy levels as follows: team i is a high-hierarchy team if i n, a medium-hierarchy team 3 if n 3 < i 2n 3 and a low-hierarchy team if i > 2n 3.

12 12 To understand which type of misalignment causes more or larger errors, we fix the hierarchy level to consider teams at the same hierarchy level. We then construct the real landscape which contains no misalignments. Afterwards, we introduce unmatched interactions into the landscape and allow teams to conduct search. The teams obtain a fitness value on this perceived landscape at each time period. A simulation run averages the fitness values over 200 time periods to provide the performance level. We repeat the same process with teams only having unmatched interactions. Finally, we change the hierarchy level and repeat the process above. We compute the magnitude of type-i and type-ii errors for PD teams with unmatched interfaces using f i t and f i,uf t as follows: type-i error = type-ii error = T i t=1 ft i ( si t+1 )>f t i (si t ) f i,uf t ( s i i,uf t+1 )<ft (s i t ) T i t=1 ft i ( si t+1 )<f t i (si t ) f i,uf t ( s i i,uf t+1 )>ft (s i t ) f i t ( s i t+1) f i,uf t ( s i t+1), (3) f i t ( s i t+1) f i,uf t ( s i t+1). (4) Type-I error occurs at time t when the search process on the perceived landscape rejects a design alternative while the design would have been accepted on the real landscape (rejecting a superior design). Type-II error occurs when at time t the search process on the perceived landscape accepts a design alternative while the design would have been rejected on the real landscape (accepting an inferior design). Similarly, we compute the magnitude of type-i and type-ii errors for PD teams with unmatched interactions using f i t and f i,ua t type-i error = type-ii error = as follows: T i t=1 ft i ( si t+1 )>f t i (si t ) f i,ua t ( s i i,ua t+1 )<ft (s i t ) T The results are reported in Section 4.1. i t=1 ft i ( si t+1 )<f t i (si t ) f i,ua t ( s i i,ua t+1 )>ft (s i t ) f i t ( s i t+1) f i,ua t ( s i t+1), (5) f i t ( s i t+1) f i,ua t ( s i t+1). (6) To compare the effects of misalignments in different hierarchy levels, we abstract away from different types of misalignments and instead consider two teams in two different hierarchy levels. To compute the magnitude of type-i and type-ii errors using f i t at a given hierarchy level, we have and f i,uf t and for the set of teams

13 13 type-i error = type-ii error = T i t=1 ft i ( si t+1 )>f t i (si t ) f i,u t ( s i i,u t+1 )<ft (s i t ) T The results are reported in Section 4.2. i t=1 ft i ( si t+1 )<f t i (si t ) f i,u t ( s i i,u t+1 )>ft (s i t ) 3.5. Convergence in the NK(C) Search Model f i t ( s i t+1) f i,u t ( s i t+1) (7) f i t ( s i t+1) f i,u t ( s i t+1) Misalignments not only cause errors but can also affect the convergence of the search process. Convergence occurs whenever no team can increase its fitness value by further local search, i.e., a local optimum design is reached (Mihm et al. 2003). We adopt a slightly different approach to examine the convergence behavior in PD projects. In particular, we assume that convergence occurs when majority of teams have achieved a better design than their initial design. We choose this approach for two reasons. First, the complexity of the NK(C) model requires a substantial computational power not available to the authors. Unlike NK search models, NK(C) search landscapes do not contain many local optima and because multiple teams are searching, convergence requires significantly more time. 2 Second, in practice, product design teams are sensitive to the time it takes to find a solution. If the design process does not converge in a reasonable amount of time, project managers take remedies such as reverting back to solid base designs or decreasing the number of iterations by freezing the design of some subsystems and changing the design of the remaining until a solution is reached (Mihm et al. 2003). In experiment 3, where we investigate the convergence (Section 4.3), at any time t we let the PD teams to examine new alternatives. Once the comparison of fitness values of new alternatives s i t+1 with those of the current alternatives s i t is accomplished, we check to see whether the majority of teams. i.e., more than n, have achieved a better fitness value in comparison to their initial fitness 2 value f i 0(s i 0). If this condition is satisfied then we record t + 1 as the convergence time and otherwise continue the search process. The convergence quality is the sum of all teams fitness values at the time of convergence. 4. Experiments and Results In this section, we report the experimental setup and results. We first detail the simulation procedure and ingredients and then discuss the results of three experiments. 2 The number of design solutions in which all teams are in a local optimum is a subset of the total number of all local optima of all teams.

14 14 The parameter N in the NK(C) simulation model is the total number of elements in the landscape, i.e., N = n n e. This number in the literature of complex landscape simulations varies from six to twelve (see e.g., Rivkin and Siggelkow (2003), Siggelkow and Levinthal (2003), Rivkin and Siggelkow (2007), Baumann (2013)). In our experiments, we assume the organizational structure to develop a project consists of five teams with each team controlling five elements, i.e., N = 25. With five teams, our hierarchical organizational structure (see Section 3) allows for at least one team at each hierarchy level. Table 1 shows the network structure for the experiments. Table 1 Network structure. PD Team Our allocation procedure in Section 3.4 then implies that team 1 is a high-hierarchy, teams 2 and 3 are medium-hierarchy and teams 4 and 5 are low-hierarchy teams. The average number of interactions among the elements of a team, K, can vary in [1, N 1] which in our setting is [1, 4]. This range appears plausible for complex products in the automotive, printing, semiconductor, and power plant industries (see Table 1 in Rivkin and Siggelkow (2007)). Reviewing the empirical studies on the characteristics of complex product, Rivkin and Siggelkow (2007) found that the average interactions among the elements of a subsystem may vary in [1.4, 6.8]. Finally, the number of interactions among the elements of a team and that of another team (if they have interactions), C, can vary in [1, n e 1] which in our setting is [1, 4]. A higher value of C represents a higher degree of complexity in the project. The ruggedness of the search landscape depends on the intensity of interdependencies among the elements, i.e., K and C (Levinthal 1997). Low values of K and C imply that the contribution of one element is rather limited and independent of the other elements. Therefore the landscape is smooth as a change in the state of one element leaves the fitness of others largely unaffected. At the extreme, the NK(C) landscape with independent elements (K = C = 0) has a single peak and the fitness at any state can be improved unless the system is already at the peak. Incremental search in such landscapes may eventually converge to the global optimum. However, for high values of K and C, the fitness landscape becomes more rugged and a change in the status of one element may exert a significant impact on the fitness values of others. In such landscapes an incremental search process may stop at a local optimum. In our experiments, we assume (K, C) pairs are (1, 1), (1, 4), (4, 1) and (4, 4) to capture smooth and rugged landscapes.

15 15 We next elaborate on the dynamics of interaction between teams, i.e., the probability distribution of misalignments among teams, p ii. An instance of p ii captures the likelihood of two teams, possibly at two different hierarchy levels, having a misalignment. Therefore, we can adjust these probabilities to change the locus of misalignments in the project at a given hierarchy level. For example, i pii = 0.9 for all i in the set of high-hierarchy teams implies that the misalignments affect mainly the interdependencies of high-hierarchy teams with other teams. Figure 2 shows the interactions between the elements of teams in the network structure in Table 1. This figure also shows the misalignments that are concentrated at the high-hierarchy level. Figure 2 The bold show the network structure as in Figure 1. The light show the connections at the element level, known as the Design Structure Matrix (DSM). The red boxes show the misalignments when the misalignment locus is at high-hierarchy team 1. In the network structure in Table 1, the weight assigned to the interdependencies of the highhierarchy team (team 1) with others is P H = p 12 + p 13 + p 14 + p 15. The weight on medium hierarchy teams (teams 2 and 3) is P M = p 12 +p 13 +p 23 +p 24 +p 25 +p 34 +p 35 and that on low-hierarchy teams (team 4 and 5) is P L = p 14 + p 15 + p 24 + p 25 + p 34 + p 35 + p 45. For research question (1) (provided in Section 1), we test the effects of unmatched interactions and unmatched interfaces in a given hierarchy level at different intensity levels. In other words, observe that p ii = 1, hence a high value for P H implies that the misalignment locus is the interdependencies of the high-hierarchy team. We let this intensity to vary in [0.50, 1]. For research questions (2) and (3) (see Section 1), we study misalignments, independent of their form, when the locus of misalignments is in (1) high vs medium, (2) high vs low, and (3) medium vs low hierarchy levels. We will use P H, P M and P L as defined above to capture the intensity of misalignments in a certain hierarchy level.

16 16 For any given (N, K, C), the hierarchy level, and misalignment locus, we generate 35 random landscapes and report the average performance measures. This procedure rules out any effect that may arise from the particularity of one landscape. The remainder of the section, describes our findings in different experiments Experiment 1: Misalignment Forms and Errors Search on perceived, rather than real, landscape results in lower performance. In particular, teams are likely to reject superior designs (type-i error) or accept inferior designs (type-ii error). Our first set of experiments investigates, given the hierarchy level, which type of misalignment affects the performance more. Is it likely that the occurrence of unmatched interactions results in more frequent or larger errors of a particular type than the occurrence of unmatched interfaces? How do the effects change across hierarchy levels? We study two performance measures: (1) the magnitude of errors, (2) the frequency of errors. While the frequency captures how prone the teams are to commit errors under each type of misalignment, the magnitude captures the size of the error whenever one happens. The choice between the two measures depends on managerial priorities. If the intention is to reduce the overall product development time (and costs), managers should focus on error frequency. However, if the quality of the subsystems is the priority, size of the errors, whenever they happen, is more important. Figure 3 shows the magnitude of type-i and type-ii errors when the intensity of the misalignments and the hierarchy level at which they occur vary. It also shows how the interdependencies among the elements of a team K and those among subsystems C affect the magnitude of the errors. For each scenario, we generate 35 landscapes, construct the corresponding perceived landscapes with unmatched interactions and unmatched interfaces, simulate the search process for 200 time units, compute type-i and type-ii errors each time, and finally sum the errors of all teams over all landscapes and simulation time and average them. The upper graphs in Figure 3 report the results for project structures with a low interaction level among the elements of a subsystem (K = 1) and the lower graphs with a high interaction among the elements (K = 4). Figure 3 also reports the bounds of a confidence interval of a two-sample independent t-test at α = 0.1 significance level to statistically compare the magnitude of errors due to unmatched interactions and unmatched interfaces. 3 The null hypothesis of the t-test is that the mean of the magnitudes of type-i, respectively type-ii, errors due to unmatched interfaces and unmatched interactions (see Section 3.4) are equal. Because we generate 35 landscapes and for each landscape construct the corresponding perceived landscape with unmatched interactions and interfaces, we can assume that the two samples are 3 We obtain similar results at α = 0.01 significance level.

17 17 from a normally-distributed random variable. To apply the t-tests, we further need to examine the equality of variances. To this end, we construct Bartlett tests for homogeneity of variances (Zar 1996) at α = 0.1 significance level. If the equality of variances is established, we use t-test with unknown but equal variances.4 Otherwise, we apply the test with unknown and unequal variances that uses Satterthwaite s approximation for the effective degrees of freedom. Figure 3 (a) Type I error, K = 1 (b) Type II error, K = 1 (c) Type I error, K = 4 (d) Type II error, K = 4 The magnitude of errors. In this figure, boundary of a 90% confidence interval; : lower boundary of a 90% confidence interval; : mean errors of unmatched interfaces; : upper : mean errors of unmatched interactions. Panels (a) and (c) in Figure 3 show that unmatched interactions cause a higher type-i error. This result is robust at different levels of C, K and misalignment locus. All confidence intervals are nega4 For a given misalignment locus, only 26 of 160 Bartlett tests reject the homogeneity of variances at different hierarchy levels. For a given hierarchy level, only 11 of 48 Bartlett tests reject homogeneity of variances at different misalignment loci (α = 0.1 significance level).

18 18 tive implying that the null hypothesis, i.e., the means of type-i error due to unmatched interactions and unmatched interfaces are equal, is not supported. As a result, if a product design manager intends to reduce the magnitude of type-i errors, she should focus on reducing the occurrences of unmatched interactions rather than unmatched interfaces. Figure 3 also shows that at the α = 0.1 significance level unmatched interactions and unmatched interfaces tend to result in similar type-ii error magnitudes in most of the scenarios. In other words, our model cannot statistically predict which type of misalignment may cause a larger type-ii error. The only exception is when K = 1, C = 4, and the misalignments are concentrated in the low or medium hierarchies. With these designs, unmatched interfaces result in a statistically significant larger type-ii error. One needs to notice that these system designs are problematic as the interaction among the elements of a subsystem is low while the elements of different subsystems are highly correlated. This might point to a flawed product and organizational design as a result of poor architectural knowledge of the design space (Ethiraj and Levinthal 2004b). If the product design has inevitably the characteristics described above, then our results imply that removing unmatched interfaces is most effective in reducing type-ii error magnitudes. Otherwise, to reduce the chances of inferior designs being approved by teams, resources should be allocated to eliminating both types of misalignments. We next investigate how misalignments affect the frequency of type-i and type-ii errors. Figure 4 shows the average number of errors over different values of K, C, hierarchy and misalignment locus. Panels (a) and (c) show that the average number of type-i errors due to unmatched interactions is greater than that due to unmatched interfaces. The difference is statistically significant at α = 0.1 level. Together with Figure 3, this implies that unmatched interactions not only cause larger but also more frequent type-i errors. We observe a reverse with respect to type-ii errors: unmatched interfaces induce more frequent type-ii errors in comparison to the unmatched interactions. We find one exception to this result when (K = 1, C = 1) and the misalignments are concentrated in high-hierarchy teams. In this particular case, we observe that unmatched interactions cause a higher number of type-ii errors in the search process than unmatched interfaces. These results are all significant at the α = 0.1 significance level. Though our model does not distinguish between unmatched interactions and unmatched interfaces in the magnitude of type-ii errors, it does show that type-ii errors occur more frequently due to unmatched interfaces than unmatched interactions. Therefore, if the goal of the manager is to avoid accepting too many inferior designs, it is more effective to focus on curbing the unmatched interfaces, i.e., identifying the connections among the subsystems or using teams with dense organizational interactions and communications.

19 19 We finally investigate whether the hierarchy and misalignment locus have a significant effect on type-i or type-ii errors. To this end, we apply one-way ANOVA tests. These tests require that samples are normal, independent and have equal variances. We established the normality and homogeneity of variances of errors earlier in this section. The difference between the type-i or type-ii errors due to unmatched interfaces and unmatched interactions at various hierarchy levels and various misalignment loci are independent samples because the errors are caused by search on independently generated landscapes. The first set of ANOVA tests compare errors when misalignments occur at different hierarchy levels with all other parameters (K, C, and misalignment locus) constant; see Table 2 in Appendix. We compute the difference between type-i errors due to unmatched interfaces and unmatched interactions at three hierarchy levels and then compare the means by an ANOVA test (we repeat the same procedure for type-ii errors). The null hypothesis is that these means are equal, that is everything else constant, changing the hierarchy level at which a certain type of misalignment happens does not cause a higher magnitude of type-i (or type-ii) error. Table 2 (in Appendix) shows that, at α = 0.1 significance level, the results depend on the interaction intensity within or across the subsystems (K, C). The results also depend on the type of error under consideration. For product designs with low interaction intensity within and across subsystems (K = 1 and C = 1), type-i error does not seem to depend on the hierarchy level. However, type-i error appears to be dependent on the hierarchy level on which misalignment instances occur if there is high interaction intensity within or across subsystems (C = 4 or K = 4). Overall, it seems hierarchy plays a more important role in the magnitude and frequency of type-i errors the more complex the project is. For type-ii error, the comparison depends on the interaction level within the subsystems (K). Type-II errors do not differ over different hierarchy levels if elements of a subsystem have high interaction levels (K = 4). However, when K = 1, the means are significantly different across hierarchy levels. Thus, we find that accepting inferior designs is associated mainly with interactions within subsystems. We next test the effect of a change in misalignment locus on type-i and type-ii errors keeping all other parameters (K, C, and hierarchy level) constant. The results of the ANOVA tests are reported in Table 3. Thus, the null hypothesis for these tests is that the mean type-i, respectively type-ii, errors due to unmatched interactions and unmatched interfaces are equal. Table 3 shows that the null hypothesis cannot be rejected at α = 0.1. We find no significant effect of misalignment locus on errors due to unmatched interfaces and unmatched interactions. Therefore, the errors are not sensitive to the misalignment locus. In other words, at a given hierarchy level, higher misalignment intensity (in the interactions of teams in that hierarchy among themselves and

20 20 with other teams) does not result in higher magnitude of errors. We conjecture that this result is due to our definition of a misalignment locus that mainly captures the intensity of misalignments and interdependence of design spaces of teams. Note that as the intensity increases, the occurrences of misalignments not only increase in between the teams in the same hierarchy level but also between those teams with teams in other levels. Therefore, misalignment instances are dispersed over the search landscape and it appears that their presence rather than their locus drives the results. Figure 4 (a) Type I error, K = 1 (b) Type II error, K = 1 (c) Type I error, K = 4 (d) Type II error, K = 4 The frequency of errors. In this figure, of a 90% confidence interval; 4.2. : lower boundary of a 90% confidence interval; : mean errors of unmatched interfaces; : upper boundary : mean errors of unmatched interactions. Experiment 2: Misalignments and the Effects of Hierarchy In this section, we study the effects of misalignments on PD teams performance depending on the hierarchy level at which misalignments occur. We allocate the teams to three hierarchy levels (low,