1 4_Bukh Sida 87 RE-EXAMINING THE CAUSE-AND- EFFECT PRINCIPLE OF THE BALANCED SCORECARD 1 4. Introduction Since the mid 1980 s accounting has attempted to turn strategic. In the area of strategically oriented performance measurement Balanced Scorecard (BSC) has been one of the most debated suggestions for developing a framework for performance measurement and management (Kaplan & Norton 1992, 1993, 1996ab 2001, 2004). According to questionnaire-based research BSC seems to be widely used by Scandinavian firms. Bengtsson et al. (2000; cf. Dabhilakar & Bengtsson 2002) suggest that 32% of firms within the Swedish engineering industry in 1999 used some variant of the balanced scorecard concept with an even higher adoption rate in large firms. Toivanen (2001) found that in 2000, 23% of top 500 firms in Finland used BSC and 15 % were implementing it. Similarly, a survey among Danish manufacturing firms in 2001 indicated that 82% of the respondents had a high knowledge of BSC while 32% used the concept (Nielsen & Sorensen 2004). Even though these adoption rates appear to be high both in the Nordic countries and in many other countries (Chenhall 2004; Ittner et al. 2003; Silk 1998; Speckbacher et al. 2003), it is still somewhat unclear what in practice qualifies a performance measurement system or a business model to be classified as a BSC, and how BSC is actually used. Should we look at the type of measures used, the categorization of the measures, the management practice related to the use of the BSC, the implementation process or should we focus our attention on the existence of a cause-and-effect relationship between the measures? These questions are 1 We wish to thank the editors, Sten Jönsson and Jan Mouritsen, for their guidance in the process of writing the paper. Further, we thank Fredrik Nilsson and Nils-Göran Olve for their encouraging and helpful comments. All remaining errors are of course the responsibility of the authors. The authors are listed in alphabetic order.
2 4_Bukh Sida Re-examining the cause-and-effect principle of the Balanced Scorecard important with respect to research in this area because surveys of BSC applications and experiences from case based studies must be interpreted in light of what is being placed under the microscope. Furthermore, attempts to advance the understanding of aspects such as the logic of the methodology (e.g. Nørreklit 2000), the implications for performance evaluation (e.g. Lipe & Salterio 2000), the effect on financial performance (Davis & Albright 2004; Ittner et al. 2003), the similarities with other performance management concepts (e.g. Bourguinon et al. 2004; Mouritsen et al. 2004; Epstein & Manzoni 1997, 1998) or integration in various strategy processes, as for instance the alignment of continuous improvement initiatives in production teams to the overall strategic planning of the firm (Dabhilkar & Bengtsson 2004), will necessarily need to take specific features of the BSC methodology as a starting point. So the question will be what constitutes the core of the BSC. In most presentations of the balanced scorecard the so-called cause-andeffect principle has been a core feature of the BSC. The cause-and-effect principle has been developed with inspiration from the service management literature (e.g. Heskett et al. 1997) towards a generic business model in the form of the strategy map (Kaplan & Norton 2001, 2004). Generally the cause-andeffect relationships are argued to be the feature that distinguishes a balanced scorecard from other kinds of scorecard, whether they are labeled stakeholder scorecards or KPI scorecards (Kaplan & Norton 2001, p ). Currently, strategy maps and cause-and-effect relationships attract considerable interest among practitioners in Scandinavia, and some firms are developing their BSC based on strategy mapping (e.g. Kasurinen, 2002, Bukh et. al 2004). Some authors (Nørreklit, 2000, 2004; Brignall 2002) have, however, questioned the logic of the cause-and-effect principle. They have questioned the practicality of BSC as a method on the premise that no cause and effect relationship exists between measurement areas. Our aim is to contribute to that debate. We discuss some of the alternative interpretations of the cause-andeffect principle and we position our paper against some of the views presented by Norreklit and her colleagues. More precisely, we argue that their view on cause-and-effect is mistaken, and given an alternative interpretation, BSC can be a practical approach. We outline alternative ways to apply cause-and-effect in practice, both analytically and organizationally. This is based on practical concerns, as we see accounting as fundamentally an applied research area that should ultimately provide new insights for practice (Ittner et al, 2002, p.788, see also Mitchell, 2002). But we also question the impression Kaplan & Norton create, which implies that cause-and-effect would be a proper starting point for strategy scorecards in all circumstances. In other words, we attempt to theorize on conditions where cause and effect may, or may not, be a good starting point for the design and use of BSC. We focus on the cause-and-effect principle as that is what is often questioned. We acknowledge that cause-and-effect, as well as other design principles of scorecards, may only partially explain the poten-
3 4_Bukh Sida tial benefits from BSC. It seems that research and practitioners should pay much more attention to how people in organizations actually use various types of scorecards to commit action. Given our focus on the cause-and-effect principle, however, we deal less with action in this paper, addressing it mainly within our discussion on practice of cause-and-effect and contingencies. In the paper we will draw not only on the research based evidence of applications of the balanced scorecard, experiences from Nordic firm s as has been expressed in various writings by employees from the firms, but also on our own experiences from actively participating in developing strategy maps and implementing the balanced scorecard in a number of firms and public sector organizations within the last ten years. Thus, all examples in this paper are based on our own experiences from various Nordic firms that are using BSC. We have chosen not to present these experiences as empirical evidence as there have been no (or only limited) formal research based data collection involved in these consulting assignments. Furthermore, we draw on our experiences reported in the management oriented literature (e.g. Bukh et al. 2000; 2004, Malmi et al., 2002.) The remainder of the paper is structured as follows: The next section provides a thorough treatment of the cause-and-effect principle. Section three discusses what it implies to implement strategy using the balanced scorecard, what it implies to use the strategy mapping methodology and how this can be used in cascading scorecards to lower organizational levels. In section four we address possible contingencies of cause-and-effect and finally, we summarize and discuss the research implications of our analysis in section five. The logic of cause-and-effect principle Kaplan and Norton s (1996b, p. 149) definition of strategy [a] strategy is a set of hypotheses about cause and effect calls for the existence of a causeand-effect relationship in a properly (ibid, p. 31 and p. 149) constructed BSC. They argue that a BSC should contain outcome measures and that the performance drivers should be linked together in cause-and-effect relationships (ibid, p. 31). Although much acclaimed, there seems to be some confusion, both in practice and among academics, with respect to how this cause-andeffect principle should be interpreted and implemented. In interviews with Finnish firms in 1998 about their use of BSC, Malmi (2001) found that although most interviewees stated that they have derived their measures from strategy, based on cause-and-effect reasoning, the claimed link between strategy and measures appeared weak in most companies. Also, it appeared from the interviews that the initial idea of linking measures was often not well understood. Similarly, many applications of the balanced scorecard concepts as we have seen them and as they have also been documented in
4 4_Bukh Sida Re-examining the cause-and-effect principle of the Balanced Scorecard the practitioner oriented literature reporting on experiences from Nordic firms (e.g. Bukh et al, 2000, 2004; Hoff & Holving 2003; Laitinen 1996; Malmi 2001; Malmi et. al, 2002; Olve et al. 1997, Puolamäki 2004) adopt only a rather simplified interpretation of the concept of BSC, which is also the point conveyed by Ittner & Larcker (2003; Ittner et al. 2003). Often scorecards are only composed of a collection of indicators sorted in four dimensions without any attempts to map the relationships between the indicators, and thus they resemble more the kind of scorecards that Kaplan & Norton (2001, p. 103) term KPI scorecards. Furthermore, the scorecards are rather dense when it comes to the actual indicators of strategically important issues. This leads to questions about focus, as more measures used in the BSC imply that less weight is attached to any single indicator, all other things being equal2,3. We have, however, also seen and worked with firms both in the private and the public sector, which have used cause-and effect logic and strategy maps to illustrate and communicate their strategy, and to come up with measures. (See also Malmi et. al., 2002; Kasurinen 2002; Bukh et. al.; 2004). The cause-and-effect has also been a concern in an academic debate. A foremost contributor to the debate has been Nørreklit who has questioned the theoretical foundation of the cause-effect-relationship (Nørreklit 2000, 2003). Moreover, she and her colleagues have questioned the control model BSC assumes (Nørreklit 2000, 2003), the innovativeness of the approach (Bourguignon et al. 2004) and the style of argumentation (Nørreklit 2003). Although, in this paper, we concentrate on the cause-and-effect relationship the other possible problems discussed by Nørreklit are not independent. It is, for instance, important to gain insight into the functioning of the cause-and-effect of the BSC in order to be able to assess the question posed by Nørreklit whether the form of rhetoric used may be decisive in winning an audience over to a management theory such as the balanced scorecard while the content of the theory may be less important? (ibid. p. 592). As she claims to have demonstrated that the cause-and-effect principle is flawed, she concludes that the great interest towards BSC is largely due to promotional rhetoric rather than BSC's substance as an innovative and practical theory (Nørreklit 2003). Nørreklit does a good job in showing that the rhetoric used by Kaplan & 2 In practice this seems to lead to a situation where new measures, if there are too many of them, do not get much attention, as people tend to focus on those that have been in use for a while and they are familiar with or on the more general measures, including the traditional financial, that can more easily be compared between firms or departments instead of those reflecting the specific strategies (see also Lipe & Salterio 2000) unless action is taken to involve managers more actively in the performance assessment (cf. Libby et al. 2004) 3 Authors like Banker & Datar (1989) who focus on the theoretical properties of reward systems have demonstrated that more measures are preferred to fewer if the additional information is costless and provides information related to the agent s actions that the principal wishes to motivate. It should be noted that we are in this paper not discussing the optimal design of incentives and that we interpret lack-of-focus in an organization as a cost.
5 4_Bukh Sida Norton (1996) is in many places more persuasive than convincing, and that it is ambiguous, leaving us number of alternative ways to interpret the presented ideas. Hence, it is easy to agree that the success of fashions, and organizational adoptions of BSC, in particular, are based at least partly on successful promotional rhetoric (see also Malmi, 1999). But what she does not do well is her rejection of the alternative explanation she suggests, namely BSC as an innovative and practical theory. She rejects this explanation by referring to the lack of cause-and-effect relationships between some of the suggested areas of measurement, and the control model BSC assumes. When it comes to cause-andeffect logic, we think she misses the point. Norreklit (2000) question the reliability of BSC by arguing (1) that there should be a time dimension in the BSC in order to be able to talk about causality, (2) that there is no cause-and-effect relationship between some of the suggested areas of measurement in the BSC and (3) that the four dimensions are not independent. We will here re-examine these three claims that together form the basis for the rejection of BSC as a practical and useful methodology. As the third claim does not relate directly to the cause-andeffect principle we will not discuss it as much detail as the other two in this paper. The aim of our analysis is not to argue that BSC is an innovative theory. Rather we wish to argue, however, that it is a practical one. Whether its practical appeal, promotional rhetoric or something else better explains its success is not the focus of this paper The lack of a time dimension The term causality is in the management (accounting) literature often used without a clear definition. Kaplan & Norton (1996) do, as Nørreklit (2000, p. 70) point out, not define, the concept. The causality concept they adhere to can probably best be inferred from their examples. They clearly attach another meaning to causality than the philosophically oriented literature (Edwards 1972; Føllesdal et al. 1997) referred to by Nørreklit (2000) where causality has a rather restricted definition. Kaplan & Norton s examples seem to suggest a finality relationship, as also discussed by Norreklit (2000, pp.76-77). However, let us briefly consider the time dimension in causal relationships and provide examples of how time dimension is built into the scorecards in practice. Looking to econometrics and the understanding of causal relationships in systems of economic time series variables, causality is often defined according to forecasting principles exploiting the idea that a cause must precede its effect in time. Testing of so-called Granger causality (Granger 1969; 1980) has become a standard step when analyzing linear systems of time series. Causality here can both be instantaneous, where X and Y refer to the same time and the current value of X is better predicted if the present value of Y is included in the prediction than if not (Granger 1969, p. 429), and lagged as is the case that
6 4_Bukh Sida Re-examining the cause-and-effect principle of the Balanced Scorecard Nørreklit assumes. In practical applications of cause-and-effect we see often relationships that are instantaneous causal, e.g. when the percentage growth in revenue and revenue from a certain customer segment is measured, and relationships that have a causality lag, e.g. between new products introductions and revenue growth. Although we agree with Norreklit that Kaplan & Norton do not consider time dimension explicitly, it is actually implicitly part of the methodology. Often firms have the time dimension built into their strategy maps and scorecards in simple way. Strategy maps are, for instance, created for the period of strategy, say 3 to 5 years. Managers we have worked with understand that they need to create competences and improve their processes before they can expect e.g. customer satisfaction to increase or financial returns to follow. These assumptions are reflected in the target values these companies set for their measures. For example, firms have as part of their scorecards specified the hypothesis with respect to e.g. how many customers or what growth in order volume in one year are necessary to achieve a specific revenue growth next year. The cause-and-effect relationship between measures Nørreklit (2000, p. 72) uses customer satisfaction, customer loyalty and financial performance as an example, suggesting that it is not generic that increased customer loyalty (and satisfaction) is the cause of long-term financial performance. In her analysis she states that: We may therefore ask whether it is the case, as Kaplan and Norton state, that if a company delivers much value and quality to its customers, then the customers will be loyal and profits will necessarily, or highly probably, roll in? (Nørreklit 2000, p 72). Since she further claims that Kaplan and Norton base this on Jones & Sasser (1995), she analyses the logic of Jones & Sasser s arguments which leads her to conclude that the BSC makes invalid assumptions about causal relationships, leading to the anticipation of performance indicators which are faulty, thus resulting in dysfunctional organizational behavior and sub-optimized performance. Two points in this analysis require attention. First, a quest for generic relationships: In other words, should these relationships be assumed to hold universally, or in a certain organization at a certain point in time and in a certain setting? Second, a desire for existing relationships as opposed to assumed or expected relationships. Moreover, a closer inspection of what Kaplan & Norton actually wrote is necessary.4 Our reading of Kaplan and Norton suggests, that they use this and other similar examples to convey the message that certain desired outcomes that are
7 4_Bukh Sida of interest, e.g. long-term financial performance, are partly caused by organizational actions, e.g. efforts to build customer loyalty that can be measured (see also Kasurinen, 2002).The basic argument is not that the relationship holds in any organization, given any contingencies. Thus, it is not suggested by Kaplan & Norton that there exists a generic cause-and-effect relationship between two measurement areas, or two measures, but rather that it would be possible to identify such measures in a particular organization, given all its contingencies, that a) reflect an organization s strategy (customer loyalty should be part of the strategy in this case) and b) the improvement of which will with high probability lead to a desired outcome in some other (outcome) measure. Actually, Kaplan & Norton state explicitly that [n]ot all customer demands can be satisfied in ways that are profitable to an organization (1996 p. 71) and suggest customer profitability analysis be applied. Following this example, in any particular situation there might be an endless number of actions and associated measures that may have an impact on long-term financial performance. The trick is to choose between those potential actions, which is what strategy (i.e. means to desired ends) is supposed to be all about, and choosing those actions organizations can influence. Kaplan & Norton argue fundamentally that measures should be derived following an assumed cause-and-effect relationship. An assumed relationship is different from an established relationship. In other words, does the BSC as a method assume some established relationships? Are BSCs to be built based on assumed relationships or established relationships? Kaplan & Norton argue that strategy is a set of hypotheses about cause-and effect. Some of these hypotheses may prove right, some wrong. Management seldom, if ever, operates in an environment where the most important relationships are known, or can be established before setting up a strategy and management control system. Managerial work would be quite simple if we were able to establish with certainty that increasing this particular input, or improving it in this respect, will certainly lead to that output. What follows then is that if these relationships are not known for sure, strategy is guesswork. It becomes a hypothesis (to use Kaplan & Norton s rhetoric) of what management believes is the best thing to do. Organizations need to make their best estimate about the actions (strategy) that lead to desired outcomes. If we follow Kaplan & Norton s logic, measures can be derived from these hypotheses and are there to convey these 4 As a third point in relation to Sasser & Jones (1995) it could be noted that these authors main arguments are related to the empirical fact that very satisfied customers are much more loyal than satisfied customers as reflected by the difference between scoring 4 and 5 on a 5-point Likert scale in a satisfaction survey. The practical implication in a BSC is that an indicator like percentage of customers scoring 5 is much more likely to be correlated with loyalty than average customer satisfaction. In most BSCs that we have seen firms use average customer satisfaction so when evaluating practice one has to take into account aspects that relate to the implementation which cannot be separated from the principles according to the literature.
8 4_Bukh Sida Re-examining the cause-and-effect principle of the Balanced Scorecard guesses to the employees and other constituents. And these relationships now need to be probable only for this particular company in a time of strategy. An argument that the BSC should be based on both generic and existing cause-and-effect relationships is even more disturbing. In fact, this would make the whole idea of BSC as describing organizations strategy and as a method of translating strategy into action useless. Where do we need strategy, if we are to discover universal truths about the causes of superior performance, and design our measurement system accordingly? Would strategy now be the same as following these generic regularities? If not, how then would strategy be reflected in the BSC? How long would firms following known regularities sustain competitive advantage? Can there be competitive advantage in an environment where everyone knows that X leads to Y? Also, other authors have argued that the logic behind the balanced scorecard is questionable. Meyer (2002, p. 3) has, for instance, stated that if performance measures were strongly correlated, then all would contain essentially the same information and there would be no need for a. balanced scorecard. This argument would be valid, if we assume there is a strong correlation between measures and all members of the organization understand these correlations and how their actions are related to the chosen measure. Further, there should be no delay from changes in the causes to the effects, which is generally not likely. In practice Meyer s argument could be valid if we see the BSC only as an information system, not as a management system. If e.g. we need to convey a message to the process owner or department head about the strategy of the organization, we need to translate that strategy to something they can relate to. They are likely to understand, and may like to follow process measures such as process cycle time instead of Return-on-Capital- Employed (ROCE), even if these two were highly correlated in that particular organization. Hence, from an information perspective one measure might be enough, but from a managerial point of view it is likely to make a difference which measures you choose to ask people to believe in and commit to. Moreover, it is more likely that most of the measures in any strategy map are not highly correlated. Let us use Nørreklit s example as previously discussed to illustrate this. If we assume that financial performance is measured by ROCE then, based on the same logic as in the usual DuPont analysis of financial key ratios, it can be argued that customer loyalty is likely to have more impact on margin than on asset utilization. There are also likely to be tens or hundreds of other factors having an impact on ROCE. Hence, it should not be expected that these measures correlate highly. In other words, it is not as much a question of a specific correlation between measures as it is about deciding which one of the factors with an impact on ROCE management believes can be influenced, and given this, which would have the strongest impact on ROCE. Therefore, it can be expected that some correlation exists, but perhaps not to an extent that they develop closely in parallel.
9 4_Bukh Sida In conclusion, there is no need to assume that there should be generic cause-and-effect relationships. It will be a matter of managerial insight in the concrete organization to state the hypothesis regarding the actual relationship and, ideally, to validate it, if or when data is available. Kasurinen (2002) is one of the first to provide research-based evidence on how a Finnish retail organization developed their BSC practices successfully by creating a strategy map and using BSC measures based on assumed cause-and-effect relationships between measures. The four perspectives and their interdependence Although the basic causality in the BSC is most often discussed at the level of specific success criteria or performance measures, Kaplan and Norton (2001, p. 77) argue that the four generic balanced scorecard perspectives are fundamentally also related in a causality chain. Both Nørreklit (2000, p. 75) and Brignall (2002) have pointed out that the linear causal relationships between the four perspectives should more correctly be seen as interdependent perspectives since. R&D activities., for example, in the Learning & Growth perspective will often be dependent on results such as revenue and profit which are measured in the Financial perspective. In principle, we agree with Nørreklit and Brignall. But we also argue that for the purpose of identifying measures we should avoid thinking of linkages between perspectives, and should rather restrict ourselves to understanding causality, or finality, between individual actions and measures. The reason for this is practical. Assuming that the purpose is to identify and measure those actions that are required to improve desired outcomes (e.g. profitability), it does not help much to argue that these actions are influenced or constrained by financial realities. Of course they are. The idea is to identify those actions that have biggest assumed impact on financials, are realistic in financial terms, and are actionable by the organization. Hence, even though the measures in different perspectives are likely to be interrelated, these interrelationships will be taken into account in practical applications as actions and associated measures are identified. Moreover, we believe it is a misinterpretation of the meaning of causality to conclude that Kaplan & Norton s (1996) reasoning is circular (Nørreklit 2000, p. 76) as this phenomenon, termed feedback in econometrics is very common, or as Granger (1969, p. 428) states: [i]n many realistic situation however one suspects that feedback is occurring. Hence, the existence of feedback should not cause important indicators or critical variables to be excluded from the scorecard just as the existence of feedback should not be an argument against using the balanced scorecard. As discussed above, the idea of cause and effect in the BSC should be interpreted as a tool to identify most influential and realistic actions to be undertaken, not as a true representation of all prevailing complex interrelationships in any business situation. In cases
10 4_Bukh Sida Re-examining the cause-and-effect principle of the Balanced Scorecard where firms have time series data related to their balanced scorecard with which to validate the strategic hypothesis build into the strategy map, proper statistical procedures that enable the estimation of the feedback effects could, of course, be applied. The practice of cause-and-effect As can be witnessed in practice there are many ways to use elements and principles from the balanced scorecard literature. Most firms do not interpret the cause-and-effect relationships build into the strategy map as deterministic relations or relationships that have been verified by time series of data where one indicator statistically has been demonstrated to be causally related to the other. Rather, the relationships are strategic hypotheses, i.e. presumptions about how performance is influenced by other factors, which is also the way that Kaplan & Norton (2001, p ) suggest that they should work. Thus, it is only through the implementation and continuous use of the balanced scorecard that it can be determined if these relationships can be found and if not, the balanced scorecard should be revised to reflect this. As such, the balanced scorecard is not different from any other approach to developing strategy. Concept of strategy It is easy to argue that performance measures should reflect strategy, or that strategy should provide the starting point for a measurement system. It seems much more difficult for practitioners to establish this link. One problem we have observed relates to the concept and understanding of strategy itself. In any organization especially large ones strategy is likely to be given a variety of meanings. Sometimes these meanings are dependent on the educational background of managers, previous experience and leadership style and sometimes they reflect different functional orientations or simple the positioning in the organizational hierarchy. For some managers, strategy is, for instance, about acquiring or divesting businesses. Similarly, in some organizations strategy seems to be more about strategic decisions, e.g. where to set up a new factory, what firms we should engage in partnerships with, etc., than about issues related to major on-going operations, i.e. what makes our current operations more and more attractive to our customers. What is of interest is in relation to our paper, is on the one hand what perceptions of strategy lead to strategies that could be supported by a balanced scorecard and on the other hand what attributes BSC indicators should have in relation to these strategies. From a measurement system perspective, it may be argued that indicators that do not have a range of possible outcomes should seldom be included in a
11 4_Bukh Sida BSC. In practice, many firms include so-called soft success criteria like implementing a new IT systems, opening a sales office in the capital or hiring two new managers. While the decisions and plans that these soft criteria reflect can be critical for achieving the strategic objectives of the firm, the associated measures are not useful when strategy is to be communicated using the scorecard or if the BSC is to be used as a strategic monitoring system because the possible monitoring or follow-up upon such binary measures (have been done/have not been done) is not improved by making them part of a BSC. Some major projects may be regarded as exceptions here, where measurement against project plan serves to a highlight project s strategic importance among other BSC measures. But in these cases the measures have a range of outcomes, e.g. hours of work consumed by the project instead of just implement a new system or number of customers instead of just marked a new product. It may further be argued that we should include and communicate that part of the strategy the organization in large can have an impact on. This appears crucial if we aim to commit people to the strategy and to make the strategy actionable. It is not worth, and could even be counterproductive, to include those strategic items that are only at the discretion of top management or even specific managers. To illustrate the point, consider one medium sized Finnish organization where the management team at the corporate level had decided to use the Balanced Scorecard. They told us that they followed a growth strategy and as they perceived that the desired growth was not to be attained organically, they aimed to grow by acquisitions. Consequently, they decided to measure their acquisitions in their BSC. There were, however, two problems with this measurement: First, how wise would it be to spread information widely about potential acquisitions. Second, how should you consider the signal of such measurement for divisions? It might suggest that it does not make much difference how you run your daily operations, the achievement of strategic objectives is determined by the success of acquisition activities administered by the corporate HQ. Some of the firms we have worked with have decomposed their strategies to functional responsibilities, i.e. marketing strategy, supply-chain strategy, finance strategy, IT strategy, environmental strategy, etc. These functional strategies may aid in developing scorecards for each function, but for an organization or SBU as a whole competitive strategy (Porters 1985, 1998; Treacy & Wiersema 1993, 1995) should provide the starting point. Focusing on how to compete reduces the complexity of strategic measurement systems remarkably. In sum, accepting that the BSC may not need to reflect all facets of strategy may answer some of the criticisms towards the BSC by e.g. Vaivio (1995), who questions the idea that a handful of quantitative measures can portray the various facets of a company s strategy. Even if we accept the idea that competitive strategy should provide a start-
12 4_Bukh Sida Re-examining the cause-and-effect principle of the Balanced Scorecard ing point for the BSC, it is hardly enough, however, to state that the firm follows e.g. a customer intimacy strategy, a differentiation strategy or another of Porter s generic strategies; and it is not sufficient just to state e.g. that the strategy is to create value for the core customers, optimize resource utilization and focus on development of employee competencies. These examples are all from scorecards that we have seen in actual use. Many firms that have successfully implemented the BSC have, in the process of developing a strategy map, gradually detailed the understanding of such broad statements. This is necessary if the strategy is to be translated to cause-and-effect relationships and scorecards. What often happens in practice is that these ideas of how firms actually attempt to make a difference to their customers, whether it is formulated as a value proposition as Kaplan & Norton (2001) suggest or not, become more concrete by the plans and ideas of how firms actually are going to make this happen and what sort of capabilities and skills this would require the firms to develop (see e.g. Kasurinen, 2002). These aspects of the strategy are made concrete in the Internal perspective and the Learning & Growth perspective, respectively. Our experience is in line with the latest arguments by Kaplan & Norton (2001, 2004) who claim that strategy mapping aids in developing organizations strategy, instead of just translating it into action. It seems that strategy becomes more detailed in practice, extending to cover process and learning and growth issues. Our experiences from many BSC projects also indicate that strategy implementation almost always requires strategy development because few strategies are well developed enough for immediate implementation. But perhaps even more importantly, it appears that such a refinement and detailing of strategy seems to help to make strategy actionable. It may be that people understand process and learning related issues better than broad strategy statements, and therefore commit to these plans, and take actions to make these plans to happen. Or it may well be that the refinement of these strategies is mostly to be done with those organizational members who have the first hand knowledge of processes. Participation and influence in strategy work is likely to increase commitment and hence action. It may also be that mapping truly helps people to understand how their actions are related to more broad objectives of the firm, and therefore mapping results in action. To summarize, the way strategy is perceived will have an impact on how cause-and-effects are modeled in practice. Leaving some aspects of the strategy out of the strategy map and the BSC will simplify mapping. Being more precise about processes and capabilities required to drive customer and financial targets than seems to be the case in most strategies would also make mapping easier. Precision and detail in strategy may also help to build commitment and ensure action follows planning.
13 4_Bukh Sida Cascading scorecards Kaplan & Norton argue often that the BSC should make strategy everyone s job (e.g. 2003, pp. 211). Although this is one of the highly rhetorical statements in their writings the practical implication cascading of scorecards is important. Cascading can be done in various ways, both conceptually, i.e. how measures are linked together, and organizationally, i.e. who defines these linkages and measures. It is likely that both conceptual and organizational choices have an impact on BSC project success and its outcomes. Therefore, we outline some of the alternatives that we have encountered. The easiest approach to cascading the scorecard is to let higher level, here termed Strategic Business Units (SBU), measures, say in customer perspective, directly form lower level, here termed Business Units (BU), customer measures. The specification of the measurement would, from this perspective, ideally be similar in both layers (see e.g. Malmi et al. 2002). For this to work the lower level BU should be very similar and the BU should operate under the same contingencies for the same indicators to be relevant. This way of cascading has been used in large Nordic banks where a BSC at SBU level is cascaded to all branches, which are typically required to measure customer satisfaction, employee satisfaction, growth in revenue from specific products, etc. In case it is not possible to come up with similar measures for more organizational layers, then organizations often attempt to measure drivers of the SBU measures at BU level (see e.g Toivanen 2001). The logic behind this approach is to determine connections between perspectives in a way that the BU scorecard can be said to include indicators that are leading indicators for the performance of the SBU scorecard. Some firms establish only the connection between organizational layers as a relationship such that the measures in e.g. the BU s customer perspective together are thought to be influence the group of indicators in the customer perspective at the SBU level. One of the problems encountered with this approach in practice is that it easily turns out to be an exercise in filling out perspectives with indicators that might be meaningful at the BU level, but where no strong hypothesis is developed with respect to how they relate to the indicators at the SBU level. In many cases a third approach for cascading measures based on the causeand-effect logic is more appropriate. Basically, the Strategy Map describing assumed causal links between measures at the corporate or SBU level is constructed. Lower level scorecards then inherit indicators or measures from upper level scorecards. Each lower level entity identifies which measures in that map it can have an impact on. These will now become outcome measures for those subunits. Then they need to think how they are going to achieve those outcomes. In other words, they formulate their own strategy and develop a strategy map, and based on that map the units define their leading indicators. This approach tends to produce far fewer measures than the
14 4_Bukh Sida Re-examining the cause-and-effect principle of the Balanced Scorecard approach described above. Furthermore, the same logic can be applied to support function measurements, even if their activities are not directly linked to any of the higher level strategy map items, or measures in the higher level BSC, however defined. Each support function needs to think how it could improve, even if that is not of strategic importance for the firm as a whole. Using this type of cause-and-effect logic seems to make the definition of support function measures more comprehensible compared to providing them with the BSC template with its predetermined perspectives. We have in this section more or less implicitly assumed that the high level scorecard to be cascaded in the organization is developed at the SBU level. When a scorecard is developed at a corporate level with limited synergies between business units, Kaplan & Norton (2001, p. 103) suggest that a stakeholder scorecard can be more useful as a starting point because each business unit has a different set of internal drivers. But also Kaplan & Norton (2001, p. 380) question whether it will be meaningful at all to develop a corporate-level BSC when firms operate in multiple industry segments. Following the cascading logic outlined above, SBU s can still inherit indicators or measurement areas from the corporate scorecard. The limited synergy will imply that only a few indicators are inherited (possibly financials) and thus the SBU assumes more freedom to develop its own strategy using the normal strategy mapping procedure. An even weaker cause-and-effect relationship can be found in conglomerates where only the strategic themes and suggestions for core processes with only a few or no specific performance measures are common across business units. Here the strategic themes are taken as a point of departure at the SBU level and specific strategy maps are developed for each SBU reflecting the value proposition, performance drivers, etc. relevant at that level. An example of this cascading principle is found in the US global diversified manufacturer of industrial and commercial equipment, Ingersoll-Rand (see also Kaplan & Norton, 2004 pp ) where a corporate level scorecard was developed in 2000 and was cascaded to all SBU s including a small Danish firm in the building supplies industry. Here, the first BSC was developed by management in 2001 using the strategy map procedure and taking the three corporate wide strategic themes as a starting point. An alternative could be to leave the decision to use BSC to the business units, as Olve et al. (2003) suggests, if no particular synergy across the organization exists. The involvement of managers in developing strategy and strategy maps Common to the three approaches that we have outlined above is that the upper level scorecard is determined before the lower level scorecards. It has been an underlying assumption in our discussion that the responsibility for developing the scorecards at a specific organizational level was with the man-
15 4_Bukh Sida agement of that level. In that respect the so-called top management team plays a major role in the process as strategy formulation starts here. This is not necessarily the same as stating that the BSC builds entirely at a top-down control model, that the BSC does not facilitate emerging strategies (cf. Mintzberg 1994) and strategic learning or that the BSC is incompatible with the Scandinavian culture and management style. Nørreklit (2000) has argued that the control model as implied by the BSC approach is perhaps not suitable for Scandinavian culture and that it is not easily rooted in a dynamic environment or in the organization (ibid, p. 81). Although we agree that Kaplan & Norton give top-management a privileged role in determining the strategic direction of the firm, we have also seen a number of Scandinavian firms working with their scorecards in a collaborative fashion (see e.g. Toivanen 2001, Malmi et.al. 2002; Bukh et al. 2004). Again, the crux of the argument by Kaplan & Norton is that strategy needs to be developed for an organization (SBU) as a whole and then cascaded down to its parts. It is different from the question of who develops that strategy, or what the development process looks like. We have, for instance, seen one public sector organization (fairly small though) which involved every single employee to create an organization level strategy and BSC. In our experience, usually BU managers are either part of the management team of the SBU, or are invited to take part in the development of SBU level strategy and measures. Subunit managers then take these to their units and assess together with their team on how they can contribute to those SBU level targets and issues. Sure, this is a top-down approach, but also a collaborative approach, as subunit managers are both involved in creating SBU level strategy and have, at least to a certain extent, the freedom to come up with means to contribute to those higher level objectives. We believe that this involvement of managers from all levels of the organization in designing scorecards is one prerequisite for BSC and associated strategies to translate into action (see also Toivanen, 2001). Some contingencies of cause-and-effect We have discussed the logic of the cause-and-effect principle and some of its managerial implications above. In this respect we have departed from Kaplan & Norton s literature on the balanced scorecard where it is more or less explicitly assumed that firms should develop strategy maps and design their scorecards based on cause-and-effect. However, as we discussed above, far from all firms develop their scorecards in this way. Thus it is reasonable to ask if we should assume that all firms are better off if they were to follow cause-andeffect in determining their strategy and scorecard, or are these methods more suited to certain circumstances than others? Eventually, this question is also
16 4_Bukh Sida Re-examining the cause-and-effect principle of the Balanced Scorecard related to how the strategy mapping should be used under various contingencies. In order to structure the discussion of possible contingency factors we distinguish between four types of contingencies: strategic, organizational and political, communicational and environmental. What follows in this section is not intended to be a comprehensive contingency theory of cause-and-effect, but a presentation of propositions of factors that may make the use of causeand-effect problematic. Such factors may be examined in further studies and considered in practical applications. Strategic contingencies Meyer (2002) argues that it is very difficult in practice to specify simple and useful cause-and-effect relationships; and we partly share this experience. It is especially difficult in organizations where there are no major strategic intents (cf. Prahalad & Hamel 1989), or on pre-established understandings of what problems or bottlenecks are to be solved. Hence, we suggest that a clear strategic agenda may be seen as a prerequisite for the successful use of the causeand-effect principle. In organizations where success is believed to emerge from incremental improvement in a number of areas, strategy mapping could be less suited, as it picks some areas to be in focus thereby leaving others in the background. It is not possible to include a large number of development areas to be equally strategic important and all be reflected in the strategy map. Although it would make sense to analyze causal determinants of each incremental improvement area, this may not be regarded as strategy, but rather e.g. traditional quality improvement activity. Hence, strategy focusing on incremental improvements is perhaps not well supported by strategy maps. A related issue has to do with management s ability to select the right areas to focus on and to specify appropriate measures to be included in the BSC. A difficulty in selecting focus areas is easily understood by referring to the classic framework by Merchant (1982), where one of the prerequisites to the use of results based controls was the ability to understand what actions produce desired outcomes. Moreover, even though the strategy mapping procedures seems straightforward as it is described in the literature, it is in practice still a major challenge for a management team to develop a balanced scorecard. This also means that it is likely that not all measures suggested in the initial scorecard are essential and that some measures are likely to be miss-specified. However, this would be tested over time, as the exclusion of areas will be based on the strategic hypothesis that these are less important. Again, this is a concern for the practical use of BSC and based on our experiences, for firms with a clear drive to improve performance remarkably. Cause-and-effect mapping has enabled management to focus their attention, projects and activities in the areas that are most crucial with respect to their strategy.
17 4_Bukh Sida Furthermore, it follows from our discussion of the concept of strategy in section 3.1 above that the usefulness of the strategy mapping might depend on how strategy is perceived at a specific organizational level. It seems, for instance, that for those organizations, or organizational levels, where strategy is mainly understood as investments, cause-and-effect mapping may not provide as much value as when a strategy is seen as a matter of market positioning. This is due to fact that such investments are more one-off activities and can be easily followed by other means. Moreover, investments, acquisitions, etc. are often managed as projects and therefore at least in the initial phase of such investments, control is in the hands of project manager and team, not the organization as a whole. To sum up, the use of cause-and-effect principle with the BSC may not necessarily prove beneficial if a) organization does not have a clear strategy, b) there is no understanding or agreement on the underlying causalities, i.e. what actions will produce desired outcomes c) strategy is about improvements in general with no priorities and d) strategy is conceived as a set of decisions or soft success criteria. Organizational contingencies Another concern is the nature of the organization and the time frame of actions specified in the strategy, given that strategy covers Processes and Learning & Growth issues. To illustrate, assume that a BSC is developed using the logic of cause-and-effect between indicators, as suggested by Kaplan & Norton (1996, p. 31). Financial outcomes are here assumed to follow from customer loyalty, which in turn is assumed to be influenced by on-time deliveries. On-time delivery in turn is assumed to be improved by improvements in quality and speed of the order-to-delivery process. Quality and speed of that particular process in turn are assumed to be influenced by skills of the people working in that process. Extending this example, improving on-time delivery performance may take two months or couple of years. If the strategic challenges in relation to improving on-time delivery can be solved quickly, e.g. in few months, then an organization should probably shift its focus from on-time delivery to something else after that period. If we further assume that management believes that after having improved on-time delivery performance, the factor that will have the second largest impact on customer loyalty will be brand association. Improving brand association may require improvements in another business process than order to delivery process and other skills of other persons than those skills of order to delivery process personnel. Hence, the measurements should change also.. If strategic concerns in customer interface, internal processes or learning and growth can be solved rapidly, and hence strategic focus in these perspec-
18 4_Bukh Sida Re-examining the cause-and-effect principle of the Balanced Scorecard tives tends to change rapidly from one area to another, it might be difficult to make use of strategy-maps and cause-and-effect as a basis for a measurement system. An organization may be able to solve strategic concerns quickly for a number of reasons. We propose that this may happen either because problems are fairly trivial in nature, just requiring attention rather than complex problem solving abilities, a situation typical of growing and immature organizations. Or an organization possesses capabilities for solving problems quickly. Furthermore, as changes in strategy should be reflected in the strategy map and associated measures, keeping measurements up to date may take more effort than the benefit provided. So, it may not be that difficult to come up with cause-and-effect relationships, but updating those and associated measures may become a challenge. Epstein and Manzoni (1997) raised similar concerns earlier regarding the effort required to maintain the BSC. A major advantage of strategy map and balanced scorecard, as argued by its proponents, is that they communicate strategy. Ironically, this feature can in practice be a major obstacle to the implementation of the balanced scorecard. Not only is it questionable if the widespread distribution of sensible strategic information in the organization is advisable, as we discussed above, but it is not necessarily in the interest of management to communicate strategy clearly to the board, or in a public sector setting to the Ministry or politicians as this weakens management s position in negotiations of budgets and incentives, etc. Manzoni (2002) even states: I have seen top managers espouse clarity and transparency in public, but nurture opaqueness in practice. They do so because opaqueness has significant advantages for bosses (ibid, p, 18). We have come across similar concerns. In many large organizations, business units have substantial autonomy, as long as they are able to meet financial targets. The idea of establishing both outcome measures (lagging indicators) and leading indicators related to how outcomes are to be achieved may easily be against the prevailing autonomy, and hence resisted. Therefore, the political environment, internal or external, may determine how useful management finds strategy mapping. To sum up, the use of cause-and-effect principle with the BSC may not necessarily prove beneficial if a) strategic concerns are quickly solvable, either due to a1) immature state of an organization or a2) dynamic capabilities, and b) the internal or external political environment poses risk to the dominant coalition. A contingency of the use on external communication Another question is whether management plans to communicate the balanced scorecard externally and how this decision influences the design of the scorecard and the choice of measures. The question is here how a BSC can include strategically critical performance indicators and at the same time be used for commu-
19 4_Bukh Sida nications, possibly even to external stakeholders. The key problem can be illustrated by an example: If the strategy of one of the large Nordic banks, for instance, includes increase profitability by creating value for wealthy clients in the suburbs of the capitals, it would be likely that measures like market share, number of wealthy customers (e.g. defined by net fortune) attracted, average profitability per customer in this segment, etc. would be part of the BSC. But it is not likely that this information would be in a form that the bank would like to include as part of the external communication. It would both be strategically sensitive and a harmful signal in relation to the non-suburb customers. Furthermore, most examples of scorecards being used for external communications reveal either only a few and very general key performance indicators, as was the case of the Swedish insurance company Skandia5 mentioned by Kaplan & Norton (1996, p ; cf. Mouritsen et al. 2001), or it is only the strategic themes that are reported externally in firms such as the global manufacturer, Ingersoll-Rand (cf. Kaplan & Norton pp ). Thus a BSC with the strategic themes and critical areas shown in a strategy map can be very useful as a basis for communicating the elements of the strategy but most often the detailed elements of the scorecard should not be communicated widely. This is also how balanced scorecard is used in most of the cases that are reported in the practitioner oriented literature (e.g. Bukh et al. 2000, 2004, Hoff & Holving 2002). Our interpretation of Kaplan & Norton s books, including their examples, is basically that BSC should not be seen as a stakeholder model (as is also explicitly stated by Kaplan & Norton 2001, p ). Thus managers should not strive for balancing various stakeholders interests as this complicates the mapping of the cause-and-effect chains. If it is to be used as a stakeholder model, as is often the case in the public sector, then each stakeholder s expectations may provide a useful staring point instead of mapping. To sum up, the use of cause-and-effect principle with BSC may not necessarily prove beneficial if BSC is to be used for external communications Environmental contingencies Perceived environmental uncertainty may have an impact not only on what measures the scorecard should include but also on the usefulness of strategy mapping in itself. In general, the literature on contingency approaches to strategic performance measurements, see Ittner et al. (2003) for a review, sug- 5 From the mid-1990 es Skandia published Intellectual Capital reports, that included a few measures said to be from the internally used Skandia Navigator model, i.e. the BSC, used by Skandia. It has, however, been reported that the BSC was actually not used in practice by the firm (Olve et al., 2003 chapter 3), After a number of organizational changes in 1998 a new generation of BSC s was developed and Skandia ceased producing the externally published Intellectual Capital reports.
20 4_Bukh Sida Re-examining the cause-and-effect principle of the Balanced Scorecard gests that there should be a positive association between perceived environmental uncertainty and the perceived importance of externally-oriented and non-financial information (e.g. Gordon & Narayanan 1984), and the broader scope of the measurements as well as more timely information (Chenhall & Morris 1986). All these factors suggest that developing a BSC would be an appropriate approach when environmental uncertainty is perceived to be high. It should be noted, however, that the above research assumes that the primary role of performance measurement is to support decision-making (as opposed to control, i.e. to reduce goal divergence). However, uncertainty may also influence the willingness of management to strongly commit to any one strategy. As there is a risk that the chosen strategy may prove wrong, aligning measures and organizations to that strategy may be risky. Leaving strategy unspecified, and relying on an organization s ability to cope with uncertainties may be more viable approaches than selecting and mapping out a strategy and asking an organization to commit on that. Finally, rapid changes in the environment could also influence other aspects of the use of the BSC. Kaplan & Norton suggest e.g. the use of a value time line (Kaplan & Norton 2004, p. 379) that describes how the economic benefits from the different parts of the processes that are to be improved are distributed over several years. Because of the anecdotal evidence from some disguised cases (Kaplan & Norton 2001, p. 123ff) it is difficult to say how this will work in practice as it requires a very determined effort to keep focus over time, especially in changing environments. In practice, even firms in rather stable environments experience frequent adjustments and changes to strategies that need to be reflected in the strategy map, thus providing a space for the so-called emergent strategies (Mintzberg 1994). Additionally, it is our experience that very few firms are able to succeed with their first attempt to develop a BSC. Usually, the experiences gained from developing the first BSC lead to major changes in both the structure of the BSC and the management processes around. This further complicates the idea that the overall BSC should carve out a path for several years of improvement oriented activities, especially in environments characterized by rapid change. Moreover, constant amendments to strategy maps and the BSC required to cope with changing environment cast some doubts on the claim by Kaplan & Norton (2001, p. 115), who bluntly state that the BSC is especially useful in fast changing environments. To our knowledge, no research exists to support such a claim. To sum up, the use of cause-and-effect principle with BSC may not necessarily prove beneficial under the conditions of environmental uncertainty and rapid change, if the prime purpose with BSC is to commit people to act on strategy, instead of providing them with information to support decisionmaking. This, however, is also an area where more empirical studies could be beneficial to understanding when the balanced scorecard and strategy mapping could be useful.
How to Develop and Monitor Your Company's Intellectual Capital Tools and actions for the competency-based organisation The Frame project Nordic Industrial Fund How to Develop and Monitor Your Company's
NOVEMBER 2005 OLIVER BAKEWELL & ANNE GARBUTT SEKA Resultatredovisningsprojekt The use and abuse of the logical framework approach INTRODUCTION The logical framework approach (LFA) has come to play a central
Research report January 2010 CREATING AN ENGAGED WORKFORCE CREATING AN ENGAGED WORKFORCE FINDINGS FROM THE KINGSTON EMPLOYEE ENGAGEMENT CONSORTIUM PROJECT This report has been written by: Kerstin Alfes,
Making Smart IT Choices Understanding Value and Risk in Government IT Investments Sharon S. Dawes Theresa A. Pardo Stephanie Simon Anthony M. Cresswell Mark F. LaVigne David F. Andersen Peter A. Bloniarz
Using Case Studies to do Program Evaluation E valuation of any kind is designed to document what happened in a program. Evaluation should show: 1) what actually occurred, 2) whether it had an impact, expected
REPORT General counsel: vague about value? A survey and discussion paper Contents 01 Foreword 02 Flashback: The GC value pyramid 05 Key findings 06 Adding value 08 Case study: Daragh Fagan, General Counsel
E U A P U B L I C A T I O N S 2 0 1 0 Examining Quality Culture: Part 1 Quality Assurance Processes in Higher Education Institutions By Tia Loukkola and Thérèse Zhang Copyright by the European University
Self-Paced Course In this course you will explore and develop your unique leadership style, and identify what kind of leadership would be most effective for your particular situation. You will create a
How to manage performance booklet We inform, advise, train and work with you Every year Acas helps employers and employees from thousands of workplaces. That means we keep right up-to-date with today s
Are you innovation ready? Plotting your journey on the Innovation Readiness Model Message from Andy Green CEO, Logica I believe that the future success of our global economy relies on building a dynamic
OVERVIEW Brief description This toolkit deals with the nuts and bolts (the basics) of setting up and using a monitoring and evaluation system for a project or an organisation. It clarifies what monitoring
Basic Marketing Research: Volume 1 Handbook for Research Professionals Official Training Guide from Qualtrics Scott M. Smith Gerald S. Albaum Copyright 2012, Qualtrics Labs, Inc. ISBN: 978-0-9849328-1-8
A PRACTICAL GUIDE FOR TEACHERS AND CURRICULUM DESIGNERS GIVE ME TIME TO THINK DETERMINING STUDENT WORKLOAD IN HIGHER EDUCATION UNIVERSITY OF OULU TEACHING DEVELOPMENT UNIT ASKO KARJALAINEN KATARIINA ALHA
PEOPLEINAID Code of good practice in the management and support of aid personnel The People In Aid Code of Good Practice, 2003, is a revision of the People In Aid Code of Best Practice, 1997 PEOPLEINAID
The Systems of Innovation Approach and Innovation Policy: An account of the state of the art By Charles Edquist Home page: http://www.tema.liu.se/tema-t/sirp/chaed.htm E-mail: Charles.Edquist@tema.liu.se
JCMS 2006 Volume 44. Number 2. pp. 391 417 Who Has Power in the EU? The Commission, Council and Parliament in Legislative Decisionmaking* ROBERT THOMSON Trinity College, Dublin MADELEINE HOSLI Leiden University
How to Choose a Leadership Pattern by Robert Tannenbaum and Warren H. Schmidt No. 73311 MAY JUNE 1973 How to Choose a Leadership Pattern Robert Tannenbaum and Warren H. Schmidt Since its publication in
EVALUATION PRINCIPLES AND PR ACTICES AN INTERNAL WORKING PAPER THE WILLIAM AND FLORA HEWLETT FOUNDATION Prepared by: Fay Twersky Karen Lindblom December 2012 TABLE OF CONTENTS INTRODUCTION...3 History...
DOI 10.1007/s10734-007-9065-5 Is that paper really due today? : differences in first-generation and traditional college students understandings of faculty expectations Peter J. Collier Æ David L. Morgan
How to develop thinking and assessment for learning in the classroom Guidance Guidance document No: 044/2010 Date of revision: November 2010 How to develop thinking and assessment for learning in the classroom
EUROPEAN CENTRAL BANK WORKING PAPER SERIES WORKING PAPER NO. 26 WHICH KIND OF TRANSPARENCY? ON THE NEED FOR CLARITY IN MONETARY POLICY-MAKING BY BERNHARD WINKLER AUGUST 2000 EUROPEAN CENTRAL BANK WORKING
Integrated Reporting Performance insight through Better Business Reporting Issue 1 kpmg.com 3 Section or Brochure name Contents 2-3 Introducing Integrated Reporting 4-5 Some common questions answered 6-9
european cultural foundation www.eurocult.org Why we need European cultural policies by Nina Obuljen the impact of EU enlargement on cultural policies in transition countries Why we need European cultural
How Can a Medium-Sized Bank Develop Its Own Asset/Liability Risk Management System? MANUAL Based on presentation of Dr. Judit Burucs, Project Manager, Russia Banking Advisory Project Acknowledgements A
Rapport 18 Promoting and assessing value creation in communities and networks: a conceptual framework Etienne Wenger Beverly Trayner Maarten de Laat Ruud de Moor Centrum Ruud de Moor Centrum Open Universiteit