Submission to the Select Committee on Government Contracting and Procurement Processes

Transcription

1 page 1 Submission to the Select Committee on Government Contracting and Procurement Processes December 1999 Prepared by: Athol Yates Public Policy Unit Engineering House 11 National Circuit Barton, ACT 2600 tel: , fax: policy@ieaust.org.au www:

2 page 2 Introduction The Institution of Engineers Australia (IEAust) is a professional association representing 65,000 members of the engineering team. Of the Australian-based, practicing engineers about 25% work in the public sector and 75% in the private sector. Contract management in the Australian Public Service is a significant issue for members in both sectors as government engineering contracts are mostly delivered by the private sector. Improving the contracting process not only increases the efficiency and effectiveness of the public sector and reduces the cost of government, but also aids the development of Australian industry and encourages private sector innovation if the government is an informed buyer. 1 Government as an informed buyer Over the last decade, the Commonwealth, State and Local Governments' public sectors have been reduced in size considerably. This has resulted in a corresponding decrease in the number of specialists, including engineers within the public sector. Considering these changes, the IEAust believes that governments need to adopt new approaches in partnership with the private sector to ensure that they have access to the technical expertise required to be an informed buyer of engineering and information technology goods and services. The fundamental importance of having technical expertise was highlighted in the analysis of three recent disasters; the Royal Canberra Hospital implosion, HMAS Westralia ship fire and ESSO Longford explosion. Inadequate technical and contractual expertise was also cited in association with Collins class submarines. From a professional engineering and public sector contracting perspective, there were a number of common factors which contributed to the disasters. These included government contracting staff lacking the necessary technical and contractual expertise to manage engineering contracts, and government staff not adequately assessing contractors' and sub-contractors' engineering competencies. For example, in analysing the 1997 Royal Canberra Hospital implosion which killed a spectator, the ACT Coroner found that the officers appointed by the government owned Project Director, Totalcare, to manage the contractor were asked "to undertake a function well beyond their experience, qualifications and skills". 1 [Contracting failures was one of the major contributing factors to the Royal Canberra Hospital implosion disaster, the HMAS Westralia ship fire. 2 ] Having and utilising technical expertise is a pre-condition in becoming an informed buyer for engineering, information technology and other technical goods and services. It is crucial that buyers are well informed so that they are able to select and justify the option which offers best value for money; select and justify an innovative solution; reduce risks by preparing technically specific contracts; and preventing unscrupulous contractors taking advantage of the buyer's lack of knowledge. 1 ACT Coroner, 1999, Executive Summary of the Inquest findings, comments and recommendations into the Death of Katie Bender on Sunday, 13 th July 1997 on the demolition of the Royal Canberra Hospital Action Peninsula, ACT, pp For an analysis of these disasters, see Attachment 1.

3 page 3 The link between being an informed buyer and contractor innovation was highlighted in a recent IEAust survey. The 1999 survey of winners of Australian engineering excellence awards revealed that informed customers were a key to companies creating innovative products and services. The survey revealed that the third most important factor in a company developing innovative products and services was "informed clients who were capable of assessing risk in leading edge options". Conversely, the survey revealed that the most significant factor which had a negative impact on their company's willingness to produce innovative products and services was "uninformed customers being unwilling or unable to purchase innovative solutions". 3 Government purchasing guidelines emphasise the importance of being an informed buyer. This requires two distinct skill sets: contracting expertise skills and subject matter expertise. For engineering goods and services, the subject matter knowledge required is engineering technical expertise. The need to have technical expertise does not necessitate that an engineer is the contract manager. The view that 'engineers must manage technical contracts as they are the only ones to understand technical issues' does not advance the discussion. Nor does the counter argument that 'engineers who manage technical contracts always strive for gold-plated solutions' encourage a new approach to the issues. The discussion needs to move past these stereotypes to recognise that both contract skills and technical skills must be bought to bear on contracts and this is the only way to maximise value for money. The danger lies in a non-engineering contract manager making technical decisions which may conflict with a subordinate engineer s recommendation. This can happen where price is the decider. Similarly the debate on where it is best for government to obtain technical expertise either in-house or contract it in is not productive. The critical issues are not where it is located, but how the government is guaranteed access to it when it is needed and whether it is truly independent. Access to technical expertise is becoming more of a critical issue for governments due to devolution of authority for contracting, decentralisation of control, mega-contracts and increasing technological complexity. These changes are resulting in increased decision making power of individual agencies and individual contract staff, increased size of contracts, and hence significantly increased financial costs for making uninformed decisions. While governments recognise the need to maintain and retain relevant expertise, changing conditions and contracting practice are often at odds with this objective. There are two areas of divergence often present between government contracting policy and practice. They are: loss of engineering expertise. Over the last decade there has been a 20% to 30% reduction in the number of engineers in the Commonwealth, State and Local government public sectors. 3 IEAust, 1999, Engineering input to the National Innovation Summit: Survey of winners of Australian engineering excellence awards, unpublished.

4 page 4 focus on contract management skills to the detriment of technical expertise. In the late 1980s and early 1990s, the shortage of contracting expertise and the reasonable number of technical specialist meant that priority was rightly given to improving the contracting skills of technical specialists. However since the departure of many technical specialist, the focus on improving contracting expertise has overlooked the growing problem of a decline in technical expertise. Both skills are essential to being an informed buyer. 1.1 Technical expertise varies with contract stage For engineering, information technology and other technology contracts, different types and depths of technical expertise are required for each stage of the contracting process. Using the Department of Finance and Administration's publication, Competitive Tendering and Contracting: Guidance for Managers, the contracting process can be divided into seven stages. The IEAust has identified for each stage the benefits that technical expertise brings. 1. specify the activity: Technical expertise is essential to rigorously identifying the desired functional levels, performance levels and constraints; identifying the full range of probable technical solutions; and advising on the risk, cost and functional/performance tradeoff of all options. 2. select the acquisition strategy: Technical expertise is essential to determining the appropriate acquisition strategy as different types of strategies result in different levels of technical expertise required, risk and cost. For example, evolutionary acquisitions may require greater on-going technical expertise than a one-off acquisition. 3. develop and release the tender documents: Technical expertise is essential to developing appropriate criteria so that a trade off can be made between the functionality, performance, risk and cost of each proposal; and developing performance based, sophisticated tender evaluation criteria which encourages tenderers to offer innovative solutions as they will be more confident that the subtlety of their bid will be appreciated. 4. evaluate the tender bids: Technical expertise is essential to determining if the proposals meet the users' requirements, and to assessing the validity of technical information. It is also critical in identifying engineering best practice and least life cycle costs (as opposed to just the cheapest initial price); reading between the lines of proposals to identify assumptions and omissions; and undertaking risk management assessment of the proposals which involves identifying the risks, quantifying the likelihood and consequences of the risks, and evaluating the proposed mitigation strategies. 5. decision and implementation: During debriefing, technical expertise is essential to explaining with authority why a bid was not chosen for technical reasons.

5 page 5 6. ongoing management: Technical expertise is essential to providing clear and concise engineering management oversight, control and advice to both government and contractor where appropriate. It is also essential in rapidly resolving issues which arise after contract signature such as clarification of contract wording and advice on issues too detailed to be included in the contract; ensuring quality assurance; and rapidly processing appropriate contract variations which both minimised wasted time and reduces cost by limiting claims to only those variations which are reasonable. Finally it is essential in representing the government's position in disputed warranty issues or defect claims. 7. evaluation and succession planning: If technical test methods are required to measure outcomes, technical expertise is essential to determining the appropriate test methods and their limitations. 1.2 Determining the appropriate level of technical expertise Determining the appropriate level of technical expertise required to be an informed buyer is a difficult value judgement. In addition, agencies need to decide whether the expertise should be in-house or contracted-in, and decide the best method to obtain and retain the expertise. Based on research, including interviews with over 50 government and non-government contracting personnel and contractors, the IEAust has developed a method which enables agencies to obtain and maintain the most cost effective level of technical expertise to be an informed buyer. The method is: 1. Examine the good or service to be procured to determine the level of technical expertise required to be an informed buyer. 4 The main factors in determining the level of technical expertise required to be an informed buyer are: technical complexity of the activity, risks associated with solutions, originality of the activity being contracted, strategic value of the activity, stage of the contract, ability of the contractor, and type of contract. 2. Evaluate the relevant existing level of in-house and external technical expertise available: The process to determine the existing level of in-house and external technical expertise available are: undertake an internal skills audit, identify the level of technical expertise in industry, and assess the appropriateness of the agency's current ability 4 Care has to be taken in this assessment as a non-technical manager will probably have difficulty judging the level of technical expertise required.

6 page 6 3. Undertake a cost benefit analysis of in-house versus contracted-in expertise at each stage of the contracting process. The main factors in undertaking a cost benefit analysis of in-house versus contracted-in expertise for each stage of the contracting process are: what efficiency and effectiveness improvements result from having a knowledge of the agency's operations; is the technical expertise required for the short, medium or long-term; does the contract require highly specialised expertise that is generally not required for most other contracts what is the dollar value of the contract and what is its transaction costs; what are the engagement costs (recruitment, tender costs) and on-going costs (training, management fee) for in-house and contract staff; how often do other agency staff require informal advice from technical experts; what is the fluctuation in the volume of work requiring technical expertise; and whether there is a critical mass of work which will provide the breadth and depth of experience and career path options for in-house experts. 4. Obtain and retain the required expertise. Once the level of in-house and contracted-in technical expertise is identified, the agency needs to obtain and retain it. If the expertise is to be in-house, tools to retain it include paying the appropriate remuneration for technical experts which reflect market demand, and offering a challenging and rewarding workplace to reduce attrition. If the expertise is to be contracted-in, methods to do this include developing a list of pre-qualified professional services providers, engaging engaging an above the line 5 contractor to provide advice on the work of the successful tenderer; utilising industry personnel on short-term projects and engaging recently retired experts. Conflicts of interest need to be considered for contracted-in personnel as they may also derive income from tenderers for the contract. The soon to be published, IEAust study, Government as an informed buyer, 6 identified the following recommendations so that government can be an informed buyer of engineering, information technology and other technical goods and services. Recommendation 1 Government agencies should explicitly recognise that both contract management and technical expertise skills are essential for the procurement of engineering, information technology and other technical goods and services. Recommendation 2 Government agencies should ensure that they have access to the appropriate level of technical expertise so they can be an informed buyer of engineering, information technology and other technical goods and services. 5 An above the line contractor is not a supplier of goods or of services related to specific hardware or software and therefore can provide independent professional advice. 6 IEAust, 2000, Government as an informed buyer: Recognising technical expertise as a crucial factor in the success of engineering contracts.

7 page 7 Recommendation 3 Government agencies should employ the following model to ensure they have access to the appropriate level of technical expertise in the most cost-effective manner. 1: Examine the good or service to be procured to determine the level of technical expertise required to be an informed buyer 2 Evaluate the relevant existing level of in-house and external technical expertise available 3 Undertake a cost benefit analysis of in-house versus contracted-in expertise at each stage of the contracting process 4 Obtain and where appropriate, retain the required expertise Recommendation 4 If a government agency decides that the most cost-effective way of accessing technical expertise is to have it in-house, the agency must develop a plan for the ongoing retention of that expertise. The plan can incorporate a variety of techniques including: maintaining a critical mass of in-house work by performing critical aspects of contracting activities in-house, temporary placement of public sector staff with relevant private sector organisations working jointly with contractors on critical areas, organising regular meetings and reviewing reports about critical issues with the contractor, paying the appropriate remuneration which reflects market demand to get the relevant level of expertise, and working towards offering a challenging and rewarding workplace to reduce attrition. Recommendation 5 If a government agency decides that contracting-in is the most cost-effective way of accessing technical expertise for part or all of its workforce, the agency should explore all contracting-in options to select the most cost effective method or methods.

8 page 8 Options include: contracting a consultant to supply both technical and contracting expertise, and to act on behalf of the agency in defining in detail the agency's needs, drafting the contract and procuring the good or service, selecting a number of professional service providers (PSPs) which could undertake the work and develop administrative processes to allow contract staff to engage any of them with minimal paperwork, after awarding the contract to the winning tenderer, commission an engaging above the line contractor to monitor the winning tenderer, utilising suitable industry personnel via agreements with relevant industry organisations to work on projects which offer no conflict of interest, and utilising recently retired experts. Recommendation 6 Far greater attention and resources be given to value for money, risk management and life cycle costing during the procurement of engineering, information technology and other technical goods and services. Recommendation 7 To implement the above recommendations, procurement and contracting guidelines for government agencies involved in the procurement of engineering, information technology and other technical goods and services should: explicitly recognise that being an informed buyer relies on accessing technical expertise for the procurement of engineering, information technology and other technical goods and services, explicitly recognise that both contract management and technical expertise skills are required for the procurement of engineering, information technology and other technical goods and services, adopt a methodical approach (such as in Recommendation 3) to ensure that Government agencies have access to the appropriate level of technical expertise in the most cost-effective manner, provide advice on the benefits and limitations of various methods of accessing expertise, on-going retention of technical expertise within agencies, and options for contracting-in technical expertise, and provide far greater advice on best value for money, rigorous risk management and life cycle costing. 0 Levels of accreditation and expertise of contract management personnel To be an informed buyer, two distinct skill sets are required. They are contracting expertise skills and subject matter expertise. For engineering goods and services, the subject matter knowledge required is engineering technical expertise.

9 page 9 The IEAust considers that greater attention needs to be given to both improving procurement skills and accessing technical expertise. Recommendation 8 The IEAust considers that accredited procurement training should be mandatory for government purchasing officers, particularly for complex, engineering purchases of high value. Regarding subject matter expertise, the IEAust considers that far greater attention needs to be given to ensuring that agencies have access to technical expertise and know when to access it. The expertise can be provided by in-house or contracted-in staff. Each of these options have advantages and disadvantages. 2.1 In-house expertise Advantages The first advantage of maintaining expertise within the organisation is that in-house staff have a far greater knowledge of the agency's operations than contracted-in staff. The knowledge includes both corporate knowledge, agency objectives, knowledge of relevant personnel, work currently underway elsewhere in the agency, and understanding of public sector ethics, administration and organisation. This eliminates the learning curve of contractors, which is particularly important if the work involves interacting with different parts of the organisation and gaining resources or assistance. It also enables the technical experts to set outcomes relative to the agency's operation rather than to industry-wide empirical averages or benchmarks which contractors would typically use. These averages are invariably lower than the potential ones. Secondly, in-house expertise can be redeployed within the agency quickly. This is particularly important if urgent projects arise which require rapid tasking. Thirdly, in-house expertise is generally more cost effective for long-term base workloads. Typical marginal costs for a consultant are about 2 times those of an employee. Another situation where in-house expertise is more cost-effective than contracted-in staff is where the transaction costs of organising a contract are significant compared to the contract value. For example, the ACT Department of Urban Services contracts out most design and construct jobs valued at more than $3000. However, it employs a draftsman to produce plans for a range of civil engineering work if the work costs less than $3000 Another situation is when agency staff occasionally require informal advice from technical experts. Fourthly, maintaining and developing in-house expertise ensures that the expertise will actually be available.

10 page Disadvantages There are a number of disadvantages to maintaining in-house expertise. Firstly, efficiency requires that the experts be continually deployed. In an environment where work is constant, this is not a problem but if work is sporadic, it may require creating work for them, or deploying them to other activities, which may result in an efficiency loss. Secondly, for contracts in rapidly changing technological environments, such as computer systems, ensuring in-house experts retain technical currency can be both costly and time consuming. It may entail considerable training domestically and internationally, and industry work placements. Thirdly, maintaining in-house expertise can be expensive in terms of human resource management. While it costs to recruit and provide initial training to staff, there is also a cost for providing on-going training, breath of experience and career paths so that the technical expertise remains current. For roles to be performed effectively by professionals and other public service staff it is necessary that the individuals concerned are able to maintain practice competency in accordance with the industry and professional association standards. Most agencies will have insufficient work to justify employing large numbers of specialised professional staff. Consequently, the breadth of expertise and experience which they can bring to bear on important issues will be limited. The loss of one key member of staff could expose the agency to a high risk of failure. Furthermore, these agencies will not be able to provide the structured career development opportunities that underpin the maintenance of in-house skills. 2.2 Contracting-in expertise Advantages Contracting-in engineering technical expertise has several advantages. Firstly, it provides agencies with access to a wide range, quality and quantity of technical expertise. This may be important for activities subject to large shifts in technology or where large projects are sporadic. The 1999 Review of the Bureau of Air Safety Investigation (BASI) stated that, "a few submissions noted the increasing difficulty facing BASI in obtaining and retaining personnel with the myriad of skills potentially required to address all chosen incidents (especially in the case of a major airline accident)". 7 Secondly, it reduces overheads as on-costs associated with in-house employees are eliminated. For some agencies, these on-costs can be significant. For example, not only does Defence have to pay for the person's administration and training courses, it also has to pay for relevant military training if the person is in uniform. But a higher rate must be paid to cover the discontinuous nature of the short term contract and the profit for the consulting company that supplies the person, if relevant. 7 Department of Transport and Regional Services, 1999, BASI Review 1999: A review of the efficiency and role of the Bureau of Air Safety Investigation, Canberra, pp. 46.

11 page 11 Thirdly, it can eliminate potential conflicts of interest. This was noted in the 1999 Review of the Bureau of Air Safety Investigation (BASI). Several submissions stated that one advantage of contracting-in air accident investigators was to gain "greater availability of independent advice in those cases where [the government] investigator's personal contact with industry could be perceived to result in a conflict of interest". 8 Other advantages include: elimination of the need to ensure there is sufficient on-going work to fully employ in-house experts elimination of the need to provide a critical mass of technical expertise It is important to note that even if technical expertise does exist in industry, consulting companies may be unwilling to offer their technical expertise due to commercial reasons, notably in high technology areas. This situation was noted by the Defence Acquisition Organisation. "Typically small firms of professional consultants are reluctant to insert themselves between Defence, as the customer, and large prime contractors and accept the risks to the survival of their business that comes with such a contractual relationship in a high-risk industry." Defence overcomes this reluctance by engaging specialists " under contractual terms and conditions that describe the provision of professional services, with personal and corporate liability severely limited" Disadvantages One of the main disadvantage of contracting-in technical expertise is that an agency loses control of certain functions. This can result in a loss of corporate knowledge at the termination of the contract and continued dependence on the contractors for further related work. A second disadvantage is that staff in other agency areas may be reluctant to seek informal technical advice as this has to be charged for. This can be an important issue if employees wish to get advice on whether they should seek formal, technical advice. A third disadvantage is that any experience gained by one consultant is often quickly lost as a different consultant may be used the next time or the experienced consultant has moved on. The experience usually is only gained by one or two engineers within the consulting company's office, not by the whole company. A fourth disadvantage is that the consultant may not fully understand the problems and difficulties of the client and provide an answer to what he/she think the client wants rather than what the client actually wants. This situation often occurs when the consultant is brought in for short-periods or when the consultant is not part of the team managing the contract, and is therefore excluded from knowing all the related issues. 8 Department of Transport and Regional Services, 1999, BASI Review 1999: A review of the efficiency and role of the Bureau of Air Safety Investigation, Canberra, pp Australian National Audit Office, 1999, Management of Major Equipment Acquisition Projects- Department of Defence, Performance Audit No. 13, Canberra, pp. 105.

12 page 12 A fifth disadvantage is that the principal still has to rely on a non-technical staff member to explain its requirements to the contracted-in person or body. If this relies on a fee for service approach, the service may be curtailed if the fee is inadequate and this again exposes the principal to a level of unquantified risk. Other disadvantages include: consultants do not have a detailed knowledge of the agency's operations for long-term contracts, consultants can be more expensive the desired skills are not available in the marketplace when required at the desired price the cost of selecting contractors can be expensive One approach is to set the number of in-house specialists at a level where they can complete the baseload of work available, and use contractors to handle the peaks. 3 Request for tenders and tender evaluation process There are considerable opportunities for improvements in the early stage of contracting. 3.1 Request for tender (RFT) documentation A common situation in government tender documentation is a failure to define accurately what is required and to include all relevant information. This causes tenderers to waste time seeking clarification and making educated guesses about what is actually required by government. The net result is tenders which are more complex than they should be as the solutions include unnecessary and costly flexibility, and more expensive than they should be as the solutions are inflated because of hedging. Hedging covers two main cost uncertainties. Firstly, the hedging covers the costs of liaison to ensure that the two parties are moving along the same path which at time is not very clear. Secondly, the hedging covers the costs of small changes in customer needs which are not worth going through a contract variation process. For IT contracts, the hedge is often up to 15% of the tender bid price. For the contractor, the advantages of a hedge is that the contractor can meet the agency's new needs part way into the contract without having to argue for variations, and that it gives the impression to the client that the contractor is easy to work with. However the cost to the ACT is inflated prices. Another major issue associated with RFTs is that agencies frequently ask for too much information and the cost of tendering becomes excessive. The RFT is usually put together by a team of subject experts who often ask tenderers to provide information which the author would like to know rather than need to know to evaluate bids. In some cases it has become clear that the government has not made a conscious effort to conduct an overarching review of the RFT to eliminate inconsistencies, reduce tendering requirements to matters affecting selection of the preferred tenderer and to remove unnecessary technical requirements in the specification.

13 page 13 Recommendation 9 The IEAust recommends that far greater attention be given by contracting staff to defining accurately and succinctly what is required and to including all relevant information in tender documentation. Formal tools such as value management should be used to identify stakeholders' needs. Contracting and technical staff should be cognisant of the costs of tendering and should reduce tender response requirements accordingly. 3.2 Time between the release of the RFT and the close of tenders A common situation which occurs with many government contracts is that the time between the release of the RFT and the close of tenders is too short. While the need for a short-turnaround time is understood, the short time results in tenderers having to work considerable overtime, and to put in tenders which are could be improved to include innovative solutions if more time was available. To add to the frustration of tenderers, the time for assessing the tenders by the government is often extended even though the rationale for giving such a short time between the RFT and tender close was because of the contract's urgency. An example of this problem is documented in the Coronorial report into the Royal Canberra Hospital Implosion. "The tenders for Stage 1 of the project opened on 3 rd March 1997 and closed on 18 th March It was not until 13 th March 1997 that the structural drawings provided to those on the short list were made available by way of addendum to the tender documents. It will be recalled that Canberra Day was a Public holiday and fell on Monday 17 th March Effectively this meant that the tenderers had only one working day after the probable receipt of the structural drawings, to inspect the site in any depth as to the steel mentioned in the columns and then to price their tender to take account of the size and quantity of steel in the columns in Stage 1. In the circumstances this was clearly an inadequate amount of time." 10 Recommendation 10 The IEAust recommends that the agencies acknowledge the direct link between the time available to tenderers and the quality of the submitted tender. Specifically, agencies must ensure that time allocated between the release of the RFT and the close of tenders be of sufficient duration so that the tenderers can explore and develop innovative tender solutions. Extensions of time should normally be provided where all relevant tender information has not been provided at the start of the tender period. 3.3 Tender evaluation plans A common situation which occurs with many large government engineering projects is that the detailed tender evaluation criteria are not released at the same time as the RFT is released. For these projects, often only the criteria contained in the tender evaluation plan are released with the RFT. However the tender evaluation plan is normally a high level document which provides little detailed guidance on the way individual elements of the tender will be evaluated. Typically, the time between the release and close of the RFT will be used to develop the sub-set evaluation criteria. 10 ACT Coroner, 1999, Inquest findings, comments and recommendations into the Death of Katie Bender on Sunday, 13 th July 1997 on the demolition of the Royal Canberra Hospital Action Peninsula, ACT, pp

14 page 14 Occasionally only when the sub-set evaluation criteria are developed is it realised that the wrong questions have been asked in the RFT. Recommendation 11 The IEAust recommends that for large engineering projects, the high level evaluation criteria and the sub-set evaluation criteria should be released with the RFT. If this is not possible, at the very least technical and tender evaluation experts should review the tender evaluation plan prior to release of the RFT to determine if the evaluation strategy is technically practical and will result in the statement of requirements being achieved. 3.4 Feedback on RFT documentation for contract officers A common situation which occurs with tender documentation is that it is ambiguous, imprecise and incomplete. This results in tenderers wasting time seeking clarification, nominating a low price with the aim of seeking profitable variations later, or adding a premium to the tender price which serves as a hedge against increased contract requirements. While the tenderers are well aware of the problems, they are often unwilling to bring them to the attention of the government's contract officers and their organisation as it may prejudice their bids. Consequently the contracting agency may not be aware of the problems and opportunities to improve contracting practice will be lost. One solution to this is to organise a formal review of specific tender documentation from which agency contracting staff can benefit. An independent consultant would be commissioned to interview several of the tenderers and identify areas of documentation excellence and areas where improvements could be made. This information would be presented to a relevant group of agency contracting staff with the specific goal of improving all tender documentation and processes. Care must be taken so that the review is not seen as the evaluation of one particular tender. It would also be useful to have a project debrief at the conclusion of the contract, which would include amongst other things, a review of the documentation in light of the experience gained during the contract, and to identify areas of documentation excellence and areas where improvements could be made. Recommendation 12 The IEAust recommends that consideration be given to trialing within an agency the concept of a formal review of tender documentation based on the feedback from the tenderers, as well as a project debrief at the conclusion of the contract to identify future improvements for documentation. 4 Getting value for money The IEAust considers that 'value for money' is the correct goal for procurement officers in selecting a tender. However the IEAust is concerned that due to a lack of guidance, practical methodologies and expertise, this goal is not being achieved. This view is reinforced by the conclusions of the 1999 Joint Committee of Public Accounts and Audit report, Australian Government Procurement, which noted that "most agencies could not provide evidence of their efficiency and effectiveness in

15 page 15 determining value for money". 11 The committee concluded that "there is no evidence to show that this principle is being applied correctly or consistently". 12 A review of the available government publications on 'value for money' indicates a paucity of information that is useful to members of tender evaluation teams. Guidance is, in general, of a conceptual nature and provides little in the way of examples or methodologies for assessing the merits of tenders on a 'value for money' basis. Four methodologies are listed below which all encourage the prime selection criteria to be value for money rather than lowest up-front cost. Value Management does this by providing a flexible process to contract delivery that allows and encourages contract changes to be made easily and as early as possible, if they can improve the contract's value for money. Qualification Based Selection (QBS) does this by determining the price only late in contract negotiations. Pre-qualifications schemes do this by limiting the potential contractors to those who meet range of agencydetermined attributes. Registration schemes do this by ensuring that potential contractors are competent and ethical. Value management: Value management (VM) is not a single method, but a framework within which proven methods are systematically brought together to identify better value from projects. The key practices and principles are not new, but it is the way they are integrated and deployed that makes the approach so effective. The framework provides the link between performance within an organisation and the external influences of its customers and the market place. The VM framework provides a structured approach that can be applied at all levels and stages of a project, and by all participants in the project. Of particular importance however are the VM workshops at the start of projects which bring together all the stakeholders in the project to undertake a comprehensive needs analysis and identify trade-offs for various options. Further VM activities may be undertaken with potential contractors before the tender is released to identify options which meet the stakeholders' needs and provide best value for money. A recent example of the use of VM in construction was in the South Australian Crafers Highway project. The IEAust recommends that the Committee examine the advantages that VM offers for contracts and visit the Crafers project and other projects which have extensively used VM to study the practical application of VM. The New South Wales Government also has in place a policy that agencies must have conducted at least one Value Management Study before budget approval is given for capital works or maintenance projects which will cost $5 million or more. Qualification Based Selection: QBS involves the agency issuing a Request for Qualifications to a range of potential contractors, including broad details of the client agency's service requirements. From the respondents, the agency selects three preferred bidders (on the basis of qualifications, demonstrated competence and capabilities), and commences negotiations with the most highly qualified to finalise detailed service specifications, contractual terms and contract price. The Association of Consulting Engineers Australia, the Australian Council of Building Design Professions and the Institution of Engineers Australia recommend QBS as 11 Joint Committee of Public Accounts and Audit, 1999, Australian Government Procurement: Report 369, Canberra, pp. xxii. 12 Joint Committee of Public Accounts and Audit, 1999, Australian Government Procurement: Report 369, Canberra, pp. xxii.

16 page 16 "the preferred method for procuring engineering and management services" 13 as it ensures that "the procurement process obtains the best value for money and is fair, equitable and impartial". 14 pre-qualification schemes. For example, the Government Building Services Branch of the Tasmanian Department of Treasury and Finance maintains the Consultant Pre-qualification Scheme which provides a record of the consultants qualified to perform Government work. "This Scheme serves to maintain a comprehensive record of all consultants available, their expertise, experience, capacity to perform and a history of their performance on Government work." 15 registration schemes for contractors & contracting companies: Using engineers or engineering companies which are registered on an independent competency-based register guarantees that they are professionally competent and therefore, a superior quality of work can be expected from them compared with non-competent engineers or companies. Examples of registers are the National Professional Engineers Register (NPER), National Engineering Technologists Register (NETR) and the register maintained under the Queensland Professional Engineers Act The NPER registers engineers in a number of categories of professional engineering work and to be included in a category requires the engineer to demonstrate education, competency and experience requirements. To remain on the register, engineers must have undertaken 150 hours of continuing professional development over the last 3 years. However the register's most secure protection for the client and the community is its requirements that engineers conform with its code of ethics. It requires that engineers must practice only within the limits of their personal and professional competence, and be subject t to disciplinary action if they fail to observe that constraint. The NETR is similar to the NPER except that it registers engineering technologists. The Queensland Professional Engineers Act 1988 governs the registration of Professional Engineers, Professional Engineering Companies and Professional Engineering Units. It prohibits a person who or an organisation than is not registered under the Act from offering professional engineering services for a commission. One of the objects of the Act is "to promote the public interest by ensuring only competent qualified persons practise as registered professional engineers in Queensland". 16 One of the main market failures that this Act is addressing is information asymmetry. This "occurs because the inexperienced and uneducated customer does not have the ability to determine the appropriate cost and quality trade-off when purchasing complex engineering services" The Association of Consulting Engineers Australia, 1995, Qualification based selection: The probity perspective, Sydney, pp The Association of Consulting Engineers Australia, 1995, Qualification based selection: The probity perspective, Sydney, pp Department of Treasury and Finance, 1998, Best practice for the engagement of consultants, Tasmania, pp Board of Professional Engineers of Queensland, 1997, Annual Report 1 July 1996 to 30 June 1997, Brisbane, pp Yates, A., 1999, "Legislative Reviews of Acts regulating professional engineers" in Transactions of Multi-Disciplinary Engineering,, pp. 38.

17 page 17 However these approaches will only work if the government's contracting staff have the expertise which enables them to justify their selection when it was not the lowest cost, compliant tender. Recommendation 13 The IEAust recommends that greater guidance be given to contract officers on how to assess tenders on the basis of value for money. Consideration should be given to employing relevant methodologies such as value management, qualification based selection, pre-qualification schemes and registration schemes. 5 Contract outcomes evaluation The IEAust considers that there is considerable room for improvement in evaluating the outcomes of a contract and learning for its lessons. The evaluation requires a rigorous data collection, seeking feedback from stakeholders and analysing the contract's outputs and outcomes. To make the maximum use of the evaluation, information on the lessons learned needs to be disseminated. This process is time consuming and frequently identifies issues that may put certain staff in a negative light. Consequently there is reluctance to often evaluate the outcomes of contracts and to disseminate the information. Recommendation 14 The IEAust recommends that agencies undertake a formal evaluation of the outcomes of all contracts and disseminate the information. A significant proportion of the reviews should be undertaken by external staff such as auditors and consultants. To ensure that the evaluations are undertaken, money must be set aside for them when developing the budget for the activity. 6 Improving contracting administration nationally Many professional and industry associations have an interest in contracting but none is dedicated solely to improving contracting. This is despite the fact that it has become a profession in its own right and one critical to the success of an increasing number of organisations which outsource or contract out their activities. The IEAust considers there is significant interest in forming an Institute of Contract Administrators with the aim of improving practice in contracting in both the public and private sector. The work of the Institute would be to serve as a broker in contracting training, promoting contracting competencies and highlighting better practice in contracting. It would also run a web site which lists the full range of both public and private standard conditions of contracts, guidelines and other useful information for contractors. Unlike existing sites, it would include an analysis of which contracts/guidelines are relevant for what type of contracting activities, and the advantages and disadvantages of each.

18 page 18 The Institute would also provide annual awards for contracting excellence (eg best standard contract, contracting guideline, contracting policy, and contract administration, contracting evaluation, and lessons learned) as a way of promulgating better practice. The ACT Government may need to play a facilitating role in establishing such an Institute. Recommendation 15 The IEAust recommends that consideration be given to facilitating the formation of an Institute of Contract Administrators. 7 Risk allocation 18 One of the aims of contracting out is transferring appropriate risk to the private sector and away from taxpayers. Achieving optimal risk allocation, and hence value for money, is one of the most important aims of government when considering contracts. In order to achieve this, governments need to clearly understand the risks inherent with a particular project and to allocate those risks only to those parties who are best able to control and manage them. Of course, the government itself may be one of these parties. In determining the optimal allocation of the risks governments should assess which parties have access to reasonable mitigation techniques and to what extent they can influence the probability of occurrence and the degree of consequence of the risk. In cases where risks cannot be easily managed by either party (the contractor or the government) they should be shared in a way which maximises the incentives of both of them to avoid the risk, or to manage it at least cost to the project and hence to the community. All of this presupposes that governments and government agencies are able to identify, assess and value the risks they bear when they provide services. It is then possible to assess the benefits of passing on those risks to contractors. If governments, or their agencies, fail to identify and assess the risks in a project, not only may they fail to achieve value for money, they may also expose taxpayers to unnecessary risks both now, and in the future. Considerable research has gone into determining the optimum risk transfer in BOOT projects by Raphael Arndt. The methods applicable to assessing the suitability of risk transfer in BOOT projects are also applicable to other forms of contracting out or outsourcing, albeit that asset related risks may not be relevant. There are two basic factors to be considered when determining whether a party can best control and manage a risk. These are the access of the party to suitable mitigation techniques and the ability of the party to influence the probability of the occurrence of, or the consequence of the risk. 18 This section is based on the research of the IEAust member Raphael Arndt. Arndt, R., 1999, "Optimum risk transfer in build-own-operate-transfer projects: the challenge for governments" in Transactions of Multi-Disciplinary Engineering,, pp. 1.

19 page 19 Firstly, if it is possible to mitigate against a risk at a reasonable cost then the contract should be structured in such a way to ensure that this occurs. For example if commercial insurance is available for a certain risk at reasonable terms then it should be taken out. Similarly if the project is exposed to foreign exchange risk then hedging foreign currency revenue or debt is a possible mitigation technique. It is not essential that these mitigation techniques are adopted, only that they are considered and, if rejected, that the reason for doing so is a commercial decision based on the relative cost of the mitigation assessed against the significance of the risk. If decision making parties are forced to consider these issues then the most efficient out-comes will be achieved at the total lowest cost to the community. Secondly, it is obvious that if a party has the ability to influence either the likelihood of occurrence of a particular risk, or its consequences if it eventuates then that party should bear the risk. This ensures that all practical steps are taken to avoid the risk or to minimise its effects if it does materialise. Hence the total potential cost to the community is reduced. It is important to note that this proviso applies equally to private contractors and to public agencies. A good example of this is the risk of future changes in environmental legislation as they apply to a contract for the provision of a plant and associated services. Assume the project may be designed to comply with current legislation but upgrading it would be very expensive. Alternatively it may be built to comply with anticipated future standards now, but at a greater capital cost. The designer may also have an option to build a plant which just meets current standards but is flexible. This plant would be cheaper to upgrade if standards changed (but of course it would cost more now to make it flexible). Only the contractor has the power to make all of these design decisions and factor them into the design price. It therefore makes sense for the contractor to bear the risk of changes in environmental requirements. The contractor would, of course, charge a greater amount now in order to build the flexible or larger plant (or to take the risk of an expensive up-grade of the inflexible but cheap plant if that is the option chosen). However, if the government indemnified the contractor against future changes in environ-mental standards, then they would certainly choose to build the cheapest (and least flexible) plant. If the legislation changed when the government bore this risk then the community would have to pay for an expensive upgrade. The most efficient outcome in this case is to internalize the decision making related to this risk by allocating it to the party which can most influence the consequences of the risk eventuating, the private contractor. Recommendation 16 The IEAust recommends that governments and their agencies perform a detailed risk identification and analysis study prior to entering any contracting out project. Risks which are identified as being most easily controlled and managed by the contractor should be passed to them in the contract. Other risks should be retained by government if it is the party which can best control and manage those risks, or else they should be shared. Public agencies should always assess the risks to which taxpayers are exposed in any project or when delivering any service. Services which are provided by a private contractor are no exception.

20 page 20 8 Data collection Government has a number of roles including achieving social goals, rectifying market failures, managing macro-economic policy and caretaking of community property. In order to make effective judgements in these areas, reliable data is essential. Engineering and technology-services are particularly dependant on reliable data, as information reduces uncertainty and hence ensures that unnecessary capital is not spent in over-design or, conversely, that under-design does not lead to expensive failures. The costs of uncertainty can be significant as illustrated by Figure 1. This shows the cost of over-design or under-design (failure) of a 1km long breakwater on the Gold Coast depending on the accuracy of the information (percentage uncertainty) derived from the data. Figure 1: The cost of over and under design depending on percentage of uncertainty of data 19 Another example of the cost of uncertainty is because of inadequate flood records. Most structures around rivers and coastal areas in NSW are designed to be not affected by a one in a hundred year flood. To predict the worst-case flood level with accuracy requires about 300 years of continuous records. Because this data is lacking in most areas and even today is often not being collected, an extra 500mm of height is added to the best prediction of the flood height of a one in a hundred year flood. The extra height, achieved through soil in-fill and taller footings, costs the economy hundreds of millions of dollars and much of it is probably unnecessary, according to the Institution of Engineers' National Committee on Coastal and Ocean Engineering. 8.1 Recent Changes in Governments' Data Collecting Practice and Management Traditionally, governments have had a major role in data collection. There were two main reasons for this. Firstly because most infrastructure, such as roads, utilities and ports, were owned by government, the government collected its own data. Secondly, technical staff often headed up engineering-related organisations and they were aware of the value of data and were reluctant to make decisions without having the necessary data. 19 IEAust, 1993, At What Price Data?