Picking Successful Projects By Using the PRISM^TM  Model

Introduction

Too many contractors are still going out of business within five years of start-up. Dun and Bradstreet records indicate that as many as 56% of the contractors that started up in 1995 have now gone out of business. There are various reasons for their demise; however, negative cash flow remains one of the prominent reasons cited. Contractors are bidding too much work, work that is either not a good match for their asset base, or work that is beyond their geographical or technical ability to control properly. (Engineering)

Several studies have shown that the cost of resources to prepare a complete hard-bid estimate for a construction project runs between 1/4 and 1/2 percent of the cost of the project. (Bajaj; De la Garza; Duffield) That means that if a contractor has to bid four $1 million projects in order to go to contract on one, the assets wasted on trying to procure the three unsuccessful projects could amount to as much as $150,000 dollars. In short, the contractor invests approximately $150,000 before (s)he is ever able to go to contract on the successfully bid project. Therefore, it is extremely important for contractors to make quick, accurate decisions regarding which projects to bid before committing the crucial resources to obtain the contract.

The construction industry encounters extremely high risk due to such factors as the uniqueness of every project and the exposure to external elements. (Kim) The majority of construction companies prepare cost estimates based on a traditional single figure cost estimation approach. (Leung) Although the estimators often consider risks, those risks are rarely reflected in the final estimate in a formal, systematic way. (Dexter; Diekman; Duffield) The most common method for protecting the company from potential risk is to add a contingency sum to the estimate. However, this approach has a number of weaknesses, including:

·         · The contingency figure is usually arbitrarily arrived at and may not be appropriate for the proposed project;

·         · Estimators have a tendency to double count risks;

·         · A percentage addition still results in a single-figure prediction of estimated final cost, implying a degree of certainty that may not be justified by the data available;

·         · This method assesses risk only as a negative variable and doesn’t consider any positive potential (opportunity).

As a result, the traditional single-figure approach can be inadequate, misleading and cost inefficient. (Ranasinghe) Therefore, the construction industry requires not only a different approach to managing the risks associated with project procurement but also a tool which organizes and quantifies project selection information and stores valuable project procurement knowledge for future decision makers and construction managers. The PRISM^TM Model (See Figure 1) incorporates three components in its development (PRISM l, ll, lll) and is intended to accomplish these goals.

The development of PRISM l is discussed in detail in this paper; whereas, PRISM ll and lll are discussed briefly primarily to generate creative discussion and feedback regarding their development. PRISM 1 focuses on creating a customized list of questions, which are quantified and ranked using Expected Monetary Value, Utility Theory, Pair-wise Comparisons, and the Critical Risk Ranking Matrix.

By developing this specific list of project selection questions, the model overcomes one of the greatest difficulties associated with project selection models: customization to specific company operations. Indeed, this customization is one of the model’s unique characteristics, for once the list has been customized, it reflects specifically how good a match a project is to the inherent abilities and assets of a company.

Further, the model provides a systematic approach in the identification and assessment of potential positive as well as negative risks; this, in turn, provides a management tool to help project managers choose appropriate courses of action in managing project selection and risk. In this way, critical knowledge can be stored which is easy to access, analyze and use in future project selection. Indeed, the PRISM 1 Model not only provides an innovative approach to project risk analysis and procurement using tried and tested risk management tools but also creates a template to give construction managers an at-a-glance quantitative assessment of the project as compared to other possible projects. The template developed is unique to the operations of the company for which it was developed and is robust enough to be used as a starting point tool for the development of customized templates for small, medium and large companies with different types of projects, assets and competencies. In this way, the PRISM l can provide the basic structure for multiple industry-wide use in project selection as well as risk management.

Methodology: The PRISM 1 Model

The PRISM 1 Model was developed using the classic risk management process, which includes identification, quantification, response and control. (Kerzner) The identification of potential risks, both positive and negative, for any construction project constitutes the most difficult part of this process. For purposes of this paper, negative risk will be called ‘Risk’ and positive risk will be called ‘Opportunity’. The PRISM l Model provides a process to identify (Step One), quantify (Step Two), mitigate (Step Three), and control (Step Four) risks and opportunities that can then be controlled throughout the project.

Step One: Risk Identification

Risk is defined primarily as the potential for monetary loss or gain associated with the project. Risk analysis uses a set of techniques and tools to identify, prioritize and investigate opportunities, problems, and risks in order to assess their cost and time impact on the project. To develop the risk management framework, the potential sources of risk must first be identified and categorized. Then a unique measurement system can be used to quantify the risks.

There are three categories of risks associated with any project. Risks can be classified as either knowns, known-unknowns, or unknown-unknowns. (Kerzner)

·         · Known risks are items or conditions that are understood but cannot be measured with complete accuracy. Generally, such risks occur at a relatively high rate and contain a wide range of possible outcomes. Labor productivity is a good example of a known risk. The PRISM 1 model is especially good at working with this category.

·         · Known-unknown conditions or events are foreseeable because historical data documented their occurrence on previous projects; however, they are not easily quantified. Normally, such events have a relatively low frequency of occurrence but may result in severe consequences. Earthquakes, hurricanes, unanticipated strikes or an unusual challenge with a new sub-contractor are examples of this type of risk. The PRISM ll model will be especially good at working with this category.

·         · Unknown-unknown conditions or events generally cannot be predicted because historical data do not suggest them as possibilities. These items can be catastrophic in nature and may have a very low probability of occurring. Indeed, examples of unknown-unknowns are hard to provide because historical data do not exist to support their possibility. Remodeling, excavations and unique, never been built before, one-of-a-kind projects would be examples of situations where a high number of unknown-unknowns would exist. (Kerzner) The PRISM lll model will be especially good at working with this category. Once an unknown-unknown happens and is identified, it is logged into one project database and then becomes a known or known-unknown for all future projects.

Currently, in PRISM 1 development, risk identification is heavily dependent upon the experience and perception of project managers and a quality historical database. Unfortunately, this historical database is rarely fully developed and maintained by most construction companies. In his research article "Practice, Barriers, and Benefits of Risk Management Process in Building Service Cost Estimation," Leung contends that approximately eighty percent of the construction industry uses intuitive assessment in measuring risks. (Leung)

Qualitative risk assessment models, which compare both risk and opportunity variables, are presently used in other industries to assess relative project viability before committing the resources required to develop a project plan and customer proposal. (Ranasinghe; Baker) These models are usually completed by the project manager working with the project team in order to assist the project team and senior management make decisions regarding which opportunities should be managed to ensure project success. However, these models should not be used as a set of absolute rules or as a substitute for good business judgment but rather regarded as guidelines for good qualitative decision-making. That is, these models have been used to date primarily by senior management to make go-no go project selection decisions, not to analyze and manage risks. They tend to be qualitative in nature, not quantitative. The PRISM 1 Model adds to the value of these tools by making the transition to valid quantitative output, which provides a much more valuable decision-making to tool for the contractor.

There are three steps involved in implementing most existing qualitative Risk/Opportunity models.

1. The first step is to identify and quantify the risks. A series of questions are formulated, answered and quantified using a Likert scale (1-10) and a weighting factor based on relative importance to the company. The risk scores are calculated by multiplying this raw score times the Risk Probability (P) and the pre-established Risk Impact (I). These scores are summed for a Total Risk Score.

As with existing models, the users of the PRISM 1 Model develop and then qualitatively answer a series of detailed questions, approximately 40, that will identify the risks (20). Step two (below) repeats this identification process for opportunities (20) as they relate to a potential project. Determining which questions should be asked is one of the most difficult tasks in this process. The questions are developed and reviewed by as many of the stakeholders who impact the performance and profitability of a project to ensure comprehensiveness as well as applicability to the company. Based on stakeholder feedback, questions are stated in such a way so that they can be used for every project and yet answered to reflect a specific project’s risk. Also, a wide range of questions can provide the basis for analyzing groups of projects, past and present, as a portfolio. The questions are then pared down by using the Critical Risk Ranking Tool (Figure 2), which compares risks to each other (e.g. Risk A to Risks B, C, D, E). This comparison can then be used to pare down the final list of questions/risks (down to +/- 20 in total, with 10 risks and 10 opportunities)..

2. The second step is to evaluate opportunity in a similar fashion, using the opportunity questions, weighting factor and Likert scale. The opportunity scores are then calculated by multiplying the Possible Opportunity (P), times the pre-established Impact (I). These scores are summed for a Total Opportunity Score.

3. The third step is to map the opportunity and risk scores. The total scores for opportunity and risk are used as coordinates on the matrix provided within the model, with the opportunity score placed on the y-axis and the risk score on the x-axis. The location of this score on the matrix helps determine the potential profitability of any one project opportunity compared to another and serves as an indicator of the level of risk that will have to be managed in order to ensure that project’s success. However, the PRISM 1 Model presents a further innovation: it converts these data into dollars using cost indices. (See Figure 5 later in the paper) Inputting as much possible data from as many past projects as possible improves the accuracy of this cost index; in this way, the data are expressed in the language of business: money.

The determination of the correct cost index constitutes the most difficult part of the development of the model for a specific contractor. Fine-tuning of the model must continue with each new project because the model actually helps to identify risk and opportunity events that were not predicted to be significant originally by the developers of the model. For example, the PRISM 1 model predicted the correct potential profitability index for seventeen out of twenty sample projects input to the model. The predicted profit margins fell within 1/2 percent of the actual on the profitability index scale. When analyzing the three projects where the actual profit fell outside of the accepted variance range, it was discovered that 2 of the projects contained an exceptional amount of earthwork. The research partner-contractor was exceptionally good at and well prepared for earthwork; therefore, he was able to create more profit from the project than was predicted by the model. This was cause for revisiting the opportunity questions. The result of this review was to include 2 new questions that related to the relative amount of earthwork included in any project being considered for bid. After these questions were incorporated, the prediction of profitability fell to within 1/2 percent of the actual. The adjustment to the model for the third project that fell outside allowable variance will be discussed later.

After several reviews and customizations to the lists of questions by various stakeholders and clients, the following list of questions was compiled. This list (Table l.) of questions is generally regarded by these companies to have significant impact on project profitability:

Step Two: Risk Assessment and Quantification

The quantification process applied to the list of identified risks is challenging in that each person assigning a value to the question will be subject to their own prejudices and utility values. Therefore, the quantification process relies heavily on Utility Theory. (Bernstein; Render; Dawood)

Utility Theory allows decision makers to incorporate their risk preference and other factors into the decision-making process. In order to measure the utility of an outcome, these outcomes are assigned a value of utility: the worst given a value of zero, the best a value of one. The utility assessment is completely subjective in that the value set by the decision maker is not usually measured on an objective scale. In fact, this assessment can be based on the personal preferences of the decision maker, i.e. those (s)he is most comfortable pursuing. For example, a construction company or decision-maker might place a higher utility in performing projects within a certain geographical area or designed by a certain designer. This might not be in direct relation to their ability to perform the project.

Another of the tools that a decision maker can employ is a decision tree. A decision tree is a graphical representation of the analysis of choosing between two or more alternatives. Each alternative is assigned a payoff amount and a probability of receiving that payoff . To decide which alternative to select, the expected monetary value (EMV) is computed by multiplying the payoff amount (impact) by the probability. The sum of the decision ‘branches’ is then compared to determine the best alternative. The following figure (Figure 3) is an example of an analysis using a decision tree:

Utility theory assigns a preference for each desired result. It requires the decision maker to determine the value of the probability of an event that makes him/her differentiate between alternative 1 and 2. For example, in deciding the preference between $0 and $10,000, one has to develop a utility curve that plots monetary value versus utility value. After a utility curve has been determined, the utility values from the curve are used in making decisions. Monetary values are replaced with the appropriate utility values and then decision analysis is performed as before. The utility curve indicates the amount of risk a person is willing to take. The increasingly upward sloping curve indicates a risk seeker while the curve that increases at a decreasing rate indicates a risk avoider. The following graph (Figure 4) illustrates this point:

Obviously, using utility theory to determine which projects to pursue requires the consideration of more than the possible profit. The decision to take a project might have a higher expected monetary value than to not to take the project; however, the willingness (utility value) of the contractor to accept the risk might discount the overall return of the project. Thus, it would be wise for the contractor to pass (no/go) on the project.

The next step is to quantify the impact of these risks on the project (See Table 2) As this table illustrates, the decision maker uses utility value to convert a qualitative question of impact into a quantitative input to the overall decision-making process. This part of the process is similar to the method many industries, especially information technology and communications, have already adopted using risk evaluation models. To date, however, the PRISM 1^TM Model is the first published attempt at producing a risk management/profit predictor tool specifically for the construction industry.

Most existing models ask a series of questions that evaluate risk and a separate set of questions to evaluate opportunity. The total scores from the questions are mapped as coordinates on a matrix. (Aspinwall) When using the PRISM 1^TM Model, the project manager places the opportunity score on the right vertical axis and the risk score on left vertical axis. (See Figure 5) The intersection of the scores on the center profitability index determines the final profit value of the risk/opportunity assessment for a specific project. The location of this score on the index helps determine the quality of an opportunity and serves as an indicator of the level of risk to be managed for overall project success.

Thus, this model utilizes a different approach to risk quantification using a combination of Decision Tree Analysis, Utility Theory, Expected Monetary Value and Risk/Opportunity Assessment Models.

On a final note regarding risk identification, project estimates are often based not only on the knowledge of the estimators and schedulers as well as on the experience and data for similar projects completed previously, but also on a large number of assumptions made regarding productivity rates and materials prices. Some components of the projects are prone to variation, such as material prices. Other items such as labor productivity rates can be sensitive to many factors, including weather, temperature, state of the economy, union involvement, and project duration and cost. This model is intended to provide a logical mechanism for predicting the extent of these variations and forecasting their impact on the project. The principal use of the output from the model is not to discourage a company from bidding on projects with high risk but to identify risk and encourage proactive risk management in a more cost effective and timely manner.

Step Three: Risk Response

A spreadsheet program, such as Excel, can be used to rank the relative importance of each risk area. Table 3 presents a sample output from the data collected during the research.

As illustrated, a simple descending order ranking can set all of these risks up as a prioritized list, making the list an ideal foundation for developing risk mitigation strategies as part of the overall risk management plan. The contractor can now develop an effective risk response to each identified risk and opportunity on a cost/time effective prioritized basis. Many of these identified as critical risks need to be addressed in the contract itself. If the model is not used, the opportunity for mitigation using the actual contract itself might be missed entirely. These responses can take the form of avoidance, acceptance or mitigation. If mitigation is selected as the most effective response, the contractor develops a response strategy that either reduces the probability of the Risk event occurring (or increases the probability in the event of an Opportunity) or reduces the impact if the event occurs (or increases the impact in the case of an Opportunity).

Step 4: Risk Control

As risks occur during the execution of the project, the strategies suggested in Step 3 Risk Mitigation are implemented. The results are then evaluated and documented for future projects. Experience has shown that the best way to prepare future risk management plans is to access and apply the historical data from past projects, the lessons-learned database. This is one of the important benefits of using the PRISM 1^TM Model: the database generated is primarily graphic in format. This graphic facilitates rapid understanding and, with additional information such ‘assumptions’ and ‘constraints’, can become a useful lessons-learned database for future projects.

Summary: The PRISM l Model

Information from twenty (20) past construction projects has been used to date to provide input to this phase of the model. The research conducted using the model indicated that as more good historical data from past projects were generated using the process and templates, the accuracy of the model as a predictor of project profitability improved significantly. Indeed, contractors recognized the value of the model as a powerful means of improving project procurement.

Several issues needed to be addressed in the development of PRISM l. For example, the model received input initially from project managers and estimators who were intimately familiar with each of the projects being evaluated. As they answered each of the risk and opportunity questions, they thought they were responding in terms of the risks and opportunities that existed back at the time the company was initially evaluating the projects as worthy of full estimating effort. It became apparent that their judgment (utility theory) had been influenced by what they actually knew to be valid risks and opportunities from the perspective of hindsight. Therefore, the profitability predicted (See Figure 5) actually was reflecting the potential profitability of the project after all of the risks and opportunities had been realized. This probably should not come as to too much of a surprise, since it would be difficult for the project managers or estimators to divorce themselves from the realities encountered in the project. The data point resulting from the input from past projects correlated very closely with the actual profitability for each project.

As the results from more projects were collected, the PRISM 1 model was fine-tuned to reflect this additional information and suggestions for improvement in the model. For example, when researchers found that several of the questions regarding potential risks or opportunities have 0 or very low expected value, the natural question arose: Is this issue really that important as it relates to the way this company does business on this type of project? If not, eliminate the question and look for others that suggest greater impact on the profitability of the project. The other question that arose was: Are we missing issues that should be included but were not because we assumed they were not of major impact on the profitability of the project for this company? This possibility was highlighted when several project results understated the actual projected profitability of a project. Further study indicated that this particular company was uniquely successful on projects that had a large amount of earthwork, as mentioned earlier. Additional questions were formulated both on the risk and the opportunity side. These modifications provided more accuracy in predicting profitability on future projects with similar characteristics.

To summarize the results to date, the PRISM 1 Model has proven thus far to be a successful tool to assist management to:

·         · Make high level go/no go decisions on bidding new projects (See Figure 5)

·         Identify risks and opportunities

·         Quantify and prioritize risks and opportunities

·         Allocate risks to the party best able to handle the risk

·         Manage those risks which cannot be transferred

·         Document risk/opportunity events from past projects

·         Reduce the cost of resolving disputes

·         Place a limit on a firm’s financial exposure in the event of a claim, and

·         Archive decision making wisdom of top management for use by future generations

As modifications to the model are applied to future projects, time and experience will further validate the accuracy of the predictions of the model. In any event, the risk management process has been improved through the use of the model as a tool to help create prioritized lists of risks and opportunities to be mitigated. Rarely is there sufficient time and resources available to address all risks and opportunities; therefore, the process should be extremely helpful to management in prioritizing their risk management mitigation and control efforts. At least project managers and estimators will have invested their management time and resources on the right projects and in the risk areas that have the greatest return on investment.

As Pareto stipulated, 80% of the impact on an endeavor would be realized as a result of only 20% of the possible influences. (Quality) Therefore, if we can identify and mitigate the 20% most influential risk/opportunity events, we will have gone a long way toward influencing the ultimate success of the project.

Additional work and further refinement of the present PRISM 1^TM Model will continue in these areas. After all, this model was purposely designed to use the best practices and information presently available combined with the traditional time-tested tools of risk management. Why?...so that contemporary constructors might avoid becoming future statistics on Dun & Bradstreet's report of Construction Business Failures.

Ultimately, when all is said and done, a model is only as ‘used’ as it is perceived to be ‘useful’. Indeed, the authors recognize that many critical decisions are governed by some very simple yet powerful variables that can overshadow more significant ones. In interviews with over 200 contractors, the authors determined that there are certain variables (over 600 identified to date) that influence the contractor’s final decision as to how much profit shall be added to the bid. Among these, there is an overwhelming influence is exerted by the question: ‘How badly do we want or need this project?’ (Auchey) Indeed, this influence exerted a significant impact on why the potential profit of one of the projects was not accurately predicted by the PRISM l modeling and fell outside allowable variance. The contractor wanted to win this project so badly that many of the risks predicted were overshadowed by the need to win the bid. These variables are now being quantified and incorporated into the model on a custom contractor project basis in order to improve the accuracy of the PRISM^TM Model and the effectiveness of the process of procuring successful projects.

References

Aspinwall, Elaine M., Joanna C. Bennett, George A. Bohoris, and Richard C. Hall. , Risk Analysis Techniques and Their Application to Software Development. European Journal of Operational Research, 1996, February, 467-475.

Auchey, F. L. Survey: Variables Influencing Bid Strategies. Independent surveys conducted with over 600 Associated General Contractor (AGC) members in Florida, North Carolina, South Carolina in Virginia from 1986 through 1989.

Bajaj, D., D. Lenard, and J. Oluwoye. An Analysis of Contractors' Approaches to Risk Identification in New South Wales, Australia. Construction Management and Economics, 1997, October, 363-369.

Baker, Scott, David Ponniah, and Simon Smith. Risk Response Techniques Employed Currently for Major Projects. Construction Management and Economics. 1999, November, 205-213.

Bernstein, Peter L. Against The Odds: The Remarkable Story of Risk. 1998, New York, John Wiley & Sons, Inc.

Dawood, Nashwan. Estimating Project and Activity Duration: A Risk Management Approach Using Network Analysis. 1997, Division of Civil Engineering and Building, School of Science and Technology, The University of Teesside.

De la Garza, Jesus M. and Khalil G. Rouhana. Neural Networks Versus Parameter-Based Applications in Cost Estimating. Cost Engineering,1995, February, 14-18.

Dexter, William F. Risk Management: A Mine Field. Consulting-Specifying Engineer, 1999, May.

Diekman, James, Roger S. Pecosk, and Anthony D. Songer. Risk Analysis for Revenue Dependent Infrastructure Projects. Journal of Faculty of Civil, Environmental, and Architectural Engineering, University of Colorado. 1997, January, 377-382.

Duffield, Colin F., Aminah Fayek, and David M. Young. A Survey of Tendering Practices in the Australian Construction Industry. Engineering Management Journal, 1999, December,29-34.

Engineering News Record, April 7, 1997, Vol. 238, #14, 9.

Kerzner, Harold. Project Management. 1998, New York: John Wiley & Sons, Inc.

Kim, Soon. Risk Management in Construction: An Approach for Contractors in South Korea. Cost Engineering,2000, January, 38-44.

Leung, H.M., C.K. Mok, and V.M. Rao Tummala. Practices, Barriers, and Benefits of Risk Management Process in Building Services Cost Estimation. Journal of Department of Manufacturing Engineering, 1996, City University of Hong Kong, April, 161-175.

Quality for Project Managers. 1998, ESI International. 2.25.

Ranasinghe, Malik. Risk Management in the Insurance Industry: Insights for the Engineering Construction Industry. Journal of School of Building and Real Estate, 1997, National University of Singapore. February.

Render, Barry and Stair, Ralph M. Jr. Quantitative Analysis for Management. 2000, Seventh Edition, Prentice Hall.

Addresses of authors:

Dr. Flynn L. Auchey, 122 Burruss Hall, Department of Building Construction, Virginia Polytechnic Institute & State University, Blacksburg, VA, 24061-0156, USA

Dr. Gloria J. Auchey, PMP, President, The Success Institute of America, Inc., 1500 Midpines Road, Blacksburg, VA, 24060, USA