Bridging the Gap: Performance and Efficiency in Design Build Delivery

Introduction: Historical Procurement Environment

The City of Peoria is Arizona’s ninth largest city.  It covers nearly 178 square miles and is home to over 132,300 residents.  The projected 10 year growth of the City is estimated to be over 486,720, which will prompt the significant expansion of the City’s municipal services and facilities.  The City of Peoria has taken steps to enhance development through revitalization plans to provide for further development of its central downtown area to support new and existing businesses as well as economic growth (City of Peoria, 2004).

The City of Peoria’s Capital Improvement Program (CIP) for fiscal year 2006 is $188 million dollars.  An ongoing challenge with the CIP is the predictability that change orders will be received on virtually every project, in other words the initial cost (low bid) will not equal the final cost but it is nearly certain that the final cost will be greater than the initial cost (low bid).  The change orders experienced by the city are comprised of owner initiated changes, constructability issues and design professional initiated errors and omissions. 

Historically, in the low bid environment the performance received from the design professional and/or the construction contractor necessitated improvement. The City’s continued use of the low bid environment has perpetuated the problems experienced by the City. It is perceived by the City that the low bid environment has created a need for providers to assign their low performing design professionals and low performing contractors to city projects in order to maximize profits (inefficiency = increased profits). The assignment of resources experienced by the owner is typified by Figure 1 (Kashiwagi, 2004).

In Figure 1 it shows three separate types of customers, or clients, along with the resource pool of a contractor separated into three separate levels of training or experience.  Every contractor has employees and craftspeople with different degrees of training.  Some workers are very highly trained and experienced, some have been slightly trained, and some workers are very inexperienced or new hires.  To understand how contractors allocated these skilled employees, an example is appropriate.  Let us assume that a contractor has three different crews: highly trained crew, medium trained crew, and an inexperienced crew.  If three different owners approach the contractor for work, the contractor must decide which crew to send to each owner.  In an outsourcing situation, the contractor assumes the risk for construction non-performance.  Therefore, the contractor will assign their most skilled craftspeople to this owner in order to minimize their risk.  In a “Partnering” environment, the contractor and owner share the risk of construction non-performance. Therefore, the contractor will assign their medium skilled craftspeople.  In a “Price-Based” environment, the contractor has very little liability for construction non-performance.  The client provides specification and manages and inspects the job.  The risk of non-performance thereby stays with the client.  Therefore, the contractor assigns their most inexperienced craftspeople to the job, putting the owner at maximum risk. 

Performance Problems Under Low Bid

For many years the City operated under the low bid environment, and the problems of non-performance (delays, over-budget, and unsatisfied customers) increased. The City’s projects usually consist of design and construction elements that by state law, are required to be outsourced. Under the low bid environment, the selection and matching of a high performing designer and a high performing contractor was perceived by the materials management group to be a “crap shoot.”  Over the years, the continued use of the low bid environment lead to greater owner dissatisfaction mainly due to increased construction costs, increased design costs, lower quality of construction, longer build times, increased change order disputes, requirements for intense day-to-day management and in some cases litigation.

The change order cost to the owner has run in double-digit percentages over the initial project cost (low bid). The following table contains representative examples of the cost overruns experience by the City of Peoria when using the low bid selection process (DBB is an acronym for design-bid-build).

Table 1

Cost Overruns on Low Bid Projects at the City of Peoria

Furthermore, exploration into the change order issue showed that the majority of the changes experienced were theoretically avoidable.  On the average construction project the reasons for change order can be segmented as shown in Figure 2.

As is shown in Figure 2, 83 percent of the reasons for change orders come from additions, design problems, and deletions.  All of these issues are avoidable and have been primarily experienced by the City in the low-bid environment.  Vendor non-performance was amplified by the experienced increase in costs from changes and change management.  Overall for the City, approximately 20-25 percent of low-bid projects were completed on time, 75 percent had cost generated change orders, and the overall customer satisfaction was 20 percent (Koebergen and Zenko, 2005).     

Taking the 2006 CIP budget of 188 million dollars and adding in the average change order rate of 14%, several questions arise from past experiences:

1. How does one mange the increase in costs beyond the allocated budget?

2. Does one increase the budget dollars from $188 million to over $200 million, or build less with the knowledge that it will on average be 14 percent inefficient?

3. Do you cut projects? If so, what projects do you cut?

4.  Could the problem be the low bid delivery method?

A Change in Delivery Method to Minimize Non-Performance

Prior to August 2000, the only selection method for construction under Arizona Revised Statute, Title 34 was a price based (low bid) process. As previously described and is realized in industry, there are some significant challenges to the low bid environment. However, this would soon change for the City through revision of the state laws.

With the changes to the Arizona Revised Statutes (ARS) Title 34 in August of 2000, an owner was now allowed to use alternative delivery methods including Design/Build, Construction Manager at Risk, and Job Order Contracting. For an owner seeking construction services, there were now opportunities for a qualification based selection process.  The City of Peoria immediately began researching and implementing the use of alternative delivery methods. Over the next few years the design/build process was used on three projects and the qualification based selection process increased the odds of meeting customer expectations and the project performance (on time, on budget) expectations of the City.

Although the delivery system improved the process, it did not reasonably predict an outcome that achieved the owner’s expectations of on time, on budget and acceptable quality. The process and outcomes were improved through increased collaboration, reduced change orders, and was closer to meeting the budget. However, for the three Design/Build projects the closeout process and schedules did not meet agreed upon expectations. The process still required extensive management resources by the owner and in some cases, the use of a third party construction management team. It also became apparent that the driving factors were relationship oriented rather than the desired performance orientation.  The initial alternative delivery method projects, under design/build, did not resolve the owner’s risk of non-performing teams and did not achieve an acceptable level of owner satisfaction.

Discussions in the City began around the question that, if alternative delivery methods leave a gap in the issue of performance, how do owners achieve full performance? It was evident that there was a missing piece to the puzzle for predicting and ensuring vendor performance. The City’s search for a solution to the issue was not achieved by alternative delivery methods.

Research Concept

The fundamental concept of the research was that the non-performance experienced by the City in design and construction services was not directly due to project delivery systems, but instead to the selection process of vendors, and specifically the design-build teams.  Unless an owner uses the concepts of effective outsourcing (minimizing risk), and the performance risk is transferred to the design-build team, the use of a qualification only selection process will only increase the odds of a successful project, and cannot predict a successful outcome of quality, finishing on time and completing on budget.  The process with the highest level of predictability for a project to meet the owner’s expectations is one that is performance based and that transfers risk from the owner to the design/build team.

The research process required the City to identify the optimal procurement environment along with a selection process or tool that augmented the owner’s ability to identify a high performing vendor and successfully transfer risk to service provider.  Once identified the process must be implemented within the established rules and regulations, while meeting the best interests of the client.  The components of the research methodology are given below along with initial test implementation of the best value process in design-build delivery.

The Optimal Procurement Environment

An optimal procurement environment for any city would consist of a selection process that can increase the predictability of a favorable outcome meeting the owner’s expectations for projects that are on time, on budget and of the desired quality. The selection process should meet the following criteria:

The process must also outsource the risk of non-performance to the service providing vendor and hold accountable all parties involved.  The City’s situation could be likened to one knowing their final destination but lacking the knowledge of the path to get there.  

Strategize to Effect the Optimal Environment

The City of Peoria began an extensive search to identify a procurement methodology or tool that could be utilized within the criteria set as an optimal environment.  After continued search the City found a process that looked promising to bridge the gap in performance that had been experienced. The process is know as the Performance Information Procurement System (PIPS) (and as a political bonus was developed locally at the Performance Based Studies and Research Group at Arizona State University).

The PIPS process consists of several stages that identify a potential best value vendor and allows the transfer of performance risk to a vendor.  The stages in the PIPS process are (Figure 3 displays the process in terms of its key filters): 

1. Identification of past performance.  Past performance includes frequency of on time completion, minimal change orders, and high customer satisfaction of critical project performance elements (design firm, lead architect, critical subconsultants, the general contractor, project manager, site superintendent, and critical subcontractors (commonly electrical and mechanical for general construction)).

2. Project specific capability.  This is defined as the capability to identify, prioritize, and minimize the risk of the project in the non-technical terms of cost, time, and quality expectation.  This is achieved through a risk assessment plan, along with a bid price, and an interview of the key project individuals.  What is written and stated during these steps becomes part of the vendor’s contract upon their being awarded the project.

3. Competition based on performance (past performance and ability to minimize risk) and price.  Prioritization of bidders is done using a multi-criteria decision making model, which minimizes risk of nonperformance by giving credit to the identified critical past performance elements (recorded values of filters one and two).  This model does not penalize values which are near the mode, but penalizes values that are below the mode.  The processing of values forces vendors to provide their best value, and compete with every other best value, resulting in a two step best value process.

4. Pre-award phase.  The best value vendor (as identified by the multi-criteria decision making model) must minimize the risks identified by all competitors.  They must coordinate the requirements between critical elements, clarify or seek clarification on the project.  The contractor will then sign a contract that includes their risk minimization plan, the intent of the owner, and all clarifications.

5. Design and Construction.  The vendor is forced to manage the project in terms of risk.  The vendor passes risk information (affecting cost, time, and quality expectation) to the client’s representative.

6. Measurement of performance.  The project will be rated after completion.  All critical elements of the design-build team receive the same performance rating.  The rating becomes up to 50% of the critical element’s future performance rating.  This assigns accountability to the vendor by impacting the firm’s capability to procure future work with the client.

The process fosters an environment where vendors are allowed to complete based on performance, not just price and where performance risk is transferred from the client to the vendor.  The transfer of risk requires the client to minimize inspection and management functions and allows the vendor to maximize profits under a system that is made efficient through the vendor’s expertise.

Implementation of the Best Value Process

Figure 4 illustrates the Arizona Revised Statute, Title 34 Alternative Project Delivery Selection Process.  The process fundamentally is comprised of two main components: the Request For Qualification (RFQ) and the Request For Proposal (RFP).  The revised state statute, which allows agencies to use alternative project delivery methods, also permits agencies to select vendors based on performance and price.  Although the client may review performance data, the client is not permitted to “request or consider fees, price, man-hours or any other cost information at any point in the selection process (ARS 34-603-C-8)” during the RFQ stage.  Price can only be considered during the RFP stage, and on a design-build project, the client must pay a stipend to all unsuccessful offers (ARS 34-603-F-11).  The minimum fee is 0.2 percent of the total budget (design and construction) and on larger projects the fees can be significant (i.e. if the budget was $15 Million, and there were two to four unsuccessful bidders, the client would have to pay each one $30,000).  The purpose of the fee is to cover the vendor expenses to prepare a detailed technical proposal; however, by going through an extensive RFP stage significant time is added to the total project delivery cycle, which can become cumbersome on many job types.  The statute allows award after the RFQ stage but that award cannot consider the prices of the vendors. 

The City of Peoria Implementation of Best Value in Design-Build

The City of Peoria tested and implemented the PIPS process on 11 projects of various types, four of which were procured under design-build delivery.  Table 2 shows the four projects.

Table 2

Best Value Design Build Projects at the City of Peoria

Using the revised statutes, the City used the Request For Qualification process to identify the best-valued vendor for the four projects listed above.  This was accomplished by following the process below:

1. A Request For Qualifications (RFQ) is prepared, which includes all of the performance requirements that will be evaluated.

2. The RFQ is advertised.

3. The project is open to all vendors regardless of their experience.

4. Vendors are invited to an educational meeting.  During the 1.5 hour meeting, a presentation is given to review the PIPS process and required submittals.

5.  In response to the RFQ, vendors submit past performance information (PPI) on the firm and critical team individuals.

6. Vendors submit a Risk Assessment (RA) Plan and a Proposal form.

7. The City of Peoria then shortlists 3-5 vendors based on PPI and RA Plan scores.

8. The individuals of the shortlisted firms are interviewed, first individually and then as a group.

9. The potential best-valued firm is identified based on the PPI, RA Plan score, and interview score.

10. The potential best-valued firm conducts a pre-award meeting where the firm prepares a risk minimization plan and a quality control plan containing a detailed project schedule, quality assurance techniques, and .

11. The City of Peoria awards and negotiates the fees.

In the selection, because only the RFQ stage is used, the City considers only performance as State laws do not allow price as a consideration in the determination of the best value vendor.  The only price requirement is that the work be able to be performed within the established budget, which is given to all of the bidders in the RFQ.  If the budget is an issue, the vendor must identify it as a risk in the RA plan and offer solutions to eliminate any potential cost overruns.  

Of the initial testing of the best value process in a design-build delivery, only one project has been completed thus far.  This project, the design and construction of an 11.6 acre park, was completed on budget, and experienced no vendor caused delays (there was an internal City permitting issue that caused a delay to the completion of the work).  The approximate initial client satisfaction with the project is a 9.5 out of 10; a marked improvement over the low bid and previous design-build projects.

Conclusion

Movement from the low bid delivery method to design-build delivery method did not resolve the owner’s risk of non-performing teams and did not achieve an acceptable level of owner satisfaction.  It is hypothesized that performance based procurement, which transfers the risk of non-performance to the vendor, must be incorporated into a delivery process to realize predictable and acceptable design-build team performance and a successful outcome for all parties (win-win). 

The City of Peoria has begun the testing and implementation of a best value selection process on several projects to improve performance under design-build delivery.  Four design-build projects have been procured using a best value selection process and initial results have indicated that the design-build method, coupled with the PIPS process can result in the selection of high performing vendors who maximize profits through project efficiencies and risk minimization.  The specific objective of using the PIPS process with the design-build delivery method is to reduce or eliminate the amount of change orders (on budget), maintaining or improving the project time schedule (on time), increased construction quality, and reduced owner resource requirements (higher owner satisfaction) compared to the previous low bid process.  The research will move forward and compare change order information, schedule compliance, budget compliance, and owner satisfaction for multiple design-build, best value projects to the traditional low bid delivery process.

References

City of Peoria. (2004). City of Peoria General Plan.  URL www.peoriaaz.com/genplan

Kashiwagi, DT (2004) Best Value Procurement: How to use Information System to Minimize Risk, Increase Performance, and Predict Future Success. Performance Based Studies Research Group: ASU.

Koebergen, H. and Zenko, D. (2005). Personal communication, October 24, 2005

Savicky, J. (2005). How to Increase a Vendor’s Ability to Identify and Minimize Risk. Masters Thesis, Arizona State University. Tempe, Arizona., USA.

Sullivan, K. T. (2004). Quantification of the Cumulative Impact of Change Orders on Sheet Metal Labor Productivity. Doctoral dissertation, University of Wisconsin-Madison.