The low-bid, design-bid-build construction delivery system has been the standard delivery system for the last 30 years. In the last ten years, construction nonperformance problems with the low-bid system has encouraged owners to move to alternate delivery systems such as indefinite delivery, indefinite quantity (IDIQ), design-build (DB), and construction management at risk (CMAR). However, very few of the owners have documented performance information, which identifies that the implementations of alternate delivery systems were successful.

The State of Hawaii Department of Accounting and General Services Public Works Division (DAGS/PWD) implemented the PIPS (developed at Arizona State University in 1991) in 1998. PIPS is different from other best value procurements in the following ways:

1. Self documenting.

2. PIPS measures the risk of information in terms of number of references, number of years in business, abnormalities, and perception of the owner.

3. Past performance on projects directly impacts the competitive value of contractors on future projects.

4. An artificial intelligent processor prioritizes contractors based on performance and price.

5. Contractors are given total control of the project.

6. Contractors and manufacturers have a performance barcode that is permanent.

7. Standards are minimized.

8. Specified materials, means and methods are minimized.

PIPS includes the following steps:

1. Pre-qualification or registration of performance line.

2. Request for Proposal is sent to pre-qualified contractors.

3. Pre-bid meeting.

4. Submittal of bid price and management plan (identification and minimization of risk of not being on time, on budget, and meeting quality expectations).

5. Rating of management plans and interview of key personnel.

6. Prioritization by AI model based on past performance, management plan and interview rating and then performance and price.

7. Pre-award period and meeting where the top prioritized contractor gets clarification, coordinates all drawings and specifications, and verifies that they will construct the project.

8. Award.

9. Construction.

10. Rating of construction. The project rating counts 25% of the revised contractor performance rating.

PIPS has been run on 150 projects by DAGS/PWD:

1. Roofing: 97 projects

2. Waterproofing: 2 projects

3. Painting: 34 projects

4. Mechanical replacement: 11 projects

5. Lighting: 5 projects

6. General construction, fire alarm replacement: 1 project

7. General construction, barrier removal (ADA): 1 project

The cost of training and implementation support for PIPS has been $300K over four years (1998-2001). The total cost of the construction procured by DAGS/PWD through PIPS has been $12.2M. After considering the savings in delivery cost from the minimization of design, the actual cost of implementation is negligible.

PIPS was also tested at the Hawaii Department of Transportation on two general construction projects and implemented at the University of Hawaii on 20 roofing and painting projects. The State of Hawaii DAGS/PWD was declared the winner of the 2001 Innovation in Technology PONO competition in the State of Hawaii for the implementation of PIPS. Currently 301 contractors have past performance information and are listed on the Internet. PIPS is being used to deliver over $100M of construction in less than 10 months in 2002.

The analysis of the value of construction (price and performance) has been documented in three ways:

1. Comparison of roofing prices between low-bid and PIPS.

2. Comparison of painting prices between budget and PIPS.

3. Comparison of mechanical renovation prices vs. the budget.

Budgets are set based on previous years projects (low-bid awarded.) When the user makes new requirements or change in scope, they are not always reflected with an increase in the budget. The roofing and painting performance has been poor in the last ten years. DAGS/PWD low-bid awards have resulted in the following:

1. A number of bankruptcies among roofing contractors servicing DAGS/PWD.

2. Adversarial relationships between DAGS/PWD inspectors and roofing and painting contractors. Inspectors believed that all contractors were the same, were not to be trusted, and did non-performing work.

3. Roofing problems that DAGS was forced to repair themselves due to non-action by the roofing manufacturers and contractors.

4. Roofing warranty periods (material and labor) of only two years.

5. Painting was done by DAGS "in-house" staff. Painting contractors were identified as incompetent and noncompetitive.

DAGS/PWD selected roofing and painting as the first two areas to test PIPS. Their immediate conclusions on PIPS based on observation included:

1. One DAGS/PWD project manager could do the work of five.

2. Inspectors’ work was drastically reduced with PIPS projects.

3. Design scope and costs were reduced using PIPS.

DAGS/PWD then began to target other areas including waterproofing, mechanical, electrical, and general construction for PIPS implementation.

DAGS/PWD identified 130 low-bid award projects over the past five years, and 97 PIPS projects. Analysis of the performance included the following results:

A benefit of the PIPS program was that different manufacturers with different roofing systems were allowed to competitively bid under the PIPS. The results were that the PIPS maximized the value of the roofing systems by selecting the best value based on price and performance, a decision that designers were previously forced to subjectively decide.

In December 2000, in anticipation of the implementation of energy conservation codes, DAGS/PWD included an R-19 insulation requirement for retrofit reproofing. PIPS roofing prices seemed to increase above the budget. An analysis was performed to identify if the prices were reasonable. The prices of both the insulation and non-insulated roofing jobs were compared in Table 1.

Table 1 and 2 is for all roofs, and Table 3 is for Built Up Roofs (BUR) and modified bitumen (MB) roofs. The BUR/MB was broken out due to perceptions that the State paid too much for these systems. The following is an analysis of the roofing prices:

1. Fifty six (56) percent of all the insulated roofs were over 120% of the budget.

2. When analyzing the budget, the budget was about the same for the non-insulated and insulated roof decks. However, the contractors were charging $1.53 more for insulated roof systems. This is consistent in terms of extra material and almost double the construction time.

3. When analyzing the budget on different size of roofs (Table 2), the larger roofs had about the same cost as the smaller roofs. However, the contractors are charging much more for smaller roofs, and less for the larger roofs.

4. It is clear that the budgets were not set correctly and the contractors are competitively bidding. The use of information will assist the designers more accurately assess their budgets and project cost estimates in the future.

5. Table 3 shows the numbers of the built up roofs and modified bitumen roofs. The prices are slightly higher.

6. PIPS allowed other options: coating only and coating and sprayed polyurethane foam roof systems. When these systems were installed without removing the existing waterproofing system, the manufacturers were required to warranty all the existing systems down to the structural roof deck.

The conclusion of the State of Hawaii roofing PIPS procurement include the following:

1. Roof problems related to design, leaking, and customer dissatisfaction have disappeared over the past three years.

2. Roof warranties have been increased to 10 and 15 years (5 to 7.5 times the previous periods.) Considering the previous two-year warranties, the roofs have passed those time requirements with no problems.

3. Prices have risen 5% and have remained within the budget. There are no change orders.

4. Procurement documents have been reduced to state legal paperwork, a moisture survey, a core sample of the existing waterproofing material, and special conditions. The contractors’ submittal has been reduced to a two page management plan. Roofing specifications directing material and means and methods have been eliminated.

5. Customers have been pleased with contractor performance (on-time, on-budget, meeting quality expectations.) Contractors have communicated well with the owners, have reduced installation times to bare minimums – sometimes as short as a weekend. The average performance rating is 9.5 out of a possible 10, there have been no contractor generated cost change orders, and 99% of the roofs have been installed on time. There have been no leaking roofs in the last three years.

6. One DAGS/PWD project manager uses 30% of her time to coordinates the majority of roofing projects, an increase in production of 500%.

7. Two of the four contractors that were awarded projects increased their performance rating. Two contractors stayed about the same. PIPS was successful in encouraging the contractors to increase their performance level. One of the roofing manufacturers representative stated that the performance level of the contractors has increased tremendously in terms of minimizing the manufacturer’s risk in warranty claims (Alexander, 2001.)

The second major area DAGS/PWD testing using PIPS was painting existing buildings. Painting is very difficult to specify and inspect due to:

1. The work to prepare the existing surface is the most difficult and important stage of work.

2. Performance has been very difficult to define. It includes aesthetics and longevity, two factors that are dependent on many other factors.

3. The impact of low-bid on painting is difficult to identify, resulting in the minimization of the value of specifications.

4. Damage caused by over spray, resulting in the banning of spray painting.

5. Difficulty in identifying if the painters were using the amount of paint due to the lack of consistent measuring methods, contractors skimping on paint, and the thinning of paint.

6. The award based on price has not been successful in procuring performing painting, resulting in DAGS doing painting work using in-house work crews.

The first set of painting projects (buildings at 12 schools) was completed in 1999. To measure the quality and performance of the painting, DAGS ensured that at each site, there was an in-house project and a PIPS procured projects side by side. The following were the results:

1. The PIPS projects were much higher performing than the in-house projects. The major differences included detail and prep work.

2. All the projects were completed within budget, on time, and met the quality expectation of the users.

3. Problems of blistering due to wet walls were solved in a logical procedure. Painters proposed to watch the painted surfaces with blisters over a period of time, and repair if the blistering was minimal. If the problem persisted on a large scale, the contractors would propose a solution where both DAGS and the contractors would partner to solve the problem.

4. The painting industry took a proactive position to ensure that the painters performed, assisting in the inspection, training, and coordination of the projects. This resulted in minimizing the role of government inspectors.

The second set of painting projects (building at 16 schools) was completed in 2000. The results were similar to the first set of projects. A third set of painting projects was completed in 2001 in more remote areas of the State by contractors who did not have as much training, education, and performance of the contractors who had done the first two sets of projects. The results of the third set of projects showed no appreciable differential from the first two sets of projects.

The results of the painting projects include the following:

1. Percent of projects within budget: 52%.

2. Average price in comparison to the budget: 98% of budget.

3. Average rating of painters: 1999 9.04, 2000: 9.6, 2001: 9.99 (preliminary only)

4. Percent of projects that were on-time: 100%

5. Number of performing painting contractors who bid on the projects: 16

6. Number of contractors who were awarded projects: 10

PIPS had delivered the State with:

1. Performance painting within budget, on time, and meeting quality expectations.

2. Reduced management by the government agency.

3. Increased education, training, and regulation of performance by the industry.

4. Environment of increasing performance.

After successfully testing PIPS on roofing and painting projects, DAGS/PWD identified another area of nonperformance, the replacement and renovation of mechanical equipment. There were designers, project managers, and inspectors who felt that PIPS would not work in the mechanical areas due to the complexity of the work. Mechanical equipment replacement was causing DAGS/PWD the following problems (Miwa, 2001):

1. Equipment did not perform over time.

2. Designers were getting their solutions from manufacturers. The solutions were then given to the user, who used the solution to procure construction services. This diluted the responsibility and liability of nonperformance.

3. Designers, contractors, and manufacturers would accuse each other of nonperformance, leading to an unsolvable problem.

4. Maintenance personnel were not satisfied with the services procured.

PIPS was used on the procurement of ten renovation projects in 2000. Instead of ten designers, only one designer was used. The same DAGS/PWD project manager, who was responsible for the roofing and painting projects, managed the mechanical projects. The following were the results of the projects:

1. Price of contracts compared to the budgets (set by low-bid projects): 5% below the budget

2. Number of projects within the budget: 66% of the projects were within budget, and the overall cost of the projects was within the aggregate budget

3. Number of contractors who had performance lines generated on past references: 31

4. Number of contractors who bid: 9

5. Number of contractors who were awarded projects: 3

6. Average rating of work: 9.83 out of 10.0

7. Number of projects that were on-time, on-budget, and met quality expectations: 11 (100%)

8. Change in contractors’ performance lines: past performance average: 9.31, revised average based on work done: 9.42 and percent increase in performance level: 1%

The conclusion of the mechanical PIPS projects was that the complexity of mechanical system renovation did not impact the results. As with the other systems:

1. Design was minimized.

2. Performance was very high.

3. The contractors assisted in using the budget to produce the highest quality systems.

Based on the preliminary results of the ten tests, DAGS/PWD is continuing to issue mechanical replacement work using PIPS. The results were similar to results from the roofing, painting, and waterproofing projects.

Not everyone greeted the implementation of PIPS or the successful results with open arms. Some contractors and manufacturers who had competed work in roofing and painting under the ‘low-bid’ process, were having a difficult time competing under the new performance requirements. The situation became even more competitive, when contractors who would not bid under the low-bid system with the State, entered the competition under performance based. The high level of performance also impacted the in-house State project managers and inspectors, who were forced to change the way they did business. Due to a lack of understanding, many in-house project managers and inspectors had difficulty in changing. Project managers did more work than was required due to their experience in the low-bid environment. Another major problem was the speeding up of the implementation. The original plan was to use PIPS in conjunction with the low-bid delivery process. The process was to take five years. However, after the Director of DAGS/PWD saw the effectiveness of PIPS, he accelerated the implementation. This is not recommended. This resulted in the lack of documentation of performance information, which is critical in a performance environment. The following is a list of lessons learned from the State of Hawaii tests:

The PIPS implementation in Hawaii has been successful in the painting, roofing, waterproofing, and mechanical construction projects. The performance based contracting system has increased the level of performance (99% on-time, on-budget, meeting quality expectations (9.75 average performance rating)) of the contractors at a very small increase in first costs. The process assists the government to reduce manpower requirements, increase performance (warranty period, performance, faster delivery) pass the risk of nonperformance to the contractors who can minimize the risk through expertise, and minimize design requirements. The PIPS system also improves the efficiency of the government managers. It has been difficult for the State of Hawaii to change to the process. The level of performance was raised because high performance contractors were using their best craftspeople well. However, many of the government project managers and inspectors who were used to the adversarial, control oriented, bureaucratic low-bid process have been very resistant to change and the performance information environment. Education, documentation, and process rules are the key for government implementation of performance contracting.

Alexander, Mark (2001, January 17) Conversation at the University of Hawaii-Monoa Registry Meeting.

Kashiwagi, D.T. (2001, November 7) Getting it right! Selecting the Right Contractor for the Right Job. Nashville: Project Management Institute (PMI) Conference.

Kashiwagi, Dean T. & Mayo, Richard E. (2001, April) State of Hawaii Selects ‘Best Value’ by Artificial Intelligence" Cost Engineering, 43, (4), 38-44.

Kashiwagi, D.T., (2001) Information Measurement Theory, 4^th Edition. Tempe: PBSRG.

Kashiwagi, D.T. & Mayo, R.E. (2001) Best Value Procurement in Construction using Artificial Intelligence, Journal of Construction Procurement, 7, (2), 42-59.

Miwa, Steve (2001) State of Hawaii, statement by DAGS/PWD, PIPS Administrator.

Zeleny, Milan (1982) Multiple Criteria Decision Making, New York: McGraw-Hill.