Ranking Construction Faculty

“The Key to Continuous Faculty Improvement”

Introduction

The purpose of this paper is to propose methodologies for the ranking of construction faculty members (This paper uses the term “construction programs” to include all programs of construction higher education, including Construction Management, Construction Science, Building Construction, Construction Engineering, etc.).  The authors realize this is a very controversial topic and anticipate that this paper will be a vehicle to initiate and conduct many academic debates.  The Associated Schools of Construction (ASC) may be the forum to have these debates on faculty rankings. 

Literature Search

World Class

Some contend (Badger & Smith 2004) that the true measure of any construction program is the quality of the individual faculty members.  World class instruction is not singularly driven by curriculum, but the quality of the individual faculty member.  Great faculty members have outstanding classes and their students become outstanding alumni.  The challenge is that the faculty members do not generally want to be evaluated and compared with their peers or peer programs.  There are many metrics--student teaching evaluations, the chair’s exit interviews, publication records, research dollar expenditures, and results of the CPC exams assessments--which could be used to rank faculty members, but few programs have the desire, capability, or data to do this type of evaluation and ranking, nor can they stand the heat if they do.

Prestige and Credibility

The concept of prestige was presented and recommended as a broader evaluation concept.  Prestige is defined in Webster’s Dictionary as “reputation or distinction based on brilliance of achievement, character, etc.”  Sevier (2003) references, in a university context, a related concept termed “reputational capital,” which is a form of intangible organizational wealth that he considers a precursor to prestige.  Prestige can be built, but it must be maintained with the continual infusion of resources.  Sevier notes that to achieve prestige, the following elements generally must exist: well-demonstrated quality, selectiveness, world community membership, ability to open doors, and an old yet innovative environment. The issue of credibility in the eyes of rated recipients, peers, and the general public was considered as an important aspect that must be integrated into every evaluation.  Credibility is directly related to the reputation of the sponsor, the impartiality of the nomination and selection process, and the nature of the metrics measuring the process.

Higher Education

These important common elements are discussed by Boyer (1990), who states that higher education has four components.  These are:

He argues that all four of these activities are legitimate forms of scholarship that each is needed to fill the needs of our society, and that each should be recognized and rewarded.  Generally, construction faculty members tend to agree, though their perceptions of the weight and importance of each component varies widely. 

Teaching

According to one study (Christensen & Rogers 1992), the majority of construction faculty members prefer teaching over conducting research, service is not viewed as a significant criterion in the evaluation process, and there is a broad discrepancy between the perceived evaluation weight given to research and the percentage of faculty time spent on research.

PhD Degrees

During a visit to this country by Dr. James Summerville (2001) of Glasgow’s Caledonian University UK, he stated that it was evident to him that the trend toward requiring faculty to have PhDs and to be engaged in construction research was happening in both the United States and Great Britain. 

Faculty Turnover

The faculty members who fill construction programs reflect both the strengths and weaknesses of the current system.  They suffer from many of the same problems that every academic department suffers from but also have a relatively unique set of difficulties.  According to one study (Tufte & Hannestad 1988), they found that construction faculty turnover is an ongoing problem at many universities.  They cite the following as possible reasons for faculty turnover: 1) Difficulty in finding new faculty hires 2) Alternative opportunities 3) Economic considerations 4) Opportunities for advancement 5) Excessive workload and 6) Morale.

Tenure and Promotion

Ciesielski (1997) investigated tenure policies and criteria and promotion considerations for full professors in construction and construction engineering programs.  Principally, research scholarship holds a more prominent place in construction engineering programs where research is much more important when making tenure and promotion decisions. Conversely, teaching holds a more important place in construction programs and is more important than research in making tenure decisions.  Service is ranked last in both construction and construction engineering programs.

Faculty Competence

According to Waller (1997), competence refers to some observable evidence of performance by individuals.  Competency can be defined as having the minimum knowledge and skill to satisfactorily perform, and the automobile 'driving test' is a frequently used analogy. The Walker paper describes a competence model for project managers that attempts to include not only knowledge and skill, but further expand the model to include intellectual and moral behaviors in concert with the project manager's style.  The authors feel that construction education has to move to a competency model for faculty members.

Composure and Team Leadership

Additionally (Dainty 2004), the role of competency-based performance management is growing in significance in many industries and sectors. Unlike functional competencies, which measure performance against predetermined minimum occupational standards, competency-based systems are founded on the key behavioral competencies that underlie superior levels of performance.  In order to identify the key behaviors leading to performance excellence amongst construction project managers, in-depth behavioral event interviews were used in which managers were asked to recount critical management incidents, decisions, and actions from which their behavioral competencies could be identified.  The research identifies 12 core behavioral competencies that underpin effective project management performance, of which, two – 'composure' and 'team leadership' – were the most predictive.

Coaching and Development

As written by Bigelow (2003) putting the right project manager on the right job is what competency assessment is all about!  Competency is a buzzword in the new millennium ... but what does it mean?    Why would an organization want to evaluate a project manager’s competency?  Projects are only as successful as the people who manage them.  Evaluating project manager competency enables organizations to identify individuals who are, or have the potential to become, superior project managers and determine what is needed in the way of coaching and development to raise performance.  Faculty members need coaching and development, and all programs need some type of mentoring system.

Current Faculty Ranking Systems

Student Course Evaluations

At many universities, student evaluation of instructors is conducted each semester for each course in the program.  These evaluations typically [1] describe how well the course is developed and implemented in a management context, [2] assess the instructor’s performance style, and [3] solicit student feedback on what they liked best or least about the course and the instructor.

What is interesting is that the evaluation tool may not address the people skill attributes of the professor, the level of respect that the instructor has for the students, the relationship between the student and professor, the role of the professor being a mentor, role model, or adviser, and the ability to lead and influence the class.

Much has been written about student course evaluations, but most systems fail to consider the following important areas:

1. Demonstrated classroom leadership

2. Continuous and sustained people skills and positive relationship with students

3. Organized, well prepared, and rehearsed

4. Entertaining and interesting

5. Ego management

6. Enthusiasm and energy

7. Self-improvement inclination

8. Continuous learning for the instructor is required

9. Instructor reputation from previous classes

10. Continuously improving the application of high-tech classroom operations, e.g., Blackboard, email notification, on-line exams, electronic grade book, Power Point presentations on line for the students, and electronic handling of homework.

Faculty members are constantly measured by the students in each presentation, grading exercise, and mentoring session.  It is a highly visible position and results are measured semi-annually.  There are opportunities to listen, collect feedback, and improve.  Most student evaluation systems measure faculty on a five-point scale.

Student Exit Surveys

Many programs have a formal exit survey for graduating seniors which has a faculty evaluation component.  This is felt to be a good assessment instrument because students are ranking all faculty, and the students have had the opportunity to see them in action, not only in the classroom, but also in their capacities as advisors, counselors, and service participants.  Faculty may be ranked on a three point scale, e.g., “very effective, effective, or not effective.”  The authors suspect that these comments and rankings rarely find their way into formal faculty evaluations.

Annual Faculty Evaluations

In many universities the faculty are ranked or reviewed annually. The process may include self-evaluations by a faculty member, an independent ranking by the faculty member’s program leader, and a set of goals for the future.  Usually these components are reviewed and reconciled in a face-to-face session between faculty member and the program leader.  A percent weighting factor is agreed upon by the program leader and the faculty member.  The weighting factors might be 40 percent teaching, 40 percent research, and 20 percent service in a program with strong research expectations.  If a faculty was predominantly a teaching faculty then the percentages might be adjusted to 60 percent teaching, 20 percent research, and 20 percent service.  The program leader would collect comparable data in student teaching evaluations, the faculty's annual research expenditures, the annual publication record, and comments from graduating seniors’ exit interviews.  The program leader and the faculty would review the information and come to agreement on the scoring in each of the three categories.  This score would be added and averaged to give a total point score like 3.75 on a 4.0 system.  When all the faculty were rated the program leader listed the individual faculty member’s scores to determine a ranking in the program.  Annual raises were distributed according to the faculty member’s performance rating.  Some programs use a third component of peer faculty reviews or personnel committee reviews.

Faculty members’ contribution to service seems to vary significantly among universities.   In the university environment, a faculty member may contribute to the department, the college, and the university, including in the form of serving on committees.  In the service to professional societies, the contributions are usually service as members, as a committee chair, or in a leadership position.  In the service to the construction industry profession, the contributions may be in terms of continuing education, volunteer community service, and speaking engagements.  It is difficult to score contributions to the industry from the faculty members in a consulting role.

Promotion and Tenure

The criteria for promotion and tenure vary greatly among universities.  If the university is a teaching institution, then the emphasis will be on teaching.  If the institution has a research focus, then research will dominate.  There is a struggle between universities and the construction industry between a teaching emphasis and a research emphasis.  The industry does not relate well to a construction program investing 50 percent of their effort in creating new knowledge and publishing scholarly work.  The deliverable most industry professionals want is highly qualified graduates to enter construction as a professional.  Program and emphasis are greatly influenced by the college where the construction program is housed.  Engineering colleges seem to emphasize research more and teaching less.  Architectural colleges may be more balanced in their demands.  Technology colleges seem to promote the teaching element.

The construction faculty selected in the hiring pipeline produces new faculty who become teachers fairly quickly but researchers with more difficulty.  As more construction faculty members enter the academic profession with a PhD, a research degree, the academic discipline of construction will swing more to creating new knowledge than it does today.

Outside Peer Reviews

Outside evaluations in the form of letters from faculty in peer programs are usually required for the promotion and tenure packet of construction faculty.  These letters are critical in helping construction faculty be tenured and promoted in universities with a high research focus.  Usually construction faculty members excel in teaching and service to the industry but are behind their peers within the university in research.  A contributing factor is that the academic discipline of construction is considered to be immature within many university environments.  One of the major challenges is to improve the image of the academic disciplines of construction. 

Proposed Ranking System

Guiding Principles

The authors would like to propose the following ten guiding principles needed in establishing a faculty ranking or profiling system. Fundamental to this system is the premise that the overriding goal of the system is to promote the professional growth of the faculty member and prepare the faculty member for increasing leadership responsibility.

1.        The ranking system needs to clearly identify and define the target group to be ranked.  (One size does not fit all)

2.        All ranking systems should be open.  The information and data used should be made  available to all stakeholders. (Complete openness)

3.        The ranking system results should be simple, understandable, and reproducible. (The simpler the better)

4.        The faculty being ranked should be included in the discussion of selecting performance data and corresponding metrics. (Practice inclusiveness)

5.        Ranking systems that include self-evaluations are usually more credible. (The power of self-evaluations)

6.        New ranking systems should be prototyped, tested, and fine-tuned before implementing. (Prototyping, fine-tuning)

7.        The target group being ranked needs to develop the system, or some third-party will develop it for them. (Be the driver of the system)

8.        The information and data used must be verifiable. (Verify)

9.       Rating systems need a data pool before norms and/or averages can be established. (Benchmark)

10.     Good ranking systems provide data and information that can be used to improve the image of the academic discipline of construction.  (Image improvement)

Verification

In any ranking system there has to be an agreed-upon verification mechanism. 

1.        Self-evaluations help set the bar when those evaluations are aligned with metric data from other sources proved to be solid.

2.        Academic programs should have a mechanism to collect student evaluations, research performance data, and annual publication records that are available.

3.        Some programs use peer evaluations to provide third-party checks and balances.

4.        Another technique is to have the personnel committee composed of senior full professors to be the third-party input.

5.        In some programs, the ranked and/or profiled faculty member has a face-to-face meeting with the program leadership and signs the ratings documents. 

6.        It is critical to keep all evaluations open and freely discussed with the faculty being rated.

7.        Copies of all evaluations should be made available to those being evaluated.

Faculty Scorecard

Purpose

The proposed faculty scorecard [see Appendix A] could be used as a component of the Annual Review process.  The scorecard might be completed as a self-evaluation by the faculty member, or it might be completed by the administrator and used as a discussion vehicle to guide the faculty member toward professional growth.  The authors’ preferred approach would be for the faculty member to complete a self-evaluation of the scorecard, and then in a face-to-face review session with the administrator, go over each metric and agree on a final score and then set goals for the future to enhance the faculty member’s scores. The assignment of weights is arbitrary and is meant to convey a sense of the relative importance as perceived by the program; it is expected that programs that adopt this process will choose to assign their own weights.

Sample Scorecard Discussion

For this paper, the authors have chosen to suggest four major categories with 14 distinct metrics.  The major categories are the traditional ‘Teaching, Research and Service’ with a fourth category, ‘Professional Growth’ added since this impacts all of the traditional categories.  Weights generally reflect the division of labor normally found in a program with research expectations:

Teaching

The Teaching category includes four metrics:

Research

The Research category has three metrics:

Service

The service category has two metrics:

Professional Growth

This category has four metrics.  Rankings in this area impact the three categories above and are often not included in other scoring systems.  The authors feel these metrics are perhaps the most important and have more potential for impacting the faculty member’s performance and growth than the metrics in the above categories.

Conclusion

Whether we realize it or not, faculty members are ranked continuously.  Students evaluate them each semester when they teach.  They are evaluated annually when raises are allocated.  However, the major ranking occurs when faculty apply for tenure and promotion.

As the academic discipline of construction matures, faculty should understand the profiling and ranking systems.  The administrators need to develop skills in profiling and ranking.  There are human resources professionals that have developed techniques and methods that we in academia need to study.

Professors do develop skills in evaluating and ranking students and in most cases use some type of tests or exams each semester.  The students’ work is actually graded by points established and an official grade is awarded.  Academic administrators do not seem to use any type of testing or exams for their faculty.  One program has encouraged their faculty to take the Certified Professional Contractor Level II exam.  Others award credit for faculty members who obtain their Professional Engineer license.  In most cases the PE license requires a written exam with the annual requirement of lifelong learning.

There has been much debate about Promotion and Tenure (P&T) systems.  Some professionals from the industry and some faculty would like to eliminate the requirement to go through a P&T exercise in the six- to seven-year time frame.  The authors feel that new faculty who are properly mentored and work to improve themselves in teaching, research, publications, and service to professional societies and the industry should be rewarded with tenure and promotion.  When the P&T system is used correctly, this is an excellent way for faculty to continually improve.  On the negative side certain university-level administrators misuse the tenure and promotion system and a good faculty members and programs may be penalized. 

A well-conducted annual ranking system can keep faculty who are in the P&T process informed of their progress and motivated to seek continual professional growth.

References

Badger, W. & Smith, J. (2005) Ranking Construction Programs: The Academic Debate Begins. ASC Proceedings of the 41st Annual Conference University of Cincinnati - Cincinnati, Ohio April 6 - 9,

Sevier, R (2003).  “The Problem with Prestige” University Business, 16-17.

Boyer, E. (1990) Scholarship reconsidered. Princeton, NJ: The Carnegie Foundation for the Advancement of Teaching.

Christensen, K. & Rogers, L. (1992) Teaching, service, and research, in evaluation of construction management faculty for tenure and promotion.  ASC Proceedings of the 28th Annual Conference, Auburn, Alabama, 79-83

Tufte, E & Hannestad, G. (1988) Coping with faculty turnover.  ASC Proceedings of the 24th Annual Conference, San Luis Obispo, California, 39 - 41

Ciesielski, C. (1997). Tenure and promotion: A comparison between Construction Management and Civil Engineering.  ASC Proceedings of the 33^rd Annual Conference, University of Washington, Seattle, WA, 21- 31

Waller, R. (1997) A Project Manager Competency Model, Project Management Institute, Seminars & Symposium, (Proceedings: 452-458, Chicago, Ill)

Dainty, A. Cheng, M. & Moore, D. (2004) A Competency-based Performance Model for Construction Project Managers.  Construction Management & Economics, vol. 22, issue 8, pages 877-886

Bigelow D. & West, J. (2003) Putting the Right Project Manager on the Right Job. (Proceedings of the PMI Global Congress 2003 - North America, PMC01.PDF.)

Badger W. (2003) Letter to Bill Barnes, incoming president of ACCE date 13 March

Appendix A

Sample Faculty Scorecard

1. TEACHING [300 points]

1. Student Course Evaluations [100 points] The metric used would be the faculty member’s average score for all courses taught during the most recent academic year.  This average score would be benchmarked against a perfect score and points awarded as follows:

                                                               i.      90-100 percent of benchmark—100 points

                                                             ii.      80-90 percent of benchmark—80 points

                                                           iii.      70-80 percent of benchmark—60 points

                                                            iv.      Less than 70 percent of benchmark—no points

2. Graduate Committee Leadership [50 points]  This metric would assess the faculty member’s participation in the critical role as a member and chair of graduate committees.  For the purpose of scoring, three committee memberships would equate to one committee chairmanship.

                                                               i.      Committee chair, five committees—50 points

                                                             ii.      Committee chair, 3-4 committees—25 points

                                                           iii.      Committee chair, 1-2 committees—10 points

3. Student Exit Surveys [50 points] The metric used would be the faculty member’s average rating for the most recent academic year.  The average score would be benchmarked against a perfect score and points awarded as follows:

                                                               i.      90-100 percent of benchmark—100 points

                                                             ii.      80-90 percent of benchmark—80 points

                                                           iii.      70-80 percent of benchmark—60 points

                                                            iv.      Less than 70 percent of benchmark—no points

4. Teaching Awards [50 points] The metric used would be any teaching awards received during the most recent academic year.  The total awards could not exceed 50 points.

                                                               i.      National teaching award—50 points

                                                             ii.      College/University teaching award—25 points each award

                                                           iii.      Department/program teaching award—10 points each award

5. Peer Ranking [50 points] This metric would depend on the peer ranking system used by the program.  Outside reviewers might be asked to provide a score which could be used in awarding points.

2. RESEARCH [300 points]

1. Proposals [100 points]       The metric used would be the number of grant proposals submitted with recognition of grants awarded.  Only grant proposals in excess of $50,000 could be counted.  Total points awarded could not exceed 100 points.

                                                               i.      Grant proposals submitted—25 points each proposal

                                                             ii.      Grant awards received—up to 100 points, as follows:

1.        Grant awarded $50,000-100,000—50 points each award

2.        Grant awarded $100,000-500,000—75 points each award

3.        Grant awarded >$500,000—100 points

2. Expenditures [100 points]  Research expenditure records are normally maintained by the university and are a measure of the faculty member’s research productivity.  The metric would be the research expenditure for the most recent academic year.  The benchmark would be $200,000.

                                                               i.      Research expenditure > $200,000—100 points 

                                                             ii.      Research expenditure $150,000-200,000—75 points

                                                           iii.      Research expenditure $100,000-150,000—50 points

                                                            iv.      Research expenditure $50,000-100,000—25 points

                                                              v.      Research expenditure < $50,000—no points

3. Publications [100 points]  The metric would be refereed publications (journal articles and conference proceedings) for the most recent academic year.  The benchmark would be five.  Single author publications would receive 5 bonus points. Total points awarded could not exceed 100.

                                                               i.      Five or more publications—100 points

                                                             ii.      Four publications—80 points

                                                           iii.      Three publications—60 points

                                                            iv.      Two publications—40 points

                                                              v.      Less than two publications—no points

                                                            vi.      Textbook published—100 points

3. SERVICE [200 points]

1. Local Service [100 points]  Each of the scoring points below would be counted with the total not to exceed 100 points.  These service activities could be for the program, the college, and/or the university.

                                                               i.      Committee Chair—25 points each chair, up to 50 points

                                                             ii.      Committee Member—5 points each committee, up to 15 points

                                                           iii.      Competition Team Coach—25 points each team

                                                            iv.      Student chapter advisor—25 points

2. State/National Service [100 points]  The scoring points below would be counted with the total not to exceed 100 points.  These service activities would be with state or national organizations.

                                                               i.      Senior Leadership position [President, Board, Executive committee, etc.]—50 points

                                                             ii.      Committee Chair—25 points

                                                           iii.      Committee Member—5 points up to 15 points

                                                            iv.      Journal Editor—50 points

                                                              v.      Journal/Proceedings reviewer—5 points each up to 15 points

4. PROFESSIONAL GROWTH [200 points]

1. Self Improvement [50 points]  This metric would indicate the faculty member’s commitment to life-long learning.  Scoring is for activities in the most recent academic year and may not total more than 50 points.

                                                               i.      Faculty internship with industry [three-month minimum]—50 points

                                                             ii.      Courses completed [on-line or in person, 15-hour minimum]—10 points each course

                                                           iii.      Attendance at seminars [8-hour minimum in construction topic]—5 points each seminar

                                                            iv.      Delivery of continuing education courses—1 point per delivery hour

                                                              v.      Attainment of CPC, PE, or similar—50 points

2. Mentorship [50 points]

3. Collegiality [50 points]  This metric measures the faculty member’s ability to get along with colleagues and students in a way that is cordial, constructive, and loyal.  It measures the faculty member’s ability to argue for a position, yet accept and support the decision of the faculty at large.

                                                               i.      Almost always collegial—30 points

                                                             ii.      Usually collegial—20 points

                                                           iii.      Frequently not collegial—no points

4. Participation [50 points]  The metric used would be the faculty member’s participation at expected events.  The metric would be scored :

                                                               i.      Always attends—50 points

                                                             ii.      Usually attends—40 points

                                                           iii.      Sometimes attends—20 points

                                                            iv.      Rarely attends—no points