Assessment Tools for Construction -Education Service-Learning Projects

Introduction

Ideally, service-learning provides students with experiential opportunities to use newly acquired academic skills and knowledge in real life situations in their own communities to enhance and reinforce what was taught in the classroom.  Kahne and Westheimer (1996) added “educators and legislators alike maintain that service-learning can improve the community and invigorate the classroom, providing rich educational experiences for students at all levels (p.593).”   

 Kinsley (1994) stressed that service learning is an education process - not a program - where the service experience is directly related to the academic subject matter. In the construction industry the majority of decisions are made with the benefit of experience with previously encountered cases or situations.  Service-learning provides actual practice in encountering these various experiences for students.  Service-learning could also be better than many internship experiences, because the instructor is an essential facilitator of learning in the service-learning project (Senior, 1998).

The Problem of Assessment

 Many technical fields of study have only incorporated service-learning as " service projects" not specifically tied to learning outcomes with the judgment of knowledge gained simplified to a final student paper about the experience.  Eyler (2000, p. 11) referenced several sources (Eyler, Giles & Gray, 1999; Berson & Younkin, 1996; Marcus, Howard & King, 1993; Miller, 1994; Kendrick, 1996) and stated that

“The effect of service-learning on cognitive outcomes has been less well studied and relatively little attention has been given to defining learning outcomes that would be expected to be enhanced by service participation.  Most of the reports of learning are based on student self reports or faculty testimony.  Where attempts have been made to use grades as measures of learning, evidence is mixed.”

The challenge with assessment of grades for service-learning experiences exists in creating tangible learning aspects to measure. However, DeZure, (2002, p. 75) stated,

“After a decade of prodigious efforts, outcome assessment and service-learning initiatives are proliferating in American higher education, moving from margins to the mainstream.  It is only a matter of time until these two powerful movements converge in a meaningful way.  It is also requiring the assessment movement to create new conceptual approaches and models to evaluate the impact of experiential and service-learning on various constituencies.”

The traditional assessment of student learning is necessary "to ensure quality for student and community participants, to justify resources investments, and to inform the improvement and expansion of such programs” (Gelmon, Holland, Driscoll, Spring, & Kerrigan, p. 1, 2001).  Hence, a significant challenge to the incorporation of effective service-learning possibilities into the curriculum is the successful or at least effectual cognitive assessment of students upon completion of the experience. Service-learning assessment should prompt educators to foster the experience as an aspect of crucial growth. Further, it should provide evidence to professors that service-learning isn’t just about community service.  Service-learning assignments should not only meet identified community needs, but also should make students reflect on the service activity to gain further understanding about course content (Gujarathi, & McQuade, 2002).

Preparation for Project Assessment – Seeking and Identifying Potential Service-learning Project and Educational Objectives

The assessment aspect of service-learning should also be used to assess the projects selected (Thomas & Landau, 2002). Construction management departments should be actively seeking and continually aware of opportunities for service-learning projects.  These opportunities can be more easily and effectively established when a strong relationship exists between the department and business, industry, and civic groups. Students themselves can be a valuable resource for developing projects through student clubs and organizations.  Basically, when a project is identified as a possibility, the professor establishes what could be learned during the project and how this could best be achieved using real-life experiential learning.  In the beginning of a project, it will be important to identify specific objectives relative to the project's success and also the education objectives or expectations for students' skill and knowledge acquisition in order to prepare both the professor and students for the final evaluation.

Time Frame for a Service-learning Project

Sufficient time is necessary during a project to effectively collect applicable data from a variety of assessment sources.  However, generally a semester or term class works well within the time frame.  Some are more suited for a traditional semester, some for spring or summer terms, and some need subsequent semesters and terms to fulfill the requirements.  The time frame factor can be evaluated after the project has been identified.  All of the projects discussed in this paper whether seven weeks or a year were all long enough to incorporate diverse assessment tools and illustrate that non-traditional service-learning projects can be effectively adapted to the typical traditional trimester, semester, or intensified summer term course.

Service Learning Projects Samples

Through these real life learning experiences, practical tools for service-learning assessment were developed to effectively calculate student performances to satisfy traditional grading requirements.

**The Kids On The Move Design Project, 2001-02 - Students from the Construction Management program at Brigham Young University.  Students enrolled into CM 155 Architectural Drawing sections specifically altered to accommodate the service-learning experience.  Kids On The Move is a non-profit organization in Orem, Utah that assists Down-Syndrome children to become mainstreamed into schools and also helps families cope with the related challenges.  They had outgrown their current facility and these different sections were responsible for all of the architectural drawings necessary to build their new structure.  The students worked closely with the director of KOTM, the architect, related contractors, and engineers to design the future 25,000 square foot structure.

**Building for Hope, 2000 - A group of Technology Teacher Education students from Brigham Young University were involved in this project.  Generally for this class, TTE 120, a production project would be implemented to help the students understand how the process operates and proceeds would be used for assistance to the program.  This class, however, worked through a process to design, build, produce and market children's playhouses.  All proceeds went to benefit the National Childhood Cancer Foundation. 

**Texas A&M Construction Science 375 Project –  The Fall 2000 semester service-learning projects in Construction Science 375 provided an application-based format for learning.  Students developed estimating guideline for various charitable organizations.  Some examples include developing guidelines for estimating repairs to a local Girl Scout camp, guidelines for remodeling residential property for wheel chair access, and providing rural schools in Texas with a web-based conceptual estimator.

**The University of Louisiana Construction Management 356 Project – During the fall semester of 1999, Construction Management 356 student team leaders worked with the Dean of Community Service and with the university Americorps officer to find a suitable project.  The project chosen was a small remodeling job at a government housing project in Lafayette, Louisiana. The center served as an after school care and tutorial center for many African American children.

**The West Plains, Missouri Downtown Redevelopment Study, 1998 - The West Plains "Architecture on the Square" project involved students from Southwest Missouri State University architecture program enrolled in a "Special Topics" elective course for a research based architectural study to determine the possible future of the historic downtown square.  Students created a scale model of the square with a written proposal and presented their work to the Greater West Plains Chamber of Commerce.

 **The Oklahoma City Bombing Memorial Design Submission, 1996 - Architecture students from Oklahoma State University enrolled in a highly requested "Special Topics" course to research, create, and submit one of the over 500 design proposals for the Oklahoma City Bombing Memorial.

**The Guthrie, Oklahoma Architectural Study, 1995 - Architecture and construction students from Oklahoma State University enrolled into a "Special Topics" course called "The Guthrie Experience" and spent a summer in Guthrie, Oklahoma creating an architectural proposal for the possible enhancement for its historic downtown district.  A model along with drawings and a written proposal was given to the mayor and city council of Guthrie, Oklahoma.

The Las Vegas, Nevada Homeless Transition Proposal, 1989 - The students in the Architecture Club at the Community College of Southern Nevada proposed plans for a "Homeless Transition Center" to accommodate and to teach homeless persons how to succeed in their lives.  The drawings were then presented to community organizations for possible implementation.

Qualitative Assessment Tools for Students in Service-learning Projects

Some of the tools that can be utilized for assessment during a service-learning project include:  Preliminary Focus Group Session/Evaluation Expectation Review, Focus Group Sessions, Student Journal Entries, Professor/Participant Observer-Journal Entries, Outside Observer Documents, Final Meeting With Benefactor & Display/Publication and/or Sharing of Project Results, Final Focus Group Session, Peer Evaluations & Student Evaluations, Final Assessment, and Compilation of Data. Using any combination of these tools will increase the validation of effective assessment.

Preliminary Focus Group Session

This meeting between all parties should take place before the project actually begins.  It also provides an opportunity for the students to be introduced to the benefactor or sponsor of the service-learning project.  The students need to understand the previously identified objectives both for the project and educationally, and they need to be appraised from the beginning concerning the method of assessment, which will be used to identify both the success of the project and the student's final grade.  Discussion needs to take place on the orchestration of the project using class, lab, and individual student time frame. 

 The importance of keeping proper documentation of the project must be discussed with the students and outside observer.  Actual sample forms for self, peer, outside observer, and professor observations/evaluations can be shown and explained to students.

Soliciting student input into the class structure, the students and the professor can map out the direction and specifics for the service-learning project.  Thus the service-learning project can become a student-directed, teacher-guided, hands-on learning opportunity related to the coursework that provides assistance or meets a real community need.

Focus Group Sessions

Hawe, et al. (1990, p. 174) stated "A focus group is another name for a group interview or a group discussion where the focus is on a particular topic of interest . . ."  Focus group sessions should happen weekly during the project and should not last more than an hour.  There are several purposes for the focus group sessions.  A primary purpose is to gather all of the students together and discuss the progress, direction, and specific assignments related to the project.  Another is for the professor to ask pointed questions to the group about the learning process in order to ascertain skill acquisition, to assess additional instruction needs, and also to determine the rigor of individual student participation.  Complete and thorough records must be kept of the focus group sessions either written, video, or audiotape for use in the final evaluation of the students.  These records can be very valuable in determining the assessment of each student’s participation individually, since determination of goals and individual assignments take place during these sessions.

Student Journal Entries

Gelmon et.al. (2001, p. 17) stated that student journals can provide important data that will "augment primary data and reveal information not otherwise provided."  The student journal entries will be one of the primary tools used in evaluation.  In the beginning, the students must have a good understanding of their responsibility of keeping a comprehensive log based upon their viewpoints of the project and achievement of educational goals.  The failure to adequately comply with this process could be reflected upon the student's final grade.  At the end of a designated period of time, each student should be required to send an e-mail journal entry to the professor. These can be forwarded from e-mail to a folder created specifically for the students' journal entries.  Within the folder, different directories can be segregated to accommodate each of the different aspects of data accumulation.  Programs such as Blackboard are also good forums for facilitating student journal communications.  The professor can also make comments electronically on the entries as needed. A standard journal entry form can be attached electronically in e-mail and sent to each student. Table 1 shows an example of a form developed for the KOTM project:

The student journal entries will provide great insight for the professor concerning issues that might not be noticed from any of the other forms of data collection.  Also, these entries keep the professor informed as to the frustrations that could be alleviated with additional instruction while also providing a "heads up" on any potential problems arising.  Topics for the next focus group session can also be identified including any possible teaching or instructional aspects.  In addition, the professor can give individual student attention and encouragement from information received in journal entries.

Professor as Participant Observer

Being a participant observer means the professor becomes a natural project participant. Most of all it gives the professor an opportunity to work hand-in-hand with the students, actively participating and gaining first hand knowledge about the project's general progress.  Also, it gives the professor direct insight as to the individual skill levels and progress of each student and the time and need for further instruction.

Table 1

Assessment Example Form #1

By association the professor will be able to identify, direct, and stimulate discussion concerning many of the issues that will arise during the focus group sessions.  This method also presents a good opportunity to collect and make note of pertinent information necessary for assessment at the end of the project.  Here the professor can begin using real life problems and hands-on experiences rather than just the traditional lecture based instruction - teacher to student format becomes teacher with student format.  Dobbert (1982) stated that the participant professor systematically seeks out and organizes information on the project on a social theory rather than focusing on situationally defined methods, observing and retaining information subjectively based upon students’ situational activities.

Outside Observer Documentation

Another source for assessment documentation of a service-learning project is an outside observer.  This person could be a student assistant or teaching assistant, or a business/community member involved with the project.  The responsibility of the outside observer is to document the project from a non-participant’s viewpoint.  Using a video camera, still camera, interviews, or even a detailed log/journal kept of the project, or any combination of these can do this.  It is important, however, that this document be a complete and comprehensive record of the observer’s impressions of the entire project including daily activities, group activities, and focus group sessions.  The outside observer provides another dimension toward the final evaluation, offsetting bias that might exist from the other forms of evaluation.

Final Meeting with Benefactor

If it is possible, a final meeting should be established with the benefactor of the service-learning project to validate students’ contributions, to allow for student presentation, and to gain additional insight about the project, student performance, and possible future project suggestions.  Often opportunities arise through the benefactor or community publications to display or publish the students’ service-learning project.  This possibility motivates students and validates their efforts in the end.  This process also encourages a positive relationship between the community and educational institution, which in turn could lead to further service-learning possibilities.

Final Focus Group Session and Collection of Self & Peer Evaluations

The final focus group session will be a culminating meeting to discuss the entire project and re-evaluate the forthcoming assessment process.  Student self and peer evaluations should be collected following this meeting. The students should be required to evaluate each of their peers, and perhaps groups, and provide a complete personal evaluation as well.  Hansen (2000) said,

“Evaluation demands create the final set of dynamics with profound impact on collaborative classroom approaches. Instructional contexts emphasizing learner responsibility require learners to take responsibility for the evaluation of student work, their own and others'. Such a notion is foreign to the traditional teaching approaches they are familiar with, which assume that only experts--that is, instructors--are in a position to evaluate . . . If collaborative work takes up a significant portion of students' time in a class, fairness often requires a peer-evaluation component in the grading process.”

 The professor can be specific when creating the evaluation form to ask questions that will be pertinent to the final evaluations.  An example of the Guthrie, Oklahoma evaluation form in Table 2 illustrates this concept.

The project needs to now be examined from a completion standpoint.  A discussion about the level of completion as related to the predefined objectives should ensue.  The students can at this time be asked to write final statements, thoughts, and/or comments about the service-learning experience.  This could be part of a final exam given during the final focus group session.  During their experience on the service-learning project the students learned according to the specific educational objectives defined, these learning objectives quantified could be organized into a final exam similar to other traditional courses.  This measurement could then be incorporated into the final assessment tool for evaluating grades.  The weekly focus group sessions were the formative evaluation in questioning, “How are we doing?”  Whereas the final focus group session becomes the summative evaluation ”How did we do?”

Compilation of Data

When the project is finished and the final focus group session has been conducted, a complete compilation of all data should be done.  This data should be compared and analyzed, taking common thoughts and critical issues and grouping them together. Commonalities will begin to appear as multiple sources of information come together.

The purpose of corroboration in this model is to help the professor ensure credibility and documentable evidence for the final assessment (Stainback and Stainback, 1988).  An effective method of corroboration is called triangulation.  Denzin (1978) stated that one type of triangulation involves the convergence of multiple data sources.  These sources could include personal journals kept by the students, video logs or journals by the professor and outside observer, focus group discussions, self and peer evaluations (Table 3), presentation or publication of project, a final assessment, and perhaps if needed, personal interviews.  Triangulation, a method of corroboration, gains multiple perspectives of data collection, thus enhancing the credibility of the final assessment.

Development of Final Evaluation Tool

With all of the data from the diverse sources gathered and categorized, a tool for assessment can be developed.  At the beginning of the project certain educational and project objectives were identified thought critical to the project's success.  Elements from these objectives serve as an important part of building the final evaluation tool.  Aspects brought in from the collected data should also become important facets of the evaluation tool.  Table 4 is an example of a rubric developed to better organize the assessment of final student grades for a service-learning project experience where subjective data is given an objective format with specific percentages established for final grading:

 Table 2

Assessment Example Form #2

 Table 3

 Assessment Example Form #3

Table 4

 Assessment Example Form #4

Conclusions

The nature of service-learning experiences coincide directly with the nature of internships in constructio9n education, they are both highly experiential.  Because of their experiential nature these education possibilities are not traditional and therefore do not fit the traditional method for student grade assessment.   A compilation of diverse data collected from several triangulated sources provides good stable information to effectively and appropriately assess grades, combining both subjective and objective data.  By using these tools or a personalized rendition of these principles, service-learning projects and closely related internship experiences can be evaluated and grades for students can be effectively assigned which meet both the traditional need for measurement and the need for implementation of non-traditional highly acclaimed teaching methods.  These tools have been developed and used for over fifteen years and from six different higher education institutions validating the legitimacy of their nature and implementation in a variety of service-learning project experiences.  While these experiences are architectural/construction based, these tools could be adapted or modified to fit any experiential and or service-learning project.  It would be a shame to let traditional views of grading keep instructors and students from benefiting from the positive effects of service-learning.

References

Denzin, N. (1978).  The research act: A theoretical introduction to sociological methods.  New York, NY: McGraw-Hill

DeZure, D. (2002).  Essay Review - Assessing service-learning and civic engagement: principles and techniques.  Michigan Journal of Community Service Learning.  8 (2), 70-82.

Dobbert, M. L. (1982). Ethnographic research:Ttheory and application for modern schools and societies.  New York: Praeger.

Eyler, J. (2000).  What do most people need to know about the impact of service-learning on student learning?  Michigan Journal of Community Service Learning.  Special Issue, 11-17.

Gelmon, S., Holland, B., Driscoll, A., Spring, A., & Kerrigan, S. (2001).  Assessing service-learning and civic engagement.  Campus Compact.  Brown University Press.

Gujarathi M., & McQuade, R. (2002). Service learning: Extending the curriculum.  The CPA Journal; New York: 72 (2) 67-69.

Hansen, E., J. (September, 2000 Issue).  The ethics of learner - centered education.  Citing online sources.  [WWW document].  URL http://www.findarticles.com/cf_0/m1254/5_32/66278485/p1/article.jhtml?term=progressive+education+grading

Hawe, P., Degeling, D., & Hall, J. (1990).  Evaluating health promotions: a worker's guide.  Artarmon, NWS: MacLennan & Petty.

Kahne J. & Westheimer, J. (1996, May).  In the service of what?  Phi Delta Kappan.  593.

Kinsley, C. (1994).  What is community service learning.  Vital speeches, LXI 2

Senior, B., (1998). Infusing practical components into construction education.  Journal of Construction Education. 2 (2) 145-154.

 Shumer, R. (2002). Teacher Education and Service-Learning: A Critical Perspective.  Citing online sources.  [WWW document].  URL http://www.aahe.org/members_only/SL-shum.htm

 Stainback, S., & Stainback, W. (1988).  Understanding and conditioning qualitative research.  Dubuque, IA: Kendall and Hunt.

 Thomas, K., & Landau, H. (2002). Organizational development students as engaged learners and reflective practitioners: The role of service learning in teaching OD. Organizational Development Journal; Chesterfield; Fall 2002, 3 (20) 88-99.