Evidence for Validity and Reliability, and Development of Performance Standards and Cut-Scores for Job-Related Tests of Physical Aptitude for Structural Firefighters.

Abstract

This multi-part study aimed to revise an existing battery of physical aptitude tests for firefighter applicants. Test validity and reliability were evaluated and performance thresholds were determined. In Part I, 49 structural firefighters rated the similarity between the physical demands of the tests and corresponding work activities. In Part II, 23 participants completed the tests on 3 separate days. In Part III, cut-scores were determined using the Bookmark method by an expert panel of 25 firefighter supervisors. Analysis revealed high levels of validity and reliability. The expert panel provided invaluable direction through a combination of independent and group work, leading to consensus on acceptable completion times. Rigorous processes established scientific credibility for the revised battery of tests. Expert knowledge from firefighter supervisors contributed to determining cut-scores following established scientific methods.