The National Board Exam – Part 3

This is the third installment in a 3-Article series covering the genesis and maintenance of the National Board Exam for Optometry (the Exam).  There are three individual parts of the National Board Exam (Part I – Applied Basic Science, Part II – Patient Assessment and Management and Part III – Clinical Skills Exam), but this article series discusses the Exam as a whole. The following information should be helpful in providing a basic understanding of this essential tool for ARBO’s member boards.

This is Article 3 in a 3-Article series covering the genesis of the National Board Exam (the Exam). In Articles 1 and 2 we discussed how the Exam is developed and maintained using a ‘validity centered’ approach. Article 3 will finish the discussion with Standard setting and Maintaining the test.

Conduct standard setting

The primary guiding question for this step is:

  • How should we determine the cut-point on the test score scale, i.e., what will be the ‘cut score’?

The most critical outcome of a licensure exam is the pass/fail decision.  The passing score must be defensible and reflect the skills necessary for entry-level practice. The National Board of Examiners in Optometry (NBEO) recommended that the passing score[1] be obtained through a systematic standard setting study.

Standard setting is the process of defining the performance expectations of the minimally qualified candidate and translating that performance expectation into a passing score.  For the most recent study, NBEO used the yes/no variation of the Angoff[2] standard setting method (commonly used to determine passing scores for licensure programs).  The standard setting study was conducted with the input of an independent panel of Subject Matter Experts (SME) and facilitated by an independent third-party testing organization.

Maintaining the test

The primary guiding question for this final step is:

  • Once a test is created, what are the test creators’ ongoing responsibilities?

To preserve exam security, NBEO periodically publishes new exam forms.  Although these new forms are based on the same Competency Model, the forms contain new items.  To meet this need, item development meetings, item review meetings, and new form constructions are conducted regularly.  Additionally, to ensure that the test items are performing well statistically, periodic analyses (i.e., health checks) are conducted regularly.

The optometric profession is not static; over time, best practices can change and new technologies emerge. The NBEO constantly monitors to ensure that the content and performance expectations remain current and relevant.  To meet this need, NBEO conducts new job task analysis and standard setting studies.

Table 1. Frequency of Test Development Activities

Activity Frequency
Item Development; Item Review; Health Check; Form Creation At least yearly
Job Task Analysis; Standard Setting Approximately every 5 years

 

Thanks to Dr. Brett Foley of Alpine Testing for the content of this series.

 

[1] Each licensing body is responsible to determine if a candidate has demonstrated sufficient competency to be eligible for license.  Licensing bodies who apply passing scores different from those recommended by NBEO are responsible to justify and defend the decisions made using those passing scores.

[2] For information on the specific methodology, see: Impara, J. C., & Plake, B. S. (1997). An alternative approach to standard setting. Journal of Educational Measurement, 34(4), 355-368.