This is the second installment in a 3-Article series covering the genesis and maintenance of the National Board Exam for Optometry (the Exam). There are three individual parts of the National Board Exam (Part I – Applied Basic Science, Part II – Patient Assessment and Management and Part III – Clinical Skills Exam), but this article series will discuss the Exam as a whole. The following information should be helpful in providing a basic understanding of this essential tool for ARBO’s member boards.
The National Board Exam – Article 2 of the 3-article series
This is Article 2 in a 3-Article series covering the genesis and maintenance of the National Board Exam for Optometry (the Exam).
In Article 1 we began our discussion of how the Exam is developed and maintained using a ‘validity centered’ approach. We will continue the discussion along the flow chart with Developing and Reviewing content and then pre-test, analyze and assemble operational test.
Develop and review content
Guiding questions for these development steps include:
- How will quality items be created?
- Who will create them?
- How many items should be written?
- Which steps are needed to make sure items are not biased against certain subgroups?
- How can we ensure technical accuracy of the items?
Once the Competency Model is finalized, test items must be written targeting the appropriate content areas. Subject Matter Expert (SME) Committees are trained by testing professionals in the current best practice for writing high-quality items. Generally, more test items are written than are needed for a single exam form to allow for item attrition during review. Once items are written, SMEs are guided through a review process in which they help determine the congruence, accuracy, and fairness of the items. The SMEs make sure the items match the intended content areas, are technically accurate and clear, and do not unfairly disadvantage any group or contain potentially offensive language.
Pretest, analyze, and assemble operational tests
Guiding questions for these development steps include:
- How is pre-test data used to assemble operational tests?
- How do we make sure results from different forms are comparable?
Items that have been written and reviewed are eligible for inclusion on the Exam test form. Test forms are built to reflect the content representation laid out in the Competency Model and Test Blueprint. However, items are not used to compute candidate scores until they have been administered to sample real test candidates. The statistical information obtained from this pre-testing process provides additional information about the technical quality of the items. The empirical data obtained from pre-testing is also important when constructing new test forms. The overall difficulty of the new form is designed to match the old form through a statistical process known as equating.
Article 3 will cover
- Standard setting
- Maintaining the test
Thanks to Dr. Brett Foley of Alpine Testing for the content of this series.