When Should a Test be Updated or Revised?

This week, Sierra Iwanicki, PhD, clinical psychologist and project director at PAR, explains the background to a question PAR often receives when and why does a test need to be updated or revised?

PAR frequently receives questions about the need to update or revise the instruments we publish. We often look for guidance from published literature and professional organizations such as the National Association of School Psychologists (NASP) and the American Psychological Association (APA) to help guide our decisions. Although there are no absolute rules regarding when to update or revise, these professional guidelines and ethical codes provide examples of situations that would prompt the need for test revision. Here is some of the guidance we follow when determining when and if a revision is necessary:

The Standards for Educational and Psychological Testingstates that revisions or amendments are necessary when new research data, significant changes in the domain, or new conditions of test use and interpretation suggest that the test is no longer optimal or fully appropriate for some of its intended uses (pp. 83). The Standards also notes that the decision to revise or update psychological tests may be considered when there is a change in the conceptualization of the construct.

Guideline 2.4 of the International Test Commission's Guidelines for Practitioner Use of Test Revisions, Obsolete Tests, and Test Disposalrequires test publishers to justify the need for a revised test, stating that:

Test revisions may be driven by knowledge that the assessed behaviors are subject to substantial change over time, by significant demographic changes, from research that leads to improvements in theories and concepts that should impact test use, from changes in diagnostic criteria, or in response to test consumers demands for improved versions. (p. 9)

Standard 9.08, Obsolete Tests and Outdated Test Results, of the APA Ethical Principles of Psychologists and Code of Conduct, states that psychologists do not base such decisions or recommendations on tests and measures that are obsolete and not useful for the current purpose. However, no guidance is provided on how to determine when a test is obsolete.

When determining if revision is necessary, it is important to consider the type of test. For example, the Flynn Effect shows that IQ scores don't remain consistent over time, meaning intellectual assessment tools need to be updated more frequently than personality assessments, where the content remains more constant over time. Butcher notes that not everything in life becomes functionally ineffective at the same rate (p. 263), and tests do not become obsolete simply because of the passage of time.

Ultimately, test publishers are entrusted to monitor changes over time that may prompt the need to revise an assessment.

Are you using a PAR product for research? Learn more about how you can get involved with PAR's data program.