Position Summary
The Test Developer drafts exams, conducts quality control reviews, and ensures the accuracy and integrity of medical assessments. This role guides assigned examination programs through all stages of test development and production, maintaining high standards for content and compliance.
Essential Duties and Responsibilities
Key responsibilities include, but are not limited to:
- Guides multiple examination programs by developing, updating, and coordinating project schedules and SME assignments
- Maintains item banks and conducts gap analyses to ensure full blueprint coverage
- Develops exam forms aligned with blueprints and psychometric requirements
- Guides SMEs through form assembly, review, publication, key validation, and post-exam activities
- Organizes and manages exam development committee meetings, including logistics and agendas
- Prepares and contributes materials for committees, task forces, and Board review
- Assists with onboarding and training of new SMEs and updates training materials
- Fosters strong working relationships with committee members and internal teams
- Promotes quality and best practices in test development
- Contributes to test development policies and procedures
- Supports psychometricians in scoring and research-related tasks
- Maintains test development reports for internal and external use
- Provides technical support on testing applications and assists with software updates
- Participates in planning and implementation of the annual Item Writers Workshop
- Supports case development meetings and ongoing SME engagement
- Documents and improves exam development and item/case creation processes
- Analyzes and resolves problems related to exam content and workflow
- Represents ABEM at professional meetings and conferences
- Staffs exam administrations and additional meetings as needed
- Performs other duties as assigned
Minimum Qualifications
- Four-year degree and 3–5 years of experience in testing or a related field
- Minimum of 2 years of program or project management experience (5 years preferred)
- Understanding of question writing and examination construction (expertise preferred)
- Knowledge of measurement principles and data analysis
- Ability to learn and work with medical terminology
- Proficiency in Microsoft 365 (Excel, PowerPoint, Word, SharePoint); advanced Excel preferred
- Strong written and verbal communication skills
- Strong analytical, problem-solving, and decision-making skills
- High level of professionalism and confidentiality
- Ability to collaborate on cross-functional teams
- Experience with item banking platforms (e.g., ExamDeveloper), test delivery systems, and project management tools (e.g., Trello, Smartsheet, HubSpot)
- Ability to travel up to 10% of the time (approximately 1 month per year)
Physical Demands and Work Environment
- Regular verbal communication required
- Frequent hand/finger use; occasional standing, walking, and reaching
- Vision requirements include close, distance, and adjustable focus
- Noise level is typically low to moderate
This document does not create an employment contract, implied or otherwise, other than an “at will” relationship.
