AAMC Standardized Video Interview Frequently Asked Questions
Last updated April 13, 2017
What is the AAMC Standardized Video Interview operational pilot?
In an effort to improve the transition to residency process, the AAMC has developed the Standardized Video Interview as an innovative tool to enable applicants to share objective, performance-based information about themselves beyond academic metrics, to add breadth and depth to their application, as well as to provide residency program directors with additional information to assess applicants.
Following a one-year research study to confirm the tool’s validity and reliability, the AAMC has launched an operational pilot program with the emergency medicine community for the 2018 ERAS application season to explore the value the tool offers to the residency program community and to the residency selection process, and to understand how the scores predict residency outcomes.
What does the AAMC Standardized Video Interview measure?
The video interview is designed to assess knowledge of professional behaviors (professionalism) and interpersonal and communication skills, two competencies that have been identified as required for success in residency and medical practice.
What format does the video interview take?
The video interview is an online interview. Applicants can complete the interview using any internet-enabled computer, tablet or smart phone. Questions are presented in text and applicants provide an audio/video response. The AAMC Standardized Video Interview consists of six questions. Applicants have 30 seconds to read each question and three minutes to respond.
Who is eligible to participate in the pilot?
The operational pilot is limited to the emergency medicine specialty. ERAS applicants to ACGME-accredited emergency medicine residency programs are required to participate in the Standardized Video Interview as a mandatory component of their ERAS application. An applicant’s Standardized Video Interview score, as well as access to the video files, will be provided to all emergency medicine residency programs to which the applicant applied. Emergency medicine residency programs are not required—though are highly encouraged—to make use of these scores and videos as part of their screening and selection process.
The emergency medicine residency community has endorsed the use of this tool by all applicants and program directors in emergency medicine.
What is the timeline for the pilot?
|May 1, 2017||
Standardized Video Interview resources for applicants and advisors become available
Request for special accomodations process opens
|Early June 2017||MyERAS opens; applicants indicate intent to apply in emergency medicine|
|Early June - July 31, 2017||Rolling emails sent to applicants inviting them to complete the video interview online|
|July 31, 2017||Deadline to complete the video interview|
|August 2017||Residency program staff receive training on: use of the interview tool, use of the scores; unconscious bias|
|September 15, 2017||Interview videos and scores made available to programs|
How does the video interview process work?
The process begins with the applicant logging into MyERAS.
Step 1: On the MyERAS dashboard, there is a question near the bottom of the screen that asks, “Are you planning to apply for a residency position at an ACGME-accredited Emergency Medicine program?” Applicants planning to apply in Emergency Medicine, should click “yes.”
Even if an applicant is undecided on a specialty but is considering Emergency Medicine, he/she should click “yes.” There is no penalty for completing the interview and then deciding against applying; if an applicant does not ultimately apply to an Emergency Medicine program, his/her interview video and score will not be transmitted anywhere.
Step 2: Within 24 hours of responding to the question in MyERAS, the applicant will receive an email invitation from the HireVue® system to complete the interview. The email will be sent to the email account registered with MyERAS.
Step 3: Using a link in the email, the applicant should access the HireVue® system and complete a technology check with the device (phone, tablet, computer) that will be used to complete the interview.
Subsequent steps will be outlined in detail in the forthcoming resource guide for applicants (available May 1).
How can applicants prepare for the Standardized Video Interview?
Applicants should prepare for the Standardized Video Interview in the same way they would prepare for an in-person interview. During the video interview, applicants are asked six questions that focus on two ACGME competencies: knowledge of professional behaviors (professionalism) and interpersonal and communication skills. Applicants are encouraged to consider specific experiences that highlight their knowledge in these areas and be prepared to address them during the interview.
Will applicants be able to see their score? How can they prepare or improve their score?
The AAMC is still evaluating feedback from our community and assessing the feasibility of providing applicants with their numerical score in a way that protects the security of the interview. The AAMC will announce a final decision on before the ERAS application season opens in early June.
The AAMC is developing a resource for applicants that will include guidance on how to prepare for the interview. That resource will be available by May 1.
In general, applicants should prepare for the video interview just as they would any telephone or in-person interview. The questions will focus on knowledge of professional behaviors, and interpersonal and communication skills. The interview website (supported by HireVue® technology) also provides an opportunity for applicants to check their computer or mobile device system (like the microphone and lighting) to make sure conditions are optimal for the interview and provides practice questions to help students get comfortable with responding to questions in this online video format. We believe these interventions will help students prepare for the Standardized Video Interview as well as in-personal interviews. Please note, however, that applicants may complete the Standardized Video Interview once. They cannot re-take the interview to improve their score.
We encourage applicants, along with their advisors, to prepare early for the Standardized Video Interview. As with in-person interviews, effective preparation strategies include mock-interviews, reviewing curriculum vitae, and reflection on how best to respond to behavioral and situational questions.
How much does the tool cost? Is there an additional fee for emergency medicine participants?
The AAMC will absorb the cost of the entire pilot; this is a continuation of our testing phase and there will be no additional cost to applicants, schools, or programs participating in this pilot.
While I recognize that the AAMC is covering the cost of the operational pilot, what will the costs be if the tool moves beyond the pilot and who will pay?
The AAMC does not have an answer to that question at this time. We are in the early phases of research to understand the value this tool might bring to stakeholders involved in the residency selection process. The AAMC remains concerned about students’ growing debt burden as well as the financial constraints facing many of our academic health centers.
What is the AAMC doing to address concerns about bias?
The AAMC is concerned about bias and is working to minimize it in the Standardized Video Interview. First, the questions included in the interview have been reviewed by subject matter experts across a number of specialties for bias. Only questions that subject matter experts judged to be free of bias were included. Second, we have called upon experts at the AAMC to develop unconscious bias training specifically for this project; and all raters who score interviews are required to complete this training in addition to other training. The scores from raters are regularly reviewed throughout the process; and if any inconsistencies appear, including suspected bias, individual raters are either re-trained or removed from the rating pool.
The AAMC is also developing training for residency programs on unconscious bias as well as appropriate use of videos and scores. While it is impossible to completely eliminate bias, the AAMC is making every effort to develop a fair interview tool and to articulate our position on the importance of diversity in academic medicine.
Data analyses from the earlier research project phase—in which this video interview tool was tested by volunteer applicants during the 2017 ERAS application cycle—indicates that there are no significant group differences in scores by gender, race/ethnicity, or applicant type (US-MD, US-DO, IMG). These group differences were smaller than the industry average for standardized tests.
Does this tool disadvantage students with disabilities? How will the AAMC provide allowances to applicants who require special accommodations?
The AAMC is finalizing the policy and process for applicants who may need accommodations to completing the Standardized Video Interview. Information and instructions for submitting that accommodation request will be available on May 1.
Is this tool a valid measure of these competencies and does applicant score correlate with resident performance?
The AAMC has analyzed the results of the research study, looking very closely at parameters such as inter-rater agreement, group differences, and correlation of interview scores with USMLE Step 1 scores. The preliminary results from these psychometric measures are promising. The AAMC found that there is no correlation between an applicant’s Standardized Video Interview score and his/her most recent USMLE Step 1 scores. This suggests that the Standardized Video Interview measures something different than academic competency, as was intended.
However, questions about correlation of an applicant’s Standardized Video Interview score to Match success, or to performance in a residency program cannot be answered without an operational pilot. Evaluating the Standardized Video Interview score in relationship to resident evaluations and other outcome data available in the residency program would enable us to more accurately answer these important questions.
What role will computer scoring play in delivering Standardized Video Interview scores to program directors?
If the Standardized Video Interview moves beyond the pilot stage and expands into specialties that have large applicant pools, it is unlikely that the AAMC could resource enough professional raters to score the interviews and make them available to program directors by mid-September when ERAS opens. As part of our current research, the AAMC is exploring the possibility of computer scoring as a supplement to human scoring. During this operational pilot for the 2018 ERAS application season, the AAMC is conducting a parallel research project (without implications on applicants’ scores) to explore the possibilities of computer scoring. For the operational pilot, the scores delivered to residency program directors would be provided by human raters only.
My students already have separate/additional application requirements for certain specialties—is this video interview related, or just one more thing applicants have to complete outside of ERAS?
The AAMC Standardized Video Interview is not affiliated with any of the specialty-specific tools and assessments that are conducted outside of or subsequent to the ERAS application (such as the Otolaryngology Phone Interview Talent Assessment or the Orthopedic Surgery Phone Interview tool). For the 2018 application season, the AAMC Standardized Video Interview pilot is a requirement (for applicants to ACGME-accredited emergency medicine programs) within the ERAS application process. If fully realized as a tool in the future, we hope that the AAMC Standardized Video Interview would serve as one common tool to be used across specialties.
How will the Standardized Video Interview affect introverted applicants who are shy or whose positive traits may not be highlighted by this tool?
The goal of the Standardized Video Interview is to provide applicants with another way to share information about themselves that will add depth to their residency application, in addition to academic metrics, such as grades and Board scores. The tool is designed to assess knowledge of professional behavior and interpersonal and communication skills. These competencies are important to program directors along with other characteristics they may consider in the residency selection process.
Applicants will be provided with resources and tips to help them prepare for the interview. We also encourage applicants to take advantage of practice questions offered by HireVue before they complete their interview. The independent, professional raters are trained to focus on the content of the applicant’s response, and program training encourages the use of broad, inclusive measures to identify applicants who might fit their program. Our goal is to widen the band of acceptable applicants rather than narrow it.
My school does not have a professional video studio for students to complete their Standardized Video Interview. Will the quality of my interview jeopardize my application?
Based on feedback from our vendor and review of interviews completed during the research study year, applicants and medical schools do not need to invest in studio time or professional help to complete the Standardized Video Interview. We encourage you to use your computer or tablet to run a technical test with the vendor in the exact location and environment where you intend to complete the interview so that you’re comfortable that the technology you’re using provides clear audio and video outputs. We recommend tweaking the settings until you’re happy with the output.
Completing the Standardized Video Interview will require more time and increase my anxiety; yet there’s no guarantee that programs will even have time to view all interviews. How will this tool help me?
The size of a program’s applicant pool and how they integrate the Standardized Video Interview into their selection process will likely determine how many video interviews they will view. Feedback from program directors tell us that they want more than Board scores and academic metrics to support their decision-making about who they invite to interview. They also indicate that applicants who fall in the middle (i.e. may not have the highest Board scores but may excel in other areas) may be overlooked because they don’t stand out in other ways. We believe an applicant’s performance on the Standardized Video Interview can provide an additional information point that programs might find helpful when making decisions about who they invite to fill their limited interview slots. This tool provides applicants with another opportunity to help programs see why they might be a good fit for their program.
How are you preventing applicants from saying they’re applying just to get the interview questions to share with friends who are applying in Emergency Medicine?
One of the areas that the interview covers is knowledge of professional behaviors; and our first task is to challenge applicants to integrate this tool into their ethical and professional sphere for the good of all applicants and program directors. Applicants who sign up to complete the interview are asked to certify that they will not collect, share, or in any way distribute the interview questions to anyone. Applicants are subject to an ERAS Investigation if found to have breached the security of the video interview. In addition, the AAMC has taken steps to develop an extensive question bank so that questions are used for a very short period of time. Then, new questions will be introduced so it will be impossible to determine which interview questions any applicant will receive during the interview period. Applicants can use their time preparing for the interview by reflecting on the two areas covered in the interview— interpersonal and communication skills, and knowledge of professional behaviors—and thinking of experiences (clinical and non-clinical) they’ve had that address these areas.
Since this year’s implementation of the Standardized Video Interview (SVI) is the pilot phase of a research project, why is it a mandatory requirement for all EM applicants with no opt-out provision?
This project is not a research study. The results from the research year confirmed that the tool meets psychometric standards for reliability and validity, and correlates in expected ways with other selection tools. The SVI will be used by programs, in conjunction with other application information, to make decisions about which applicants to invite to in-person interviews. The AAMC will evaluate results of the pilot year and make an evidence-based decision about whether or not to continue the program.
How did the AAMC decide the timeframe for making the video interviews a requirement for emergency medicine applicants?
We carefully considered the timing of this project, and know that many schools begin sharing information about the residency selection process with rising seniors before they leave for summer activities. Although the requirement is new this year, we believe that the resources we provide for the video interview will help applicants prepare adequately. We also want to ensure that this critical information reaches program directors early enough to allow them to take this into account as they make decisions about in-person interviews in September. In order to provide sufficient time for our independent raters to evaluate interview responses and to deliver SVI scores to program directors when the ERAS application opens in September, the interview window is set for early June to 12:00 AM Eastern Time on August 1.
Will the Standardized Video Interview take the place of in-person interviews?
No, the Standardized Video Interview does not replace or replicate in-person interviews. The AAMC developed the Standardized Video Interview to provide information that is not readily available nor standardized in the current ERAS application packet when programs are making decisions about who to invite for interviews. Some programs may also use the tool when trying to finalize their rank order lists. Both applicants and programs alike indicate that the in-person interviews are a valuable part of residency selection process.
Contact the AAMC at.