Abstract

Background

The lack of a standardized language assessment process for medical students and physicians communicating in a non-English language threatens healthcare quality and safety.

Objective

To evaluate the validity of a new rating tool, the Physician Oral Language Observation Matrix (POLOM)™, in assessing medical students’ oral communication with Spanish-speaking standardized patients (SPs).

Design

POLOM scores were compared to measures of student medical Spanish proficiency to examine convergent validity and to measures of clinical performance to examine concurrent/criterion validity.

Participants

Forty-two students at two schools completed SP encounters between January 2021 and April 2022, and POLOM raters scored the videorecorded performances between January and June 2022.

Main Measures

Two approaches to generating POLOM total scores were investigated: rater average and strict consensus. Convergent validity was examined via the POLOM’s correlations with (1) the phone-based Clinician Cultural and Linguistic Assessment (CCLA) and (2) the self-rated Interagency Language Roundtable scale for healthcare (ILR-H). Concurrent/criterion validity was examined via correlations with (1) the Comunicación y Habilidades Interpersonales (CAI) scale, (2) a checklist completed by the SP, and (3) a faculty rating of the student’s post-encounter clinical note. Pearson’s correlations of r ≥ 0.5 and r ≥ 0.2 were considered evidence of convergent validity and concurrent/criterion validity, respectively.

Key Results

Both rater average and strict consensus POLOM scores were strongly correlated with ILR-H (r = 0.72) and CCLA (r ≥ 0.60), providing evidence of convergent validity. The POLOM was substantially correlated with the CAI (r ≥ 0.29), the SP Checklist (r = 0.32), and the faculty scoring of the student’s clinical note (r ≥ 0.24), providing concurrent/criterion validity evidence.

Conclusions

The POLOM has demonstrated evidence of convergent and concurrent/criterion validity as a measure of medical students’ Spanish proficiency during SP encounters. Additional research is needed to evaluate how the POLOM can be implemented with resident and practicing physicians, applied to other health professions, and adapted to other languages.

Topic

JGIM

Author Descriptions

Departments of Medical Education and Emergency Medicine, University of Illinois College of Medicine, Chicago, IL, USA
Pilar Ortega MD, MGM

Diversity, Equity, and Inclusion, Accreditation Council for Graduate Medical Education, Chicago, IL, USA
Pilar Ortega MD, MGM

Division of General Internal Medicine, Department of Medicine, University of California, San Francisco, San Francisco, CA, USA
Steven E. Gregorich PhD

Division of General Internal Medicine, Center for Aging in Diverse Communities, Multiethnic Health Equity Research Center, Department of Medicine, University of California, San Francisco, San Francisco, CA, USA
Leah Karliner MD, MAS

Language Initiatives Program, Immigrant Health and Cancer Disparities Service, Department of Psychiatry and Behavioral Sciences, Memorial Sloan Kettering Cancer Center, New York, NY, USA
Javier González MFA

Language and Communication Training Unit, United Nations Headquarters, New York, NY, USA
Cristina Pérez-Cordón PhD

Internal Medicine, Northwestern University Chicago, Chicago, IL, USA
Reniell X. Iñiguez MD

Anesthesiology, Maimonides Medical Center, Brooklyn, NY, USA
Karen Izquierdo MD

Hospital Medicine Service, Immigrant Health and Cancer Disparities Service, Departments of Medicine and Psychiatry and Behavioral Sciences, Memorial Sloan Kettering Cancer Center, New York, NY, USA
Lisa C. Diamond MD, MPH

Share