2015 Mode Experiment

In support of the national implementation of the Outpatient and Ambulatory Surgery CAHPS® (OAS CAHPS) the Centers for Medicare & Medicaid Services (CMS) conducted a mode experiment from August through December 2015. CMS used the findings from the mode experiment to assess the effects of using different data collection modes of administration and to determine whether ratings of outpatient/ambulatory care vary by patient characteristics. The mode experiment included three administration modes: mail, telephone and a mixed-mode approach (mail with telephone follow-up). The mode experiment included a nationally representative sample of hospital outpatient surgery departments (HOPDs) and free-standing ambulatory surgery centers (ASCs). The overall response rate for all three modes was 39%. By mode, the response rates were: 37% for the mail-only mode, 34% for telephone-only mode, and 50% for mixed-mode (mail with telephone follow-up).

The results of the OAS CAHPS Survey mode experiment showed no significant differences in results based on survey mode. There were, however, differences in responses attributable to patient-mix characteristics. The following six patient characteristics were found to be significant predictors: surgery type, overall health, overall mental health, age, education, and how well the patient speaks English. Therefore, patient-mix adjustments are made on OAS CAHPS Survey results that are publicly reported using the six patient characteristics identified during the mode experiment, but no adjustments are needed for mode differences. The patient-mix adjustment coefficients will be updated quarterly as national implementation data are available.

2019 Mode Experiment

CMS is interested in investigating the feasibility of offering the OAS CAHPS Survey using a web-based format (see 82 FR 59216). As part of the investigation, CMS has designed a new mode experiment to assess the impact of adding web-based survey administration. This mode experiment tested five administration modes with patients who receive outpatient surgical care: mail only, telephone only, web only, web with mail follow-up and web with a telephone follow-up. Data collection was completed in the fall of 2019. Analysis activities are underway. Preliminary response rates by mode are: 35% mail-only; 19% phone-only; 29% web-only; 39% web with mail follow-up; and 35% web with telephone follow-up.