health services research training

• Career Center
• Member Login

HSR Training Programs

Institutions offering training in the full range of health services research, including implementation and dissemination research and embedded research opportunities.

Below is a list of health services research-related training programs offering a range of experiences from master’s and doctoral degrees, post-doctoral and experiential training. This list is organized alphabetically by degree type. When you click on the name of the university, you will be directed to the program landing page. The primary program contact name and email is listed for easy access, but may be found on the landing page as well.

If you have questions about this resource, or suggestions for training programs that should be included, please contact [email protected] or fill out the submission form via the link on the sidebar.

Postdoctoral Programs

Kaiser Permanente Division of Research, Northern California Delivery Science Fellowship Program Dr. Richard Grant, Program Director [email protected]  Dr. Julie Schmittdiel, Program Associate Director [email protected]

Kaiser Permanente Division of Research, Northern California T32 Training Program in Diabetes Translational Research Dr. Richard Grant, Program Director [email protected]  Dr. Julie Schmittdiel, Program Co-Director [email protected]

Kaiser Permanente Washington Health Research Institute CATALyST K12 Washington Learning Health System Scholar Program Dr. Diana Buist, Program Co-Director [email protected]

University of California, Los Angeles, David Geffen School of Medicine Stakeholder-Partnered Implementation Research and Innovation Translation (SPIRIT) K12 Program  Dr. Michael Ong, Professor of Medicine, UCLA/Hospitalist Chief, VA Greater Los Angeles Healthcare System  [email protected]

University of Colorado, Anschutz Medical Campus, School of Medicine K12 Career Dev​elopment Training in T4 Implementation Science and Heart, Lung, Blood, and Sleep (HLBS) Disorders Rebecca Speer, Research Training Program Manager [email protected]

University of Massachusetts Medical School PRACCTIS - Prevention and Control of Cancer Training in Implementation Science Dr. Tom Houston, Program Co-Director [email protected] Dr. Stephanie Lemon [email protected]

University of Minnesota, School of Public Health Minnesota Learning Health System Mentored Career Development Program Dr. Timothy Beebe, Program Co-Director [email protected]

University of North Carolina - Chapel Hill, Sheps Center for Health Services Research T32 Training Program in Health Services Research  Dr. Mark Holmes, Program Director [email protected]  Dr. Kathleen Thomas, Program Associate Director [email protected] University of Pennsylvania, School of Nursing  Center for Health Outcomes and Policy Research (CHOPR)  T32 Postdoctoral training program in nursing health services research  Irene Hung, Administrative Director  [email protected] 

OAA National Program Contacts

U.S Department of Veterans Affairs Office of Academic Affiliations Big Data-Scientist Training Enhancement Program (BD-STEP) Dr. Stuart Gilman, Advanced Fellowships and Professional Development Director [email protected] Dr. Jeff Cully, Advanced Fellowships and Professional Development Clinical Director [email protected]

U.S Department of Veterans Affairs Office of Academic Affiliations VA Advanced Fellowship National Clinical Scholars Program Dr. Stuart Gilman, Advanced Fellowships and Professional Development Director [email protected] Dr. Jeff Cully, Advanced Fellowships and Professional Development Clinical Director [email protected]

U.S Department of Veterans Affairs Office of Academic Affiliations VA Advanced Fellowship Program in Health Services Research and Development (HSR&D) Dr. Stuart Gilman, Advanced Fellowships and Professional Development Director [email protected] Dr. Jeff Cully, Advanced Fellowships and Professional Development Clinical Director [email protected]

U.S Department of Veterans Affairs Office of Academic Affiliations VA Advanced Fellowship Program in Clinical Simulation Dr. Stuart Gilman, Advanced Fellowships and Professional Development Director [email protected] Dr. Jeff Cully, Advanced Fellowships and Professional Development Clinical Director [email protected]

U.S Department of Veterans Affairs Office of Academic Affiliations VA Advanced Fellowship Program in Medical Informatics Dr. Stuart Gilman, Advanced Fellowships and Professional Development Director [email protected] Dr. Jeff Cully, Advanced Fellowships and Professional Development Clinical Director [email protected]

U.S Department of Veterans Affairs Office of Academic Affiliations VA Interprofessional Fellowship Program in Patient Safety Dr. Stuart Gilman, Advanced Fellowships and Professional Development Director [email protected] Dr. Jeff Cully, Advanced Fellowships and Professional Development Clinical Director [email protected]

U.S Department of Veterans Affairs Office of Academic Affiliations VA Quality Scholars Fellowship Program Dr. Stuart Gilman, Advanced Fellowships and Professional Development Director [email protected] Dr. Jeff Cully, Advanced Fellowships and Professional Development Clinical Director [email protected]

PhD Programs in Health Services Research

Baylor University, Hankamer School of Business Ph.D. in Health Services Research Dr. Neil Fleming, Program Director [email protected]

Boston University, School of Public Health Ph.D. in Health Services Research  Dr. Steven Pizer, Director [email protected]

Brown University, School of Public Health Ph.D. in Health Services Research Dr. Amal Trivedi, Graduate Program Director [email protected] 

Emory University, Rollins School of Public Health Ph.D. in Health Services Research and Health Policy Dr. David Howard, Program Director [email protected]     Kent Tolleson, Program Administrator [email protected] Johns Hopkins University, Bloomberg School of Public Health Dr.PH concentration in Implementation Science Dr. Stefan Baral, Concentration Faculty [email protected] Dr. Melissa Davey-Rothwell, Concentration Faculty [email protected]

Johns Hopkins University, Bloomberg School of Public Health Ph.D. in Health Services Research and Policy Albert Wu, Ph.D., Program Director [email protected]

Northwestern University, Feinberg School of Medicine Ph.D. in Integrated Health Sciences Dr. Neil Jordan, Program Director [email protected] 

Old Dominion University, College of Health Sciences Ph.D. in Health Services Research Dr. Richardean Benjamin, Associate Dean  [email protected] Bonnie Van Lunen, Professor and Chair [email protected]

Rutgers University, School of Public Health Ph.D. in Health Systems and Policy Dr. Alan C. Monheit, Professor of Health Economics and Chair; and Associate Dean of Academic and Faculty Affairs     [email protected] 

Texas A&M, School of Public Health Ph.D. in Health Services Research Dr. Robert Ohsfeldt, Program Chair [email protected]  Dr. Michael Morrisey, Program Vice Chair [email protected]

University of Alabama at Birmingham, School of Health Professions Ph.D. Program in Administration Health Services Dr. Larry Hearld, Program Director [email protected]  University of Arkansas for Medical Sciences, Graduate School and College of Public Health  Ph.D. in Health Systems  and Services Research Dr. Holly Felix, Program Director [email protected]

University of Chicago, Division of Biological Sciences Ph.D. In Health  Services Research Dr. R. Tamara Konetzka, Associate Professor [email protected] 

University of Colorado Denver, School of Public Health Ph.D. In Health  Services Research Dr. Richard Lindrooth, Interim Chair & Professor     [email protected] 

University of Florida, College of Public Health and Health Professions Ph.D. in Health Services Research Dr. Arch G. Mainous III, Program Director  [email protected] 

University of Maryland, School of Public Health Ph.D. in Health Services Dr. Luisa Franzini, Professor and Chair, Health Services Administration [email protected]  

University of Michigan, School of Public Health Ph.D. in Health Services Organization and Policy     Edward Norton, Program Director [email protected] 

University of Michigan Medical School, Department of Learning Health Sciences Ph.D. in Health Infrastructures and Learning Systems   Dr. Gretchen Piatt, Program Director [email protected] Beth Hill, Program Coordinator   [email protected] 

University of Minnesota, School of Public Health      Ph.D. In Health Services Research, Policy and Administration Dr. Donna McAlpine, Associate Professor and Director   [email protected] University of North Carolina at Charlotte, College of Health and Human Services Ph.D. in Health Services Research Dr. Larissa Huber, Program Director [email protected] 

University of Pennsylvania, Wharton School Ph.D. in Health Care Systems Dr. Mark V. Pauly, Director [email protected] University of Pennsylvania, School of Nursing  Center for Health Outcomes and Policy Research (CHOPR)  T32 Predoctoral training program in nursing health services research  Irene Hung, Administrative Director  [email protected] 

University of Pittsburgh, School of Public Health     Ph.D. in Health Services Research and Policy  Jessica Dornin, Program Coordinator [email protected] 

University of Rochester, School of Medicine and Dentistry Ph.D. in Health Services Research and Policy Dr. Peter Veazie, Chief, Division of Health Policy and Outcomes Research [email protected] 

University of South Carolina, Arnold School of Public Health Ph.D. in Health Services Policy and Management     Dr. Saundra Glover, Department Director  [email protected] 

University of South Florida, College of Public Health Ph.D. in Health Services Research Dr. Etienne Pracht     [email protected] 

University of Texas, Houston, School of Public Health Ph.D. in Health Economics / Health Services Research Robert O. Morgan, Department Chair  [email protected]  University of Washington, School of Public Health Ph.D. in Global Health Metrics and Implementation Science Emmanuela Gakidou, Program Co-Director [email protected] Kenneth Sherr, Program Co-Director [email protected]

University of Washington, School of Public Health Ph.D. in Health Services     Dr. Emily Williams, Director [email protected] 

University of Wisconsin-Madison, School of Medicine and Public Health Ph.D. in Population Health with concentration in Health Services Research Dr. Maureen A. Smith, Director, AHRQ Pre-doctoral and Post-doctoral Traineeships in Health Services Research and Healthcare Quality Improvement     [email protected] 

Virginia Commonwealth University, School of Allied Health Professions Ph.D. in Health Services Organization and Research Dr. Jan P. Clement, Professor and Director [email protected] 

University of Nebraska Medical Center, College of Public Health Ph.D. in Health Services and Policy Research Dr. Fernando Wilson, Graduate Program Director [email protected]

PhD in Public Health & Informatics

Brandeis University, Heller School for Social Policy and Management Ph.D. in Health and Behavioral Health  Dr. Constance Horgan, Director [email protected]  Dr. Jon Chilingerian, Director [email protected] 

Saint Louis University, School of Public Health Ph.D. in Public Health Studies Dr. Mario Schootman, Director [email protected] 

Saint Louis University, School of Public Health Ph.D. in Public Health Studies Dr. Elizabeth Baker, Co-Director [email protected] 

Case Western Reserve University, School of Medicine Ph.D. in Population and Quantitative Health Sciences Dr. Nickalaus Koziura, Administrative Director [email protected]

University of Texas, Houston, School of Biomedical Informatics Ph.D. in Biomedical Informatics Dr. Dean F. Sittig, Program Director [email protected] 

PhD in Health Policy & Management

Dartmouth College, Dartmouth Medical School  Ph.D. in Health Policy and Clinical Practice Dr. Robert E. Drake, Director of PhD Program, Professor of Psychiatry, of Community & Family Medicine and of The Dartmouth Institute     [email protected]

Harvard University, School of Public Health Ph.D. in Health Policy Deborah Whitney, Executive Director [email protected] 

Penn State University, College of Health and Human Development Ph.D. in Health Policy and Administration Dr. Bobbi Robison, Program Coordinator [email protected] 

Tulane University, School of Public Health and Tropical Medicine     Ph.D. in Global Health Management and Policy Dr. Valerie Yeager, Program Director [email protected] 

University of California, Berkeley, School of Public Health  Ph.D. in Health Policy Dr. Hector Rodriguez, Program Director [email protected] 

University of California, Los Angeles, Fielding School of Public Health   Ph.D. in Health Policy and Management     Anna Lim, Program Contact [email protected]

University of Iowa, College of Public Health Ph.D. in Health Services and Policy Gwen Archibald, Director of Student Services and Academic Program Support [email protected]  

University of Kansas, School of Medicine Ph.D. in Health Policy and Management     Dr. Mary Zimmerman, Professor and Ph.D. Program Director [email protected] 

University of Illinois, Chicago, School of Public Health Ph.D. in Health Policy and Administration Dr. Anthony T. LoSasso, Professor  [email protected] 

University of Massachusetts, Amherst, School of Public Health and Health Sciences Ph.D. in Health Policy and Management     Dr. Michael Begay, Associate Professor [email protected] 

University of Pennsylvania, Wharton School Ph.D. in Health Care Management and Economics Joanne Levy, Associate Director [email protected] 

Yale University, Yale School of Public Health  Ph.D. in Health Policy and Management     Dr. Howard Paul Forman, Professor, Director, Health Care Management Program [email protected]   

Masters Degrees

Case Western Reserve University, School of Medicine M.S. in Population and Quantitative Health Sciences Nickalaus Koziura, Administrative Director [email protected] 

Oregon Health Science University, School of Public Health     M.P.H. in Environmental Systems and Human Health Dr. Rick Johnson, Program Director  [email protected] 

Oregon Health Science University, School of Public Health M.P.H. in Epidemiology Dr. Janne Boone-Heinonen, Program Director [email protected] 

Oregon Health Science University, School of Public Health M.P.H. in Health Promotion Dr. Lynne Messer, Program Director [email protected]  Oregon Health Science University, School of Public Health M.P.H. in Health Care and Health Disparities Dr. Elizabeth Waddell, Program Director [email protected]

Oregon Health Science University, School of Public Health M.P.H. in Biostatistics Dr. Rochelle Fu, Program Director [email protected] 

Oregon Health Science University, School of Public Health M.P.H. in Management and Policy     Dr. Julia Goodman, Program Director [email protected] 

University of California, Los Angeles, Fielding School of Public Health M.S. in Health Policy and Management     Anna Lim, Program Contact [email protected]

University of Iowa, College of Public Health 

M.S. in Health Services and Policy Research

Gwen Archibald, Director of Student Services and Academic Program Support

[email protected]

University of Michigan Medical School, Department of Learning Health Sciences M.S. in Health Infrastructures and Learning Systems   Dr. Gretchen Piatt, Program Director [email protected] Beth Hill, Program Coordinator   [email protected] 

University of Michigan Medical School, Department of Learning Health Sciences Online M.S. in Health Infrastructures and Learning Systems   Dr. Caren Stalburg, Program Director [email protected] Angie Sullivan, Program Coordinator   [email protected] 

Weill Cornell Medical College M.S. in Clinical Epidemiology & Health Services Research Dr. Mary E. Charlson, Distinguished Professor of Medicine Program Director [email protected] 

Skip to content

• Health Services Research Training

Program Directors: Jennifer Waljee, MD, MPH, MS and Richard Hirth, PhD

Application Deadline: December 31st, 2024

About the Program

This training program focuses on four broad pedagogic competencies to train the next generation of researchers to address issues related to quality, safety, effectiveness, access, and costs in the health system:

• Understanding the fundamentals of health services problems and issues;
• Understanding theory and methods from social science and health science disciplines, and how they can be applied to health services problems and issues;
• Conducting health services research, both in an independent and leading role as well as a supporting team-based role; and
• Developing oneself continually as a health services scholar.

The program is designed to produce independent investigators by providing a solid academic foundation, experience in the conduct of health services research, and socialization into the normative behavior of high quality, ethically sensitive research and its associated values. This program is intended for both predoctoral and postdoctoral fellowships, though for predoctoral students is usually utilized in the first one to two years of study.

Trainees will gain research experience through courses, participation on research teams led by faculty, and independent development and leadership of their own research projects. From the beginning, trainees will also be immersed in one or more research teams led by their faculty mentors and given roles that will allow them to develop new skills and contribute meaningfully to various aspects of the project.

Coursework for both predoctoral and postdoctoral trainees will be based on the Health Services Organization and Policy predoctoral curriculum in Health Management and Policy. Given the more advanced career stage of postdoctoral trainees, they will participate in a selected subset of courses from the predoctoral curriculum. The postdoctoral trainees’ coursework will vary across individuals depending on prior training. The course curriculum consists of the following components:

• Health and health services and the structure of operation of the health system (10 credit hours)
• Research methods, applied statistics and epidemiology (12 credit hours)
• Cognate disciplinary theory and methods (24 – 27 credit hours)
• Intra- and cross-disciplinary integration (9 – 12 credit hours)

The program combines clinical and social science perspectives, and leverages connections to key stakeholders such as provider organizations and payers.

Fellows will receive a stipend, benefits, travel funds, and additional research support. A salary supplement is available for PhD applicants. 

How to Apply

To apply, please send the following to  [email protected] :

• A cover letter describing your research interests, experience, and career plans in the area of health services research
• A curriculum vitae
• One current letter of reference, preferably from a faculty member in your doctoral program
• A copy of your best written work

Applications will be reviewed after the closing date. Candidates will be notified by February 1 st if invited for an interview and March 31 st if offered a spot in the training program.

NEXT CLOSING DATE: December 31st, 2024.

• Current Fellows
• Past Fellows
• Quantitative Core
• Qualitative Core
• Center for Surgical Training and Research Fellowship
• Center for Basic and Translational Science
• Center for Global Health Equity
• Center for Global Surgery
• Education Research Sciences Collaborative
• Institute for Health Policy and Innovation
• ACS Trauma Lab
• Health Design Lab
• Michigan Health Economics Affordability Lab
• Healthcare Administration Fellowship
• Multidisciplinary Cardiovascular Research Training
• Obesity and Gastrointestinal Surgery Training Program
• Surgical Oncology Research Training
• Fellow Exchange Program
• Launch Teams
• Data Center

Stanford University

Stanford Health Policy is a joint effort of the Freeman Spogli Institute for International Studies and the Stanford School of Medicine

Stanford-AHRQ Health Services Research Training Program

Program Co-Directors:

Laurence Baker , PhD, MA, Professor of Medicine, Senior Fellow, Stanford Institute for Economic Policy Research, Research Associate, National Bureau of Economic Research, and Core Faculty Member for Health Policy/Center for Primary Care and Outcomes Research

Douglas K. Owens , MD, MS, Chair of the Department of Health Policy, Associate Director, Center for Innovation to Implementation (Ci2i) at Veterans Affairs Palo Alto Health Care System, Professor of Medicine (General Internal Medicine) and by courtesy, of Health Research and Policy, and Director of the Center for Health Policy at the Freeman Spogli Institute for International Studies.

Program Associate Directors:

Alyce Adams , PhD, Stanford Medicine Innovation Professor, Professor of Health Policy, Epidemiology and Population Health and of Pediatrics (by Courtesy), Associate Chair for Health Equity and Community Engagement for Stanford Health Policy, Associate Director for Health Equity and Community Engagement in the Stanford Cancer Institute, and Associate Director for Stanford Impact Labs

Steven Asch , MD, MPH, Professor of Medicine, Primary Care and Population Health, Director of the Center for Innovation to Implementation (Ci2i) and Chief of Health Services Research at VA Palo Alto Health Care System, Vice Chief, Division of Primary Care and Population Health

Sherri Rose , PhD, Professor of Health Policy and Co-Director of the Health Policy Data Science Lab

The Stanford-AHRQ Health Services Research Program has the mission of developing independent, well-trained, diverse researchers who conduct rigorous and innovative health services research (HSR) on critical challenges facing the U.S. health-care system. Supported by a grant from the U.S. Dept. of Health & Human Services’  Agency for Healthcare Research and Quality , pre- and post-doctoral fellows will work with mentors on their topics and methods of interest, as well as career development. The program includes 30 faculty mentors, drawn from 12 Stanford University departments or programs. Trainees will find opportunities to engage in research projects in a wide variety of HSR areas, including AHRQ priority areas like health insurance, quality and safety, affordability, and access.

Predoctoral trainees will earn a PhD in Health Policy or a related field, and postdoctoral fellows with a professional degree (e.g. MD) will commonly earn an MS in Health Policy. Postdoctoral trainees with a research degree will focus on research activities complemented by targeted electives. Trainees will pursue one or more independent research projects in their area(s) of interest, working with multiple mentors with complementary areas of expertise.

Our program includes faculty and other connections to Stanford’s strong and expanding capabilities in the areas of population health management, learning health-care systems, informatics, and health law.  We take advantage of recent investments in cutting-edge data and computing resources to support HSR; the creation of HSR-related collaborations with institutions like Intermountain Health, Kaiser Permanente, Google, and Facebook; and the presence of leading investigators in advanced computing, artificial intelligence, machine learning, textual processing — and their application to new data sources and types.  

The Health Services Research Training Program is motivated by our view that excellent health services researchers need a strong grasp of core disciplinary skills, the ability to function in multidisciplinary settings, and the ability to engage with both traditional and emerging research contexts.  Accordingly, our program emphasizes strong disciplinary training in our core areas of health economics, outcomes research, and decision science. We provide this training within the rich multidisciplinary environment of our integrated campus. Our trainees will frequently be side-by-side with trainees and faculty from areas like clinical medicine, economics, engineering, ethics, informatics, and law.  

Our graduating fellows will strengthen the next generation of diverse HSR leaders and will be equipped to generate, translate and disseminate the evidence needed to improve health-care delivery in the United States.

Stanford-AHRQ Fellowship Eligibility:

Applicants for the fellowships must, at the time of award, be citizens of the United States, non-citizen nationals, or non-citizens who have been lawfully admitted to the United States for permanent residence.  All trainees are required to pursue their research training full time.

Predoctoral  candidates must have earned a baccalaureate degree, and, at time of fellowship appointment, be enrolled in a program leading to a PhD or in an equivalent research doctoral degree program.  

Postdoctoral candidates must possess a PhD, MD, DDS, ScD, DrPH, DPharm, or equivalent doctoral degree (comparable doctoral degrees include, but are not limited to, the following:  DMD, DC, DO, DVM, OD, DPM, ScD, EngD, DrPH, DNSc, DPT, PharmD, ND (Doctor of Naturopathy), DSW, PsyD, as well as a doctoral degree in nursing research).  

Individuals in postgraduate clinical training, who wish to interrupt their studies for a year or more to engage in full-time research training before completing their formal training programs, are also eligible.

Eligible individuals applying to the Stanford Health Policy PhD Program will be considered for the  predoctoral fellowship .

Eligible individuals interested in the postdoctoral fellowship:

Applications for  post-doctoral  positions under the Stanford-AHRQ Health Services Research Training Program will be accepted  September 1st  through  April 1st .

Applications will normally be reviewed in the December to April window, and applicants can expect responses in the spring

Stanford Contact: SHP-Fellowship

Center for Health Policy / Center for Primary Care and Outcomes Research  Encina Commons  615 Crothers Way, MC 6019, Stanford, CA 94305  Email:  [email protected]

Please be prepared to upload a CV, statement of career objectives and the names of three professional references.

Welch Center for Prevention, Epidemiology and Clinical Research

• Our Research
• Clinical Research and Epidemiology in Diabetes and Endocrinology

Primary Care and Health Services Research

• Renal Disease Epidemiology Training Program

Program Director

Daniel E. Ford, MD, MPH

Professor of Medicine Vice Dean for Clinical Investigation Director, Johns Hopkins Institute for Clinical and Translational Research View Faculty Profile

The Primary Care and Health Services Research Training Program has been funded by a National Research Service Award (NRSA) from the Health Services and Resources Administration (HRSA) since 1994. The aims of this training program are to support the development of primary care physicians as creative and independent investigators in primary care research. The program enjoys a strong association with the Division of General Internal Medicine in the Department of Medicine. It provides an opportunity for junior physicians to develop the research skills in study design and analytic methods, and learn how primary care may function to integrate traditional curative medicine with public health initiatives to address the major health care problems facing Americans today. The focus is on multidisciplinary training to conduct independent research relevant to primary care and to excel as teachers and role models for the next generation of primary care researchers.

Most of the Healthy People 2010 objectives are addressed in this program, including physical activity and fitness, nutrition, tobacco, alcohol and other drugs, mental health and mental disorders, violence and abusive behavior, educational and community-based programs, unintentional injuries, heart disease and stroke, cancer, diabetes and chronic disabling conditions, and clinical preventive medicine. Additional areas of emphasis have been: primary care and mental disorders, minority health, interpersonal violence in primary care, quality of care in primary care offices, evidence-based practice, community-based primary care research opportunities, physicians’ health, implementation of clinical preventive guidelines in primary care practice, physician-patient communication, and integration of population-based prevention programs into primary care.

Most trainees have elected to complete the MPH degree program. They use the capstone experience to start their first research experience. Faculty from the Welch Center typically serve as research mentors for the trainees.

• Weill Cornell Medicine

Introduction to Health Services Research

Course Director: Jiani Yu, PhD

This course is designed to introduce students to the fundamentals of health services research. Health services research is the discipline that measures the evaluations of interventions designed to improve healthcare. These interventions can include changes to the organization, delivery and financing of healthcare and various healthcare policies. Common outcome measures in health services research include (but are not limited to) patient safety, healthcare quality, healthcare utilization, and cost. Specific topics to be covered in this course include: refining your research question, identifying common research designs and their strengths and weaknesses, minimizing bias and confounding, selecting data sources, optimizing measurement, and more. There will also be a component of the course that explores how to present your ideas and iteratively refine your work, based on feedback from peers and reviewers. This course includes both lectures and interactive group discussions. Students will be able to apply the methods learned in this course to their masters’ research projects.

• Our Experts
• Affiliated Centers & Programs
• Quality Collaboratives
• U-M Academic Partners
• Local Partners
• Evaluation Domains and Activities
• Reports and Publications
• National Poll on Healthy Aging
• Telehealth Research & Policy
• MPrOVE Research Innovation Challenge
• Health Equity
• Policy Briefs
• Aging & Medicare
• Child & Adolescent Health
• Climate Change & Health
• Healthcare Delivery & Financing
• Health Equity & Disparities
• Mental & Behavioral Health
• Opioid & Substance Use
• Prevention & Population Health
• Quality, Safety & Value
• Telehealth & Health IT
• Women's Health
• Opportunities at a Glance
• National Clinician Scholars Program at IHPI
• Advanced Training in HSR
• For IHPI Members
• For Undergraduate & Graduate Students
• Early Career Development Roadmap
• Become a Member
• Data & Methods Resources
• Grant Development Support
• Communications Support
• Policy Engagement
• Education and Career Development
• Research Networks
• Research Interest Groups
• Member Newsletter
• Space & Facilities
• Sponsorships
• National Advisory Board
• Leadership Team
• Early Career Faculty Advisory Council
• Support IHPI
• Research News
• Opinion / Commentary
• Announcements & Awards
• Annual Impact Reports
• Informs Newsletter
• 2023 Year in Review
• IHPI Research Seminars

Advanced Training in Health Services Research

Ihpi offers master's level training opportunities for clinicians to develop health services research and leadership skills that will position them to drive meaningful change within healthcare., ihpi offers master's level training opportunities for clinicians to develop health services research and leadership skills that will position them to drive meaningful change within healthcare..

National Clinician Scholars Program

• 2018-20 Scholars
• 2017-19 Scholars

IHPI Clinician Scholars

Master’s Scholars in Health and Healthcare Research Program

• 2017-18 Master’s Scholars

TACTICAL Scholars Program

• 2018-19 TACTICAL Scholars

U-M is one of six NCSP program sites providing opportunities for nurses and physicians to serve as full partners in healthcare redesign, improvement, and research through this two-year program. Read more about alumni from the program

The Master’s degree in Health and Health Care Research is available as a one-year program to clinician-researchers who are not part of the Clinician Scholars Programs. Read more about the program .

Research Training in the Biomedical, Behavioral, and Clinical Research Sciences (2011)

Chapter: 8 health services research, 8 health services research.

Health services research (HSR) provides the information needed to understand the effectiveness and efficiency of our health care delivery system and its impact on the health and well-being of individuals and populations. 1 Health services research documents deficiencies in patient and population health and in the provision of health services and seeks to identify contributing factors. There are many examples: being uninsured in America reduces access to health care and contributes to poorer health outcomes (Institute of Medicine [IOM] reports on uninsured); medical errors too frequently occur in hospitals and many patients suffer injury or death (IOM, 1999); and in the community, only half of the time are individuals receiving preventive and chronic disease care consistent with scientific evidence (McGlynn, 2003).

Health services researchers seek solutions to these and other problems that adversely affect access to care, quality, safety, and cost of care. Health services research evaluates the impact of government and private-sector health policies, designs and evaluates innovations in health care organization and financing, and examines the effects of new technologies or new uses of existing technologies. Assessing the impact of health services on population health requires health services researchers to go beyond disease outcomes to examine health status and health-related quality of life outcomes, assess delivery system quality and efficiency, as well as focus attention on prevention and health promotion services.

The contributions of health services research to policy, management, and clinical care have been diverse. Planners and policy makers, for example, are frustrated by the inability to generalize and use findings from efficacy studies: persons recruited to randomized control trials testing new treatments typically are not representative of the larger population expected to benefit from the treatment. Thus, it is up health services research to fill this information gap by assessing the impact of diagnostic and treatment technologies on patient outcomes and costs across real-world practice settings and diverse populations.

Translational research has emerged as an important dimension of health services research design and analysis; translational research provides the knowledge base to move scientific discoveries from laboratory, clinical, or population studies into clinical applications at the National Cancer Institute. Yet translation alone is generally not sufficient to ensure these services are available across America. Implementation research is needed to effectively adapt new clinical applications to diverse real-world practice settings in which programs, tools, and guidelines will be utilized and need to be integrated into the existing hospitals and community practice settings (Rubenstein and Pugh, 2006). Together translation and implementation research are gaining greater visibility as we have come to recognize that many Americans are failing to receive consistent high-quality health care based on the latest scientific knowledge. Meeting the challenges of translation and implementation research requires additional disciplinary breadth, drawing on areas of organizational and operations research, psychology, marketing, education, and adult learning. Also expanded applications of health information technology are needed in support of consumer-patient decision making and real-time decision support for health care providers. The rapid growth and continuing change in scientific health information will result in the translation and implementation processes being continual and not one-time or infrequent events. The capacity to achieve this goal may require fundamental re-thinking of information flow and how it supports all aspects of health services.

Central to advances in all scientific fields are measurement tools, and for health services research measurement tools span payment and financing, appropriateness of utilization (overuse, underuse, and misuse; IOM, 2001), quality of care, and patient outcomes of care. Health services research

has provided the measurement tools being used in payment for inpatient hospital services, outpatient services, and nursing home care, as well as capitation payment methods for persons enrolled in health plans. Improved payment methods are making it possible to adjust payment for quality of care and to better reward efficiency. These measurement tools, and others to be developed, will be needed to monitor and evaluate the impact of the 2010 Health Reform legislation and how well it achieves its goals. Examples of quality-of-care measures that will require further development include: assessing the timeliness of health care, measuring coordination of patient care when multiple providers are involved in diagnosis and treatment, providing patient-centeredness of care, and equity of health care. Although these are not new, there are few if any accepted measurement tools to assess deficiencies and progress toward the goals of health reform. The training and support of researchers who focus on measurement is a continuing and growing need in health services research.

Since 2003 Congress has provided support to the Agency for Healthcare Research and Quality (AHRQ) to develop and fund comparative effectiveness research (CER). In 2009, the American Recovery and Reinvestment Act (ARRA) augmented CER support with $1.1 billion for research and training through AHRQ, the National Institutes of Health, and the Office of the Secretary of Health and Human Services (HHS). CER as defined by HHS combines key elements of health services and clinical research:

Comparative effectiveness research is the conduct and synthesis of research comparing the benefits and harms of different interventions and strategies to prevent, diagnose, treat and monitor health conditions in “real world” settings. The purpose of this research is to improve health outcomes by developing and disseminating evidence-based information to patients, clinicians, and other decision-makers, responding to their expressed needs, about which interventions are most effective for which patients under specific circumstances.

To provide this information, comparative effectiveness research must assess a comprehensive array of health-related outcomes for diverse patient populations and sub-groups.

Defined interventions compared may include medications, procedures, medical and assistive devices and technologies, diagnostic testing, behavioral change, and delivery system strategies.

This research necessitates the development, expansion, and use of a variety of data sources and methods to assess comparative effectiveness and actively disseminate the results.

The expectation is that CER will provide new information that is not currently available about what treatments and services work best for individuals across America’s diverse populations, taking into consideration the person’s circumstances and the timing of services. The new CER mandate complements the initiatives discussed above in translation and implementation research, intensifying the focus on research driving health system transformation to achieve better health outcomes for all Americans and greater efficiency.

FEDERAL SUPPORT OF HEALTH SERVICES RESEARCH

In 1968, Congress recognized the emerging role of health services research for improving health care delivery in the United States and created the National Center for Health Services Research and Development (NCHSRD) in the Department of Health, Education, and Welfare (DHEW). During the years 1968-1989, NCHSRD sought to develop research on issues of access, cost, and quality, and to develop data systems to support research on utilization and cost of care. 2 However, over time the budget for NCHSRD declined and the future of the NCHSRD became uncertain. Private foundations played a critical role in sustaining the health services research field during these years. 3

In 1989, health services research once again found strong support in Congress and a new vision for health services research was created in the authorization of the Agency for Health Care Policy and Research (AHCPR). Congress directed the Agency—subsequently renamed the Agency for Healthcare Quality and Research—to undertake research on patient outcomes, develop practice guidelines, and disseminate the research to change the practice of medicine. 4 The agency placed greater emphasis than previously on the examination of clinical practice, decision making, and comparative effectiveness of alternative approaches to diagnosis and treatment. The funding for AHRQ has grown over the years from $128 million in fiscal year 1993 to $397 million in fiscal year 2010, plus $300 million in CER funding from the ARRA appropriation.

While the National Research Service Awards (NRSA) program included support for health services research from its inception (see, for example, NRC, 1977), Congress specified in 1989 that one-half of 1 percent of the NRSA budget for training be allocated for training health services researchers through AHRQ, subsequently expanding that

allocation to 1 percent of NRSA funding in 1999, which has remained unchanged.

It should be noted that in the early 1990s Congress authorized a 15 percent set-aside for both research and NRSA training in service-related research supported by the National Institute of Mental Health (NIMH), the National Institute of Drug Abuse (NIDA), and the National Institute of Alcohol Abuse and Alcoholism (NIAAA) as part of the reorganization of the former Alcohol, Drug Abuse and Mental Health Administration into the National Institutes of Health. Even with this congressionally mandated set-aside for these NIH institutes, AHRQ remained the lead agency for health services research. NIH funding has been directed at HSR focused on questions related to the delivery of health care for specific diseases/disorders. AHRQ and NIH fund complementary research and in many instances have co-funded major health services research studies.

HEALTH SERVICES RESEARCH WORKFORCE

No national statistical system reports on the size and composition of the health services research workforce (Moore and McGinnis, 2009; Pittman and Holve, 2009). Obtaining information on the workforce in this field is a challenge. Identifying scientists who primarily do health services research is complicated by the interdisciplinary nature of the field. Health services research is an applied field, and so most health services researchers have another unique discipline or profession that they bring to health services research. Workforce data usually classify health services researchers by their primary discipline or profession and often are unable to identify the field of scientific inquiry as health services research. As NIH moves more toward trans-disciplinary research, the problem of not having multiple classifications incorporating both discipline and field of application may be an issue faced by many basic sciences and clinical researchers, as well as health services research.

In addition, anecdotal evidence suggests that some investigators involved in health services research studies do not identify themselves as health services researchers, nor do they necessarily belong to the only national professional association in this area, namely AcademyHealth. This partial or part-time involvement of many scientists in health services research only further complicates efforts to estimate the size and composition of the health services research workforce.

McGinnis and Moore addressed this issue in their study on the current status of the health services research workforce. In a conservative estimate of the field, counting HSRProj investigators (since 2004), speakers from AcademyHealth’s Annual Research Meeting in 2007, and AcademyHealth members whose membership has lapsed or joined in 2000 or later, Moore and McGinnis found that the field has more than doubled in size since the IOM’s estimate in 1995, growing from approximately 5,000 health services researchers to more than 13,000 researchers in 2007. Using a more expansive definition of the field by including researchers in disciplinary associations with subgroups that sometimes do health services research, such as the American Public Health Association, the American Society of Health Economists, the American Statistical Association, and the American Sociological Association, there could be an additional 6,000 intermitted members of the field (Moore and McGinnis, 2009).

The best data available on the composition of health services research workforce 5 likely comes from the most recent AcademyHealth membership survey in 2008 (AcademyHealth, 2008). AcademyHealth draws its members from both health services research and health policy, and includes student memberships. Although this database more than likely underestimates the total size of the workforce, it does provide some insights into its composition.

As of 2008, 51 percent of AcademyHealth’s 3,500 individual members report having a Ph.D., Sc.D., or other doctoral-level training in science. There are another 12 percent reporting an M.D. Table 8-1 shows the distribution of health services researchers by employment sector.

AcademyHealth membership has greater female representation (60.7 percent) than male (39.3 percent). This representation has changed slightly from AcademyHealth’s survey of members in 2002, when 55 percent of the respondents were women and 45 percent were men. Of note is that the youngest members were twice as likely to be female as to be male, while the oldest respondents were twice as likely to be male as to be female. The ethnic mix of members is 21 percent from minority ethnic backgrounds, including Asian/Pacific Islanders (10.6 percent), African Americans (5.2 percent), and Hispanics/Latinos (2.6 percent), plus 79 percent Caucasian and 2.5 percent other. Representation of all minorities has increased since 2002—to 21 percent from 12.8 percent.

Table 8-2 shows the primary field of interest by the members of AcademyHealth, and the largest share of the members classify their primary discipline as public health (21.5 percent). Only 13.3 percent of members identify their primary discipline as health services research.

In a study on the demand for health services researchers, Thornton and Brown (2009) found that the demand from both universities and non-academic employers is expected to increase. Based on their work one can anticipate there will be a growing demand for “people who can analyze the effectiveness of health service systems from disease management firms; investment firms with a large stake in the health care sector; state and local government; hospitals and providers that will be implementing quality reporting systems and pay-for-performance systems;” and the health

TABLE 8-1 Setting of Primary Employment, 2008

TABLE 8-2 Primary Field of AcademyHealth Members, 2008

industry including equipment manufacturers, pharmaceutical firms, and insurers.

Graduate Programs in Health Services Research

Graduate programs in health services research are not separately accredited, and because many graduates could come from doctoral programs with a different specialty than health services research, there is no accurate tally of doctoral students earning degrees in health services research (Ricketts, 2009). However, in its 2009 online directory of master’s and doctoral programs in HSR, AcademyHealth reports that there are now 41 schools providing HSR doctoral programs and 22 schools with postdoctoral training programs. Doctoral programs are mainly Ph.D. programs, including both disciplinary (e.g., health economics, medical sociology) and general training in health services research. An example of additional training opportunities is illustrated by Veterans Administration’s description of a new fellowship program:

VA Advanced Fellowship Program in Health Services Research and Development (HSR&D): This includes 16 training sites for Ph.D. associated health professionals, 8 training sites for post-residency physician associated health professionals, and 3 sites for post-doctoral physician associated health professionals. HSR&D also participates in the VA Advanced Fellowship Program in Medical Informatics which includes 7 training sites for post-doctoral and physician health professionals in medical informatics.

The NRSA program provides support for training in health services research. As discussed above, the AHRQ has received funding equal to one percent of all NRSA funds for NIH. AHRQ supplements NRSA funding with $500,000 annually. As shown in Table 8-3 , both NIH and AHRQ are funding HSR training at predoctoral and postdoctoral levels. Taken together, there were 107 predoctoral training positions in 2008, 68 percent of them funded by AHRQ. There were also a total of 85 postdoctoral positions, of which 49 percent were funded by AHRQ. The agency accepts new and renewal training grant applications every 5 years. In general, the agency has been able to fund only two-thirds of the requested training positions, and this is very similar to the rate for all NIH training awards. In addition, several NIH institutes provide NRSA awards in health services research, including NIMH, NIAAA, and NIDA. Overall, the total number of trainees is likely less than 2 percent of all NRSA training positions. No data are available on graduates of doctoral programs who are not funded by the NRSA program but who plan to pursue health services research careers. It would be expected that these numbers far exceed NRSA recipients, as they do in other health research fields.

Although there is incomplete information on the characteristics and careers of all individuals with training in health services research, there is some information of NRSA trainees supported by AHRQ. In particular, AHRQ commissioned an outcome study in 1999 of NRSA trainees between 1986 and 1997, which used information from the curricula vitae (CV) of the traimees. The results of this study were reported in the last assessment of the NRSA program. These data were updated in 2005 when data on trainees from 1998 to 2003 were added and data on the earlier trainees were made current to 2003. From 1986 to August 2003, AHRQ supported more than 1,000 individuals through different funding mechanisms. The NRSA program T32 institutional awards supported 346 predoctoral and 435 postdoctoral trainees through 27 university-based or university-affiliated training sites. Another 81 AHRQ F32 individual NRSA postdoctoral fellowships and 5 predoctoral fellowships were awarded. Some individuals had multiple awards under different mechanisms. A total of 854 individuals had support.

TABLE 8-3 Health Services Research Training Positions Funded by AHRQ and the NIH

In 2000, AHRQ launched its career development (K) award program and by August 2003 had made 48 awards. The majority of AHRQ-supported NRSA trainees and fellows between 1986 and 2003 were female (502 of 854, or 59 percent), a difference especially evident among T32 predoctoral trainees (229 of 346, or 66 percent) and F32 fellows (45 of 76, or 58 percent). There were almost even numbers of males (203) and females (225) with T32 postdoctoral trainees during this period.

The CVs of 709 trainees provided information on career progression and research productivity. CVs were received from 850: 346 had T32 predoctoral support, 428 had T32 postdoctoral support, and 76 had F32 fellowships. Of those who earned a doctorate by 2003, about 75 percent or 244 of the doctorates with a known degree field earned their doctorate in a health science field, including: health services research (81); related multidisciplinary health fields such as health policy, health administration, or public health (118); or one of the other health sciences (45). Over 90 percent of the T32 predoctoral trainees earned their baccalaureate degrees in one of the sciences, with 42 percent in the social sciences, 15 percent in the health sciences, and 19 percent in other scientific fields, including the physical and mathematical sciences. The degrees of those with baccalaureate degrees in non-sciences were either in education, humanities, or professional fields. Length of time in training for T32 and F31 predoctoral students averaged about 20 months, but 36 percent were only in training for 12 months. There was some difference in length of training by gender, with 81 percent of females in training for 24 months or less and 75 percent of males for this period. At the postdoctoral level, 84 percent of F32 fellows were in training for 24 months or less, and 86 percent of the T32 awardees were in training for this period. For both the T32 and F32 trainees, about half were in training for 24 months.

Half of the AHRQ NRSA T32 postdoctoral trainees with research doctorates earned them in the social sciences (sociology, economics, or the other social sciences); the remainder earned them in a variety of health or other fields. The other half of the AHRQ NRSA T32 postdoctoral trainees had clinical doctorates, and about half of these were earned in internal medicine; another 20 percent were earned in pediatrics and another 6 percent were earned in family practice, with the remainder earned in a wide variety of other clinical specialties. About 20 percent, or 59, of the 241 clinical doctorates with CV information earned a joint M.D./Ph.D. Just over half of the AHRQ NRSA F32 fellows held clinical doctorates, and more were in internal medicine.

The study also showed that the AHRQ NRSA trainees and fellows actively pursue research careers through a variety of employment paths. Most AHRQ NRSA T32 predoctoral trainees who completed their doctorates by 2003 did not pursue formal postdoctoral research training. First employment data were available for 555 of the predoctoral and postdoctoral trainees, and a large majority of both groups where employed in academic institutions. For the postdoctorates, 71 percent of 382 trainees were in academe, 23 percent were in for-profit or non-profit organizations, and 5 percent were in government. Of those in academic positions, 76 percent were Ph.D.s and 72 percent had clinical degrees. Most of the clinical doctorates that complete training began their academic career as an instructor. The percentage for the 165 predoctoral trainees formed a similar pattern, but only 57 percent had an academic position and 29 percent were in for-profit or non-profit organizations. The current employment of postdoctoral trainees at the end of 2003 closely resembles their first employment with 79 percent in academic positions and 13 percent in health-related employment. The remaining 8 percent were in for-profit or other organizations. For T32 predoctoral trainees, academic employment was almost as high at 67 percent, with 21 percent in health-related employment and the remaining 11 percent in for-profit or other organizations.

Of the employed NRSA T32 predoctoral trainees, about half (48 percent) reported having received post-training research support, and about 77 percent reported at least one post-training scientific journal publication. For NRSA T32

postdoctoral trainees with research doctorates, 72 percent reported having received post-training research support, and 85 percent listed at least one scientific journal publication following training. About 60 percent of the employed NRSA T32 postdoctoral trainees with clinical doctorates reported having received post-training research support, and about 78 percent had at least one scientific journal publication following training. Two-thirds of the employed former AHRQ NRSA F fellows reported having received grant support. In general, 90 percent of all trainees had at least one post-training scientific journal publication.

FEDERAL HEALTH SERVICES RESEARCH FUNDING

The broad relevance of health services research has contributed to federal funding through multiple agencies, unlike the funding of most other areas of health research. AHRQ’s research is expected to address cross-cutting issues such as access, quality and cost issues that are faced by the entire American health care system. Other funding sources seek to fund health services research in support of their organizational missions. The VA and DoD focus on their delivery systems, CMS on financing Medicare and Medicaid, CDC on prevention, and the NIH on delivery of services for specific diseases. These funding sources are complemented by private sources, including major foundations (e.g., Robert Wood Johnson Foundation, Commonwealth Fund, MacArthur Foundation, Kellogg Foundation, Kaiser Family Foundation, and a number of state-based foundations) and private corporations. The following discussion will be limited to federal funding of health services research.

In 2001 the Coalition for Health Services Research (CHSR), the advocacy affiliate of AcademyHealth, began an initiative to document health services research funding levels across the federal government. The first report was completed in 2003 and now there are annual updates. As of FY 2009, the Coalition estimates that a total of $1.48 billion was expended for health services research and related activities by the federal government in as shown below:

Agency for Healthcare Research and Quality (AHRQ)—$372 million;

Centers for Disease Control and Prevention (CDC):

National Center for Health Statistics (NCHS)—$125 million;

Extramural Prevention Research Program—$31 million;

Prevention Research Centers—$31 million;

Centers for Medicare and Medicaid Services (CMS)—$39 million; 6

Health Resources and Services Administration (HRSA)—$9 million;

National Institutes of Health (NIH) (All Institutes)—$779 million;

Veterans Health Administration (VHA)—$75 million; and

The Department of Defense (DoD)—$17 million.

Despite repeated calls from the Coalition for Health Services Research that federal agencies use a standard definition or uniform categories to report their expenditures, the data presented above are measured by these agencies using their own unique definition for what constitutes health services research. Only with a uniform definition and standard categories, would it be possible to assess how the current funding meets emerging needs.

Comparing the health services research funding of $1.5 billion to total federal health research funding of $35 billion in 2005 (Global Forum for Health Research, 2005) shows that approximately 4 percent of total funding is being devoted to health services research, based on classifications used within each agency and institute.

NIH institutes report funding health services research as shown in Table 8-4 . NIMH, NIDA, and NCI have the largest programmatic commitment, ranging from 17 to 23 percent of budget. Other institutes report smaller commitments of budget to health services research.

In summary, AHRQ provides 25 percent of all health services research funding as reported by federal agencies. Other federal agencies support more focused program-specific and disease-specific health services research. Private funding of health services research is substantial but no comprehensive source of information is available on non-federal sources.

CAREERS IN HEALTH SERVICES RESEARCH

The employment opportunities and careers in health services research are widely varied. Academic careers may be in schools of medicine, nursing, public health, and other health professional schools, as well as engineering and traditional arts and sciences departments, along with business and public policy schools. To effectively manage interdisciplinary research, academic institutions usually have organizational structures such as centers or institutes for health services research that cross school and departmental boundaries. At some institutions there are multiple centers reflecting different areas of specialization and the availability of funding for specialized centers from federal and private sources.

Private-sector health services research careers are available in many areas. Federal contract work evaluating major public policy initiatives are primarily done by private research firms. These organizations include RAND, Mathematica, Abt Associates, Westat, and others. These organizations are organized to do short-term large-scale studies that are not as easily organized and managed in most academic settings.

TABLE 8-4 NIH Institute Health Services Research Budgets Health Services Research FY 2008 Estimate (Dollars in Thousands)

Other private-sector health services research careers are in research organizations sponsored by HMOs and health plans, hospital systems, pharmaceutical firms, insurers, and other major stakeholders in health care. Health services research positions may involve directing research, translating research into practice and products, and managing and evaluating health care operations.

Associations for professional groups, manufacturers, and advocacy groups recruit people trained in health services research to strengthen their capacity to use information com-ing from health services research to advance their advocacy objectives and meet the needs of their members. As efforts to translate science into practice accelerate, the demand for individuals skilled in health services research and communication to users is likely to grow.

Government agencies recruit substantial numbers of health services research professionals to lead and manage research programs, to support policy analysis and development, and to work with managers and providers in the VA and DoD health care delivery systems.

New career paths for health services research professionals may emerge as research into effective translation of knowledge into practice grows. The 2003 Medicare prescription drug legislation mandated in Section 1013 that comparative effectiveness studies of health care services including prescription drugs increased the need for health services researchers trained in pharmaco-economics. The ARRA provided a substantial increase in CER funding for both research and investment in research infrastructure including methods and data. The development of tools and techniques to support translation is likely to become an industry that will require research skills in the design, evaluation, and testing of new technologies. Translation of knowledge for clinicians may be the initial priority, but priorities will likely expand to include managers, patients, and the public. The passage of the 2010 Affordable Health Care Act for America brings new and increased demands to monitor the success of health reform and identify unintended consequences. To achieve goals of greater efficiency in American health care and better quality, additional investments in health services research and translation and implementation in practice will be needed. The future demand for well-trained health services researchers is currently strong and growing.

RECOMMENDATIONS

Recommendation 8–1: Health services research training should be expanded and strengthened within each NIH institute and center.

Biomedical research has created a growing gap between research advances in biomedical science and the ability to apply them effectively to improve the health of the public. Thus there is a need for more effective health care delivery practices to ensure effective and evidence-based care, and to reduce waste and unnecessary risk to patients.

Recommendation 8–2: AHRQ training programs should be expanded, at a minimum commensurate with the growth in total federal spending on health services research, including comparative effectiveness research.

Recognition of the rising costs of care, with concerns about quality and consistency, have driven increases in services research. Health services research has established an important evidence base to enable patients and health care organizations to evaluate benefits and risks of diagnostic and therapeutic intervention and to compare relative values of older and newer approaches as choices proliferate. This field can also evaluate different approaches to health care delivery and financing, which will allow the nation to get more benefit from the dramatic advances in biomedical science. Ideally, the total numbers of persons being trained in HSR should grow at the same rate as national health

care expenditures. The NRSA program provides funding for a fraction of all trainees, which is divided among NIH institutes and AHRQ. Since NRSA funding is expected to ensure an adequate supply of research personnel for health research, it is reasonable to expect the proportion of NRSA funding for HSR trainees to approximate the proportion of the federal health research that is HSR. This guideline suggests the need to roughly double NRSA funding of HSR training, from the current level of approximately 2 percent of NRSA funds to 4 percent.

AcademyHealth. AcademyHealth Membership Survey 2008. Internal document.

Bergner, M., R.A. Bobbitt, W.B.Carter, and B.S. Gilson. 1981. The sickness impact profile: development and final revision of a health status measure. Medical Care 19:787-805.

Chassin, M.R., J. Kosecoff, R.E. Park, et al. 1987. Does inappropriate use explain geographic variations in the use of health care services? A study of three procedures. Journal of the American Medical Association 258:2533-2537.

Coalition for Health Services Research. 2009. Federal Funding for Health Services Research. Washington, DC.

Gertman, P.M., and J.D. Restuccia. 1981. The appropriateness evaluation protocol: A technique for assessing unnecessary days of hospital care. Medical Care 19:855-871.

Global Forum for Health Research. 2005. “10/90 Gap.” See http://light.globalforumhealth.org/About/10-90-gap .

Hussey, P.S., G.F. Anderson, R. Osborn, C. Feek, V. McLaughlin, J. Millar, and A. Epstein. 2004. How does the quality of care compare in five countries? Health Affairs 23(3): 89-111.

Institute of Medicine. 1995. Health Services Research: Work Force and Educational Issues. Washington, DC: National Academy Press.

Institute of Medicine. 1999. To Err Is Human: Building a Safer Health System . Washington, DC: National Academy Press.

Institute of Medicine. 2001. Crossing the Quality Chasm: A New Health System for the Twenty-first Century . Washington, DC: National Academy Press.

Institute of Medicine. 2002. Unequal Treatment: Confronting Racial and Ethnic Disparities in Healthcare. Washington, DC: National Academy Press.

Kaplan, R.M., and J.P. Anderson . 1988. The quality of well-being scale: rationale for a single quality of life index. In: S.R. Walker and R. M. Rosser, eds. Quality of Life Assessment and Application. Lancaster, England, 51-78. MTP Press,

Lohr, K.N., and D.M. Steinwachs. 2002. Health services research: an evolving definition of the field. Health Services Research 37(1):7-9.

McGlynn E.A., et al. 2003. The quality of health care delivered to adults in the United States. New England Journal of Medicine 348(26):2635-2645.

Moore, J., and S. McGinnis. 2009. The health services research workforce: current stock. Health Services Research 2009; 44(6).

National Research Council. 1976. Personnel Needs and Training for Biomedical and Behavioral Research. Washington, DC: National Academy Press.

National Research Council. 1978. Personnel Needs and Training for Biomedical and Behavioral Research. Washington, DC: National Academy Press.

National Research Council. 1983. Personnel Needs and Training for Biomedical and Behavioral Research. Washington, DC: National Academy Press.

National Research Council. 1989. Biomedical and Behavioral Research Scientists: Their Training and Supply. Volume I: Findings. Washington, DC: National Academy Press.

National Research Council. 1995. Meeting the Nation’s Needs for Biomedical and Behavioral Scientists. Washington, DC: National Academy Press.

Pittman, P., and E. Holve. 2009. The health services researcher of 2020: a summit to assess the field’s workforce needs. Health Services Research 44(6): 2198-2213.

Reinhardt, U.E., P.S. Hussey, and G.F. Anderson. 2002. Cross-national comparisons of health systems using OECD data, 1999. Health Affairs 21(3):10-25.

Ricketts, T. 2009. Preparing the health services research workforce. Health Services Research 2009; 44(6).

Thornton, C., and J. Brown. 2009. Employer demand for health services researchers in the year 2020. Health Services Research 2009; 44(6).

Ware, J.E., B. Gander, and the IQOLA Project Group.1994. The SF-36 health survey: development and use in mental health research and the IQOLA project. International Journal of Mental Health 23(2):49-73.

Wennberg, J.E., N.R. Roos, L.Sola, A. Schori, and R. Jaffe. 1987. Use of claims data systems to evaluate health care outcomes: Mortality and reoperation following prostatectomy. Journal of the American Medical Association, 257(7):933-936.

Comprehensive research and a highly-trained workforce are essential for the improvement of health and health care both nationally and internationally. During the past 40 years the National Research Services Award (NRSA) Program has played a large role in training the workforce responsible for dramatic advances in the understanding of various diseases and new insights that have led to more effective and targeted therapies. In spite of this program, the difficulty obtaining jobs after the postdoc period has discouraged many domestic students from pursuing graduate postdoc training. In the United States, more than 50 percent of the postdoc workforce is made up of individuals who obtained their Ph.D.s from other countries. Indeed, one can make a strong argument that the influx of highly trained and creative foreigners has contributed greatly to U.S. science over the past 70 years.

Research Training in the Biomedical, Behavioral, and Clinical Research Sciences discusses a number of important issues, including: the job prospects for postdocs completing their training; questions about the continued supply of international postdocs in an increasingly competitive world; the need for equal, excellent training for all graduate students who receive NIH funding; and the need to increase the diversity of trainees. The book recommends improvements in minority recruiting, more rigorous and extensive training in the responsible conduct of research and ethics, increased emphasis on career development, more attention to outcomes, and the requirement for incorporating more quantitative thinking in the biomedical curriculum.

READ FREE ONLINE

Welcome to OpenBook!

You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

Do you want to take a quick tour of the OpenBook's features?

Show this book's table of contents , where you can jump to any chapter by name.

...or use these buttons to go back to the previous chapter or skip to the next one.

Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

Switch between the Original Pages , where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

To search the entire text of this book, type in your search term here and press Enter .

Share a link to this book page on your preferred social network or via email.

View our suggested citation for this chapter.

Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

Get Email Updates

Do you enjoy reading reports from the Academies online for free ? Sign up for email notifications and we'll let you know about new publications in your areas of interest when they're released.

Your Browser is Out of Date

To provide you with the best experience, our websites uses features that are not supported by your current browser. For the best experience on Childrenshospitals.org, please open this site in a different browser such as Google Chrome

• Product/Program

Health Services Research Academy

Led by internationally recognized and awarded researchers Matt Hall, Ph.D. (principal biostatistician, Children’s Hospital Association) and Jay Berry, M.D., M.P.H. (chief, complex care, Boston Children’s Hospital; associate professor of pediatrics, Harvard Medical School) with over 20 years of experience, the Children’s Hospital Association Health Services Research Academy is a hands-on training program that covers every aspect of a professional research project, from conception to publication.

Through online courses and one-on-one, didactic teaching sessions, Academy participants acquire academic-level skills to design and conduct research studies, leverage and analyze large administrative health data, and disseminate important research findings through high-quality presentations and peer-reviewed manuscripts.

Since starting in 2019, there have been nearly 300 Academy participants from 54 U.S. hospitals and academic institutions, representing a variety of clinical fields (e.g., hospital medicine, complex care, surgery and nursing).

I was able to take the lead on a project and produce multiple deliverables, including platform presentations at national meetings and publication in a reputable peer-reviewed journal. Andrew Yu, M.D., Children's Health System of Texas

Whether new to research or wanting to advance existing research skills, the Academy can help.

The Academy offers:

• Twenty virtual, on-demand research modules covering a variety of topics, including building a research team, review of existing literature, development of specific aims and analytical plans, statistical analysis, exhibit generation and manuscript preparation.
• One-on-one coaching, teaching and mentoring with health services research experts and editors of leading scientific journals in pediatrics.
• Flexibility to take courses anywhere, anytime, at your own pace.
• Research project resulting in at least one peer-reviewed publication.
• Up to 11.5 AMA PRA Category 1 Credit(s)™ for physicians and 11.5 ANCC contact hours for nurses.
• Certificate of Excellence in Research Scholarship.

Four faculty from my division went through this program and have presented their work at national meetings and published manuscripts in peer-reviewed journals. Vineeta Mittal, M.D., MBA, UTSW Medical Center and Children's Health

There are multiple options for Academy participation.

Research modules only

Two-year access to 20 interactive research education modules, approximately 30 minutes to one hour in length each. Read the course description for the topics covered.  Students, residents and fellows, as well as new research assistants and program managers, have found this option particularly helpful.

Research modules and coaching

Two-year access to 20 interactive research education modules as well as one hour of research coaching monthly for one year from a senior statistician and senior health services researcher. Young clinicians new to research who are seeking support to enhance and complement their local research project and mentor team have found this option helpful.

Research modules and full mentoring and statistical support

Two-year access to 20 interactive research education modules as well as full mentoring and statistical support by internationally recognized health services researchers to design, implement, complete and publish a research project. The support team meets via conference calls with the Academy participant (and any other local research team members) at least twice monthly to steward the project.

Participants will gain access to an array of national, pediatric health care utilization and outcomes data. New and mid-level clinicians and faculty with ideas about the topic and gist of a research project have found this option most helpful. Career development awardees have also used this option in their successfully funded grant applications.

Individualized academy experience for an institution or group

The Academy resources described across the options above are available at the institutional, department/division and/or research group levels to support the development of research skills across multiple individuals. For example, divisions of pediatric hospital medicine have found it helpful to (a) enroll their research assistants in the research module only option, (b) enroll mid-level faculty in the research module and coaching option, and (c) enroll junior faculty in the full mentoring and statistical support option. Please contact us  to discuss how we can best serve you.

Click here to learn more about associated costs and mentors.

Frequently Asked Questions

What makes the academy unique.

We designed the research curriculum and experience for learners so that they will acquire and/or enhance their research skills from start-to-finish with a research project. The topics covered are practical: how to set up a research team, create specific aims, devise an analytical plan, design a beautiful figure for a manuscript, manage the bibliography, respond well to peer-review comments and suggestions, etc.

The learning opportunities are purposely flexible. Academy participants choose when and how to progress through the curriculum and support offerings. We understand that it can be very challenging to find time to learn about and conduct research amongst existing clinical, administrative, family and life responsibilities.

Academy participants choose their research topic and project based on their interests and needs. Academy faculty will support and enable any clinical research idea proposed by a participant. Examples of recent research conducted by Academy participants include:

• McDaniel CE, Hall M, Markham JL, Bettenhausen JL, Berry JG. Urban-Rural Hospitalization Rates for Pediatric Mental Health. Pediatrics. 2023 Jun 1;151(6):e2023061256. doi: 10.1542/peds.2023-061256. PMID: 37139706.
• Friesen TL, Hall M, Ramchandar N, Berry JG, Jiang W. Evolving Management of Acute Mastoiditis: Analysis of the Pediatric Health Information System  Database. Otolaryngol Head Neck Surg. 2023 Aug;169(2):382-389. doi: 10.1002/ohn.286. Epub 2023 Feb 7. PMID: 36939424.
• Sharp EA, Wang L, Hall M, Berry JG, Forster CS. Frequency, Characteristics, and Outcomes of Patients Requiring Early PICU Readmission. Hosp Pediatr. 2023 Aug 1;13(8):678-688. doi: 10.1542/hpeds.2022-007100. PMID: 37476936; PMCID: PMC10375031.
• Attard TM, Hall M, Berry JG. Improving Cohort Definitions in Research Using Hospital Administrative Databases-Do We Need Guidelines? JAMA Pediatr. 2022 Jun 1;176(6):539-540. doi: 10.1001/jamapediatrics.2022.0091. PMID: 35312756.
• Kim G, Berry JG, Janes JL, Perez A, Hall M. Association of Maternal Tdap Recommendations With Pertussis Hospitalizations of Young Infants. Hosp Pediatr. 2022 Mar 1;12(3):e106-e109. doi: 10.1542/hpeds.2021-006323. PMID: 35132433.
• Bayer ND, Hall M, Li Y, Feinstein JA, Thomson J, Berry JG. Trends in Health Care Use and Spending for Young Children With Neurologic Impairment. Pediatrics. 2022 Jan 1;149(1):e2021050905. doi: 10.1542/peds.2021-050905. PMID: 34854922; PMCID: PMC8762668.
• Hoffmann JA, Hall M, Lorenz D, Berry JG. Emergency Department Visits for Suicidal Ideation and Self-Harm in Rural and Urban Youths. J Pediatr. 2021 Nov;238:282-289.e1. doi: 10.1016/j.jpeds.2021.07.013. Epub 2021 Jul 15. PMID: 34274309; PMCID: PMC8551015.

Who can apply and enroll?

Anyone with an interest to learn and acquire health services research skills. Physicians, nurses and research assistants are welcome. Students, residents, fellows and faculty of all academic levels are welcome. No prior research experience is necessary.

How to apply?

Submit a brief application to apply. An Academy representative will follow up to discuss your background, research area of interest and the research skills you want to acquire.

What is the commitment?

We tailor the commitment to meet the needs of each Academy participant. Although the amount of time and effort required for Academy completion varies, most participants finish in one year.

Can I incorporate the Academy with another research education program?

Yes, the Academy works as a stand-alone training program or in conjunction with another program. For example, Academy participants have incorporated Academy training into coursework for a Master of Public Health or Science. Other participants have incorporated the Academy with residence and fellowship training. Multiple participants have successfully incorporated Academy training with NIH and institutional career development awards.

I have a local mentor and local research collaborators. Can they be involved in my Academy research project?

Absolutely. In fact, we heavily encourage all Academy participants to include local mentors, collaborators and other research personnel in their projects and learning experience.

I don’t have a local mentor or any local research collaborators. Does that mean that I can’t enroll in the Academy?

Absolutely not. The Academy is positioned to support any individual, regardless of whether they have a local team and resources to help.

Does the Academy provide any scholarships?

To value and increase the quality and quantity of health services research led by members of racial and ethnic groups that have historically been underrepresented in research, the Academy offers one scholarship annually. Click here to find out more.

For more information, connect with us.

Application

Share your background, research area of interest and the research skills you want to acquire.

Health Services Research

CHA Research Network

Coding Toolkits

Children’s Hospital Association supports health services research in a variety of ways.

The CHA Research Network leverages the strength of CHA’s data and analytics to perform pediatric health services research and policy relevant research.

CHA publishes specific codes needed to classify specific pediatric illnesses, disorders, and needs.

An official website of the Department of Health and Human Services

Browse Topics

Priority populations.

• Children/Adolescents
• Racial/Ethnic Minorities
• Rural/Inner-City Residents
• Special Healthcare Needs
• Clinicians & Providers
• Data & Measures
• Digital Healthcare Research
• Education & Training
• Hospitals & Health Systems
• Prevention & Chronic Care
• Quality & Patient Safety

Publications & Products

• AHRQ Publishing and Communications Guidelines
• Search Publications

Research Findings & Reports

• Evidence-based Practice Center (EPC) Reports
• Fact Sheets
• Grantee Final Reports: Patient Safety
• Making Healthcare Safer Report
• National Healthcare Quality and Disparities Reports
• Technology Assessment Program
• AHRQ Research Studies

National Healthcare Quality and Disparities Report

Latest available findings on quality of and access to health care

Data & Analytics

• Data Infographics
• Data Visualizations
• Data Innovations
• All-Payer Claims Database
• Consumer Assessment of Healthcare Providers and Systems (CAHPS®) Program
• Healthcare Cost and Utilization Project (HCUP)
• Medical Expenditure Panel Survey (MEPS)
• National Healthcare Quality and Disparities Report Data Tools
• AHRQ Quality Indicator Tools for Data Analytics
• United States Health Information Knowledgebase (USHIK)
• Data Sources Available from AHRQ

Funding & Grants

• Notice of Funding Opportunities
• Research Policies
• Notice of Funding Opportunity Guidance
• AHRQ Grants Policy Notices
• AHRQ Informed Consent & Authorization Toolkit for Minimal Risk Research
• HHS Grants Policy Statement
• Federal Regulations & Authorities
• Federal Register Notices
• AHRQ Public Access Policy
• Protection of Human Subjects
• Funding Priorities
• Special Emphasis Notices
• Staff Contacts
• Training & Education Funding
• Grant Application, Review & Award Process
• Grant Application Basics
• Application Forms
• Application Deadlines & Important Dates
• AHRQ Tips for Grant Applicants
• Grant Mechanisms & Descriptions
• Application Receipt & Review
• Study Sections for Scientific Peer Review
• Award Process

Post-Award Grant Management

• AHRQ Grantee Profiles
• Getting Recognition for Your AHRQ-Funded Study
• Grants by State
• No-Cost Extensions (NCEs)

AHRQ Grants by State

Searchable database of AHRQ Grants

AHRQ Projects funded by the Patient-Centered Outcomes Research Trust Fund.

• Press Releases
• AHRQ Social Media
• Impact Case Studies
• AHRQ News Now
• AHRQ Research Summit on Diagnostic Safety
• AHRQ Research Summit on Learning Health Systems
• National Advisory Council Meetings
• AHRQ Research Conferences
• AHRQ's 35th Anniversary
• Mission and Budget
• AHRQ's Core Competencies
• National Advisory Council
• Careers at AHRQ
• Maps and Directions
• Other AHRQ Web Sites
• Other HHS Agencies
• Testimonials

Organization & Contacts

• Centers and Offices
• Organization Chart
• Key Contacts

• Post Award Grants Management

Research Training and Education

Sign up: Research Training and Education Email updates

AHRQ's Research Training Opportunities

The Agency for Healthcare Research and Quality (AHRQ) provides a variety of funding opportunities designed to support and enhance the education and career development of health services researchers. AHRQ’s training opportunities are designed to prepare researchers to address the vast changes occurring in health care delivery. Please see general information about the various predoctoral, postdoctoral, and career development opportunities sponsored by AHRQ. Research Training and Career Development activities are administered by the Division of Research Education in the Office of Extramural Research, Education and Priority Populations (OEREP).

Individual Programs:

Institutional Programs:

Patient-Centered Outcomes Research (PCOR) Programs:

Tips for Applicants

Tools and resources for applicants.

Building a Health Services Research Workforce

Internet Citation: Research Training and Education. Content last reviewed December 2022. Agency for Healthcare Research and Quality, Rockville, MD. https://www.ahrq.gov/funding/training-grants/index.html

• Department of Health and Human Services
• National Institutes of Health

Clinical Electives Program: Health Services Research

Eight-Week Session

Prerequisites : This elective is intended for students who are interested in population-level health services research and disability-related public health issues. Previous coursework in research methods is helpful, but not required. Selection is based upon evidence of motivation to explore a breadth of topics included in health services research and appreciate potential policy implications. Applicants must possess good standing in medical school.

Rotation Coordinator : Julia Porcino, MS

Rotation Description: The Rehabilitation Medicine Department (RMD) resides within the Mark O. Hatfield Clinical Research Center, a state of the art, 200-bed biomedical research hospital located on the campus of the National Institutes of Health in Bethesda, MD. The RMD was established to provide the highest quality rehabilitation assessment and treatments for patients with complex medical conditions and to perform cutting edge rehabilitation research. The Department initiates, collaborates in, and supports research conducted at the Clinical Center and other NIH Institutes and Centers. In May of 2007, the RMD added an Epidemiology and Biostatistics Section to support the statistical needs of the RMD staff, address departmental data management and security issues, and to expand its research portfolio to include population-level research on health services and disability-related public health issues. Since its inception, the Section has initiated and is currently involved in several collaborative research projects. Broadly, the research examines health disparities, health services, disability, and health related policies and programs. One main project is a multi-year effort to assist the Social Security Administration (SSA) explore innovative methods for augmenting the disability determination process. This work has two main objectives - to develop systematic data-driven analytic tools using existing SSA data to inform disability determination processes, and to develop a new functional assessment instrument to comprehensively and consistently assess self-reported claimant function.

The student will work with the research team to become familiar with the team's current research projects and develop an appreciation for health services and rehabilitation research more generally. A selection of seminal publications in the student's area of interest will be provided to actively engage the student in critical examination of the literature and substantive discussion with team members. With guidance from the RMD team, the student will identify an area of the ongoing research where they can contribute or can define their own research question for exploration. Expectations around scientific dissemination will depend on the scope of the contributions and the student's own goals for the rotation.

Rotation Objectives: During this training, students will:

• Develop a general knowledge of the major topics of study in the fields of health services and rehabilitation, with particular focus on people with disabilities. Topics may include health disparities, organization and quality effectiveness of health services, disparities and health service utilization;
• Select one focal area for more in depth study of the extant literature, methods and implications for policy/practice;
• Effectively communicate with mentors on relevant research ideas in both written and oral forms;
• Identify an area of research for contribution, such as by supporting an ongoing project effort or developing a small research project, which would include defining a problem, formulating hypotheses, and developing a manageable analytical strategy. Students will also learn and apply basic protections for human subjects and consider ethical guidelines when formulating the research plan;
• Gain knowledge of and experience with NIH resources supporting research;
• Time and interest permitting the student will work with Rehabilitation Medicine Department (RMD) staff to develop an abstract with intent to submit and present at a scientific conference; and
• Present the abstract and draft research proposal to RMD staff in one of the RMD standing forums.

Selection of Applicants: A telephone interview will be conducted to ensure the applicant's interest and goals are well aligned with the training experience we are able to provide.

• Luke Breitfeller, MLT
• Jonathan Camacho, MD, MPH
• Leighton Chan, MD, MPH
• Josh Chang, PhD
• Kathleen Coale, PT, DPT
• Bart Desmet, PhD
• Guy Divita, MS
• Kaushik Gedela, MS
• Howard Goldman, PhD
• Leslee Grubbs-King, BS
• Pei-Shu Ho, PhD
• Rafael Jimenez Silva, MPH
• Elizabeth Marfeo, PhD, MPH, OT
• Christine McDonough, PT, PhD
• Denis Newman-Griffis, MS
• Rebecca Parks, MS, OTR/L, BCP, FAOTA
• Julia Porcino, MS
• Omolola Quartey, RRT CSA
• Elizabeth Rasch, PT, PhD
• Carolyn Rose, PhD
• Maryanne Sacco, MA, ORT/L
• Larry Tang, PhD
• Hao-Ren Yao, PhD
• Chunxiao Zhou, PhD
• Aya Zirikly, PhD

NOTE: PDF documents require the free Adobe Reader .

This page last updated on 01/12/2024

You are now leaving the NIH Clinical Center website.

This external link is provided for your convenience to offer additional information. The NIH Clinical Center is not responsible for the availability, content or accuracy of this external site.

The NIH Clinical Center does not endorse, authorize or guarantee the sponsors, information, products or services described or offered at this external site. You will be subject to the destination site’s privacy policy if you follow this link.

More information about the NIH Clinical Center Privacy and Disclaimer policy is available at https://www.cc.nih.gov/disclaimers.html

Ask a Question

Toll Free Numbers

Hospitals and Clinics

• Vet Centers

Regional Benefits Offices

Regional Loan Centers

Cemetery Locations

• site map [a-z]

Center for Innovations in Quality, Effectiveness and Safety

• Center for Innovations in Quality, Effectiveness and Safety Home
• Conceptual Model
• Program Overview
• Leadership and Investigators
• Organizational Chart
• Diversity, Equity and Inclusion Policy
• Driving Directions
• Scientific Programs
• Behavioral Health Program
• Clinical Effectiveness & Population Health Program
• Implementation & Innovation Program
• Health Policy, Quality & Informatics Program
• Methodology & Analytics Core
• Research Impacts
• Current Collaborations
• Hall of Fame
• Education & Training Core
• Education & Training Core Directory
• Career Development Awards
• Helpful Links
• Veterans Health Administration
• Health Benefits Home
• Apply Online
• Veteran Eligibility
• Active Duty
• Families of Veterans
• Women Veterans
• Health Insurance
• Make a Payment
• Community Care Home
• For Veterans
• For Providers
• Forms & Publications
• Veterans Enrolled in VA Health Care
• Veterans Not Enrolled in VA Health Care
• Family Members
• Frequently Asked Questions
• Schedule a VA Appointment
• Caregiver Support
• Urgent Care
• See All Conditions & Treatments (A-Z)
• Mental Health Home
• Suicide Prevention
• Substance Abuse
• Military Sexual Trauma
• Research (MIRECC)
• Military Exposures
• Rehabilitation
• Spinal Cord Injury
• Womens Health Issues
• MyHealtheVet
• Quitting Smoking
• Vaccines & Immunizations
• Flu Vaccination
• Prevention / Wellness
• Public Health
• Weight Management (MOVE!)
• Whole Health Program
• Hospitals & Clinics
• Veterans Canteen Service (VCS)
• Research Home
• About VA Research
• News, Events and Media
• Research Topics
• For Researchers
• Research Oversight
• Readjustment Counseling (Vet Centers)
• War Related Illness & Injury Study Center
• Homeless Veterans
• Returning Service Members
• Rural Veterans
• Seniors & Aging Veterans
• Health Care Jobs (VA Careers)
• Travel Nurses
• Vets in the Workplace
• VA for Vets
• Performance Based Interviewing
• Clinical Trainees (Academic Affiliations)
• Talent Management System (TMS)
• VA Learning University (VALU)
• Libraries (VALNET)
• VA Software Documentation Library (VDL)
• Learn about VHA
• Access and Quality Data
• VA/DOD Clinical Practice Guidelines
• Medical Inspector
• Patient Safety
• Administrative

Health Systems Research Education & Training Core

Our goal is to provide nationally-recognized training and growth opportunities for fellows and faculty focused on academic careers in health services research. Here at IQuESt we pride ourselves on conducting innovative health services research that impacts real-world health outcomes  in a unique, multidisciplinary training environment.

Faculty Development

In order to foster faculty career development, we provide the following educational opportunities:

• Seminars and Workshops
• Continuing Education
• Methods and Analytics
• Professional Development
• Research in Progress for faculty to obtain peer feedback on research
• Faculty mentoring and peer mentoring
• Internal review of grants for submission

Fellowships

The Education Program directs VA-funded M.D. and Ph.D. Post-doctoral Fellowship programs in Health Systems Research. We also support a variety of other in-residence post-doctoral fellows. Fellowships housed at IQuESt fellowships include:

• Big Data-Scientist Training Enhancement Program (BD-STEP)
• Cardiovascular Surgery T32 Research Training Program  (Application information is within this link)
• Gastroenterology T32 Research Training Grant
• Health Professions Education Evaluation and Research (HPEER)
• Health Services Research (HSR)  MD and PhD Fellowship
• Mental Illness Research, Education and Clinical Center (MIRECC)/Advanced Physician
• Mental Illness Research, Education and Clinical Center (MIRECC)/Advanced Psychology

The goals of these fellowships are to produce independent researchers, educators, and clinical leaders. Fellows housed here are seeking opportunities in health services research and receive mentoring within our interdisciplinary team science approach. There are typically 8-14 fellows residing at IQuESt each year. All fellows join one of our active interdisciplinary teams and are matched with appropriate mentors. Fellows and mentors create personalized mentoring plans to develop the skills and experiences necessary for a position in academic health services research based on our model of interdisciplinary team science. To help you get to know Houston and our center, we have created a picture tour: Houston and IQuESt . Deadline Extended to December 15th.

Educational Partnerships

* This link will take you outside of the Department of Veterans Affairs Web site and will open in a new window. VA does not endorse and is not responsible for the content of linked Web sites.

Veterans Crisis Line: Call: 988 (Press 1)

Social Media

Complete Directory

• Regulations
• Web Policies
• No FEAR Act
• Whistleblower Rights & Protections
• White House
• Inspector General
• Apply for Benefits
• Apply for Health Care
• Prescriptions
• My Health e Vet
• Life Insurance Online Applications
• State and Local Resources
• Strat Plan FY 2014-2020
• VA Plans, Budget, & Performance
• VA Claims Representation
• Careers at VA
• Employment Center
• Vocational Rehabilitation & Employment
• Minority Veterans
• Plain Language
• Surviving Spouses & Dependents
• Adaptive Sports Program
• Veterans Benefits Administration
• National Cemetery Administration

U.S. Department of Veterans Affairs | 810 Vermont Avenue, NW Washington DC 20420

Last updated May 8, 2024

Get help from Veterans Crisis Line

• Call 988 (Press 1)
• Text to 838255
• Chat confidentially now
• Call TTY if you have hearing loss 1-800-799-4889

If you are in crisis or having thoughts of suicide, visit VeteransCrisisLine.net for more resources.

Certificate in Health Economics & Outcomes Research

The online 9-month, part-time health economics and outcomes research (HEOR) Certificate is tailored for individuals seeking an introduction to HEOR, including professionals in the pharmaceutical and biotechnology industry, clinicians and other health care professionals, graduate degree holders, and graduate degree students.

HEOR has become increasingly important. Providing affordable health care while maintaining the best possible health outcomes is a challenge to all governments and health care payers. This is especially true in the United States, which spends almost twice as much per capita compared to other high-income countries yet has substantially poorer health outcomes. Major drivers of the spending gap include higher administrative costs and the higher prices of devices and pharmaceuticals in the US.

As health care becomes more complex, health care decisions can be improved by combining information from economic analyses (health economics) and vital clinical data on patients (outcomes research) to evaluate health care  value . HEOR informs health care decision-making by allowing stakeholders, especially payers, health care providers, and patients, to understand the clinical, economic, and quality of life endpoints of various treatments and health care practices. Individuals seeking an introduction to HEOR, including professionals in the pharmaceutical and biotechnology industry, clinicians and other health care professionals, graduate degree holders, and graduate degree students, are encouraged to apply.

Benefits of the HEOR Certificate Program

• The flexible online format is perfect for local or global participation
• Students experience live webinars, active discussion boards and asynchronous lectures
• Program faculty are experts in clinical investigation, clinical epidemiology, health services research, outcomes research, meta-analysis, biostatistics, drug development, and related fields.
• No thesis is required
• Faculty post all required reading materials on the course website, so there are no books or materials that participants need to purchase.

Program Overview

The three-year gastroenterology fellowship program provides comprehensive training in gastroenterology and hepatology.

The first year of training is clinical, emphasizing inpatient rotations, outpatient clinics, consultation services and endoscopic procedures. 

In the second and third years, the training is tailored according to each fellow’s specific clinical and/or research interests.   The program emphasizes acquisition of advanced skills in research or clinical practice as deemed appropriate by the fellow and by the Fellowship Director and faculty mentors.   The program has a clinical and research track. Applicants may apply to one or both tracks.

Clinical Track

• Targeted for full-time clinicians and clinician educators
• Includes three years of clinical work
• Area of expertise encouraged through clinical and/or research opportunities
• Development of teaching skills

Research Track

• Targeted for individuals planning research careers in academic gastroenterology
• One year full-time clinical work
• Two years with three months of clinical work
• Two years with nine months of research each
• Continuity clinic throughout training period
• Supported by the Ruth L. Kirschstein National Research Service Award Institutional Research Training Grant (T32). Due to NIH eligibility requirements, applicants holding either, a J1 or an H1-B visa cannot be considered

Third Year Fellows

Second Year Fellows

• Open access
• Published: 02 September 2024

The right care in the right place: a scoping review of digital health education and training for rural healthcare workers

• Leanna Woods 1 , 2 ,
• Priya Martin 3 ,
• Johnson Khor 1 , 4 ,
• Lauren Guthrie 1 &
• Clair Sullivan 1 , 2 , 5  

BMC Health Services Research volume  24 , Article number:  1011 ( 2024 ) Cite this article

Metrics details

Digital health offers unprecedented opportunities to enhance health service delivery across vast geographic regions. However, these benefits can only be realized with effective capabilities and clinical leadership of the rural healthcare workforce. Little is known about how rural healthcare workers acquire skills in digital health, how digital health education or training programs are evaluated and the barriers and enablers for high quality digital health education and training.

To conduct a scoping review to identify and synthesize existing evidence on digital health education and training of the rural healthcare workforce.

Inclusion criteria

Sources that reported digital health and education or training in the healthcare workforce in any healthcare setting outside metropolitan areas.

We searched for published and unpublished studies written in English in the last decade to August 2023. The databases searched were PubMed, Embase, Scopus, CINAHL and Education Resources Information Centre. We also searched the grey literature (Google, Google Scholar), conducted citation searching and stakeholder engagement. The JBI Scoping Review methodology and PRISMA guidelines for scoping reviews were used.

Five articles met the eligibility criteria. Two case studies, one feasibility study, one micro-credential and one fellowship were described. The mode of delivery was commonly modular online learning. Only one article described an evaluation, and findings showed the train-the-trainer model was technically and pedagogically feasible and well received. A limited number of barriers and enablers for high quality education or training of the rural healthcare workforce were reported across macro (legal, regulatory, economic), meso (local health service and community) and micro (day-to-day practice) levels.

Conclusions

Upskilling rural healthcare workers in digital health appears rare. Current best practice points to flexible, blended training programs that are suitably embedded with interdisciplinary and collaborative rural healthcare improvement initiatives. Future work to advance the field could define rural health informatician career pathways, address concurrent rural workforce issues, and conduct training implementation evaluations.

Review registration number

Open Science Framework: https://doi.org/10.17605/OSF.IO/N2RMX .

Peer Review reports

Introduction

Globally, healthcare workers (HCWs) face multiple pressures simultaneously: increasing demand for care, co-morbidities and condition complexity, budget pressures, and rapid digital disruption [ 1 ]. The digital disruption in healthcare promises an unprecedented circumstance to improve outcomes and strengthen health systems [ 2 ]. However, this opportunity depends on a capable healthcare workforce with adequate skills and knowledge in data and emerging technologies [ 3 ]. HCW capability in digital health and clinical informatics is increasingly acknowledged as an essential component to the delivery of high-quality patient care [ 4 ]. Universities do not yet routinely teach these curricula in clinical degrees, and the capability gap in the current workforce is often filled by brief, reactive, and on-the-job training [ 5 ]. Sustainability of healthcare includes developing a skilled healthcare workforce educated and competent in digital health [ 6 ].

The rural healthcare workforce is faced with the location-based issues of resource constraints, workforce shortages, high staff turnover rates, stress, burnout, and an ageing workforce [ 7 ]. The World Health Organization has acknowledged in a recent report (2021) the complex challenge of shortage of healthcare workers globally in rural areas [ 7 ]. This report has acknowledged that the workforce density is lower than national averages in most of these areas. In places where there isn’t a national shortage, maldistribution of the workforce has been noted [ 7 ]. Digitally enabled models of care are well placed to enhance health service delivery across vast and distributed geographic regions. However, rural health service organizations require uplift to align with their metropolitan counterparts in workforce digital readiness [ 8 ]. Building digital health capability in rural settings is critical because higher digital health capability is associated with better outcomes, including the ability to maintain an accurate patient health record, track patient experience data, track the patient journey, and mitigate clinical risks [ 9 ]. Rurality is contributing to widening digital health inequities [ 10 ] with significant efforts required to adequately manage the rural digital divide [ 11 , 12 ]. Building digital capabilities of healthcare providers in rural and remote settings through education, training and support is needed [ 13 ].

Existing evidence on the education and training the rural healthcare workforce is limited. Firstly, while health science faculties are progressively integrating digital health into the undergraduate curricula for the future workforce [ 14 , 15 , 16 ], it is unclear how the education of current HCW is approached [ 14 ]. Despite global exemplars such as fellowship training for physicians [ 17 ], certification for nurses [ 18 ], and advanced education for clinical and non-clinical professionals [ 19 ], limited evidence of successful workforce programs to build digital health skills exist [ 4 ]. None focus on the rural healthcare setting.

Secondly, in literature reporting digital health in rural settings, there is a notable scarcity on workforce training programs. Existing studies focus on efficacy of delivered healthcare [ 20 , 21 ], workforce perceptions of digital health tool implementation [ 22 , 23 ] or are limited to training of specific interventions (e.g., clinical telehealth [ 24 ]). This review sought to explore the literature where these two gaps coexist, the intersection of digital health education and training and the rural healthcare workforce, and synthesize the available evidence on digital health education and training for the rural healthcare workforce.

Review question

The research questions for this review were:

What are the existing practices and approaches to digital health education and training for rural HCWs?

How has digital health education and training been evaluated following implementation?

What are the barriers and enablers for high quality digital health education and training in the rural healthcare workforce?

Participants

The review considered studies and reports on any members of the workforce in healthcare settings outside of metropolitan areas. The healthcare workforce refers to ‘all individuals who deliver or assist in the delivery of health services or support the operation of health care facilities’ [ 3 ]. All clinical (e.g., medical doctors, nurses, allied health professionals, pharmacists, Indigenous HCWs, pre-registration/qualification students undertaking placements in health care facilities) and non-clinical workers (e.g., administration, executive and management, clinical support, and volunteers) were considered regardless of professional body or government registration status. Patients, healthcare consumers, and the public were excluded.

The core concepts of digital health and training were combined in this review. Digital health and clinical informatics are often used interchangeably, and both were considered in this review. While digital health refers to the use of digital technologies for health [ 25 ], clinical informatics refers to more specialized practice of analyzing, designing, implementing and evaluating information and communication systems [ 26 ]. Specific digital health systems (e.g., IT infrastructure, telehealth, electronic medical records) were included. Training relates to the education or training initiatives (e.g., programs, curriculum, course) that build an individuals’ digital health capability to confidently use technologies to respond to the needs of consumers now and into the future [ 1 ]. Both education and training activities were considered. Education often refers to theoretical learning (e.g., by an academic institution, qualification), and training often teaches practical skills (e.g., employer-provided professional development, ‘just-in-time’ training) [ 3 , 24 ]. This review did not consider HCW education delivered at a distance through technologies (e.g., telesupervision for clinical skills training).

This review considered studies and reports from rural healthcare settings defined as outside metropolitan cities, inclusive of regional, rural, remote, and very remote settings. When the term ‘rural’ is used in this review, it refers to all areas outside major metropolitan cities as described by authors of individual studies and reports. All healthcare facilities across primary, secondary, and tertiary care settings were included in any country.

Types of sources

All research studies, irrespective of the study design, were considered. Reviews, conference abstracts and non-research sources (e.g., policy documents, program or course curriculum) were considered. The grey literature was included to capture reactionary training developed by rural health services that were not published as peer-reviewed research studies.

This review was conducted in accordance with the Joanna Briggs Institute (JBI) methodology for scoping reviews [ 27 ] and reported as per the Preferred Reporting of Systematic Reviews and Meta-analyses for scoping reviews (PRISMA-ScR) [ 28 ] (Additional file 1 ). The review protocol was registered in Open Science Framework [ https://doi.org/10.17605/OSF.IO/N2RMX ].

A scoping review approach was chosen over a systematic review to address a general, formative review question on this topic that is emerging in the literature and where the literature is complex and heterogenous [ 29 ]. An initial preliminary search of the topic in the academic databases, Cochrane Library, Open Science Framework and Prospero registry resulted in a very small number of relevant articles. It was determined that a broader search strategy and inclusion of non-research sources was required, consistent with the scoping review methodology [ 29 ]. Scoping review format is also well suited to the vast, diverse healthcare education topic across different disciplines, interventions and outcomes realised [ 30 ]. Mapping and synthesis across sources in this scoping review aims to inform research agendas and identify implications for policy and practice [ 31 ].

Deviations from the protocol

There were no deviations to the protocol.

Search strategy

The three phase JBI search process was followed. An initial limited search of PubMed was performed to identify keywords on the topic, followed by an analysis of the text words and index terms contained in the title and abstract. A subsequent preliminary search in Prospero registry, Cochrane Library and Open Science Framework informed the development of a full search strategy in PubMed. The search strategy, including all identified keyworks and index terms, was adapted for each included database and information source after refining the strategy with an information specialist. The reference lists of all included sources of evidence were screened for additional studies.

The review included only studies and reports in English (due to translation resourcing limitations) in the last 10 years (due to the relative novelty of the digital transformation of healthcare). The search was conducted in August 2023. The databases searched included PubMed, Scopus, Cumulative Index for Nursing and Allied Health Literature (CINAHL), Embase, and Education Resources Information Center (ERIC). Scopus was chosen over Web of Science as it provides 20% more coverage and the relative recency of articles indexed (publish date after 1995 [ 32 ]) was not a concern for our research question. The search for unpublished studies and grey literature included Google and Google Scholar, using a modified search strategy as required. In addition, national and international stakeholders ( n  = 29) from Asia, the Pacific Islands, Australia, USA and the UK known to have subject matter expertise on the topic were contacted via direct email. Stakeholders were asked to share any relevant work underway or otherwise undiscoverable using our scoping review methods. The full search strategy for each information source is provided in Additional file 2 .

Study selection

Following the search, identified articles were collated and uploaded into Covidence review software (Veritas Health Innovation Ltd; Melbourne, Australia) and duplicates removed. Two reviewers (among LW, JK and LG) then independently screened the title and abstract of each citation and selected studies that met the inclusion criteria. The full text articles were retrieved and uploaded into Covidence. These studies and reports were assessed independently by two reviewers (listed previously) for full assessment against the inclusion criteria. Any disagreements that arose between the reviewers at each stage of the selection process were resolved through discussion or with an additional reviewer (among LG and PM). Three meetings occurred to discuss any voting conflicts that occurred during title and abstract screening and full-text screening. Articles that did not satisfy the criteria were excluded with reasons for exclusion recorded. Search results and study selection process is presented in accordance to the PRISMA-ScR flow diagram (Fig. 1 ) [ 28 ]. Quality appraisal of selected studies was not conducted, consistent with scoping reviews methods [ 33 ].

Search results and source selection and inclusion process

Data extraction

Extracted data included the specific details about the participants, concept, context, study methods and key findings relevant to each review question. Data was extracted by one reviewer (JK) and checked by a second reviewer (LW). Data were extracted using the data extraction tool developed and piloted by the team (Additional file 3 ).

Data synthesis and presentation

The characteristics of the included studies were analyzed and organized in tabular format, accompanied by a narrative summary. Results of each research question was presented under separate headings. The data analysis for research question three (barriers and enablers of high-quality digital health education and training) was enhanced. We adopted the socio-institutional framework described by Smith et al [ 34 ] and used in education research [ 35 ] to classify macro, meso, micro level enablers and barriers to help improve the generalizability of the synthesized insights and identify stakeholders that are able to influence change. Gaps and limitations of the current literature were discovered from the evidence with recommendations for policy, practice and future research provided.

Study inclusion

Database searching yielded 1005 articles and stakeholder engagement yielded two articles. After removing duplicates, 660 articles were screened for title and abstract, after which 29 articles underwent full text review. Of the 29 articles, 24 articles were excluded: the setting was metropolitan or otherwise inadequately described as non-metropolitan ( n  = 6); the intervention was not a training or education initiative for digital health or clinical informatics ( n  = 16), or the population was not rural healthcare workers ( n  = 2). In total, following full-text screening, five articles were included in the final review (Fig.  1 ).

Characteristics of included studies

Of the five included articles, three were academic publications including two case studies [ 36 , 37 ] and one feasibility study [ 38 ] (Table 1 ). The two articles identified through stakeholder engagement presented course summaries [ 39 , 40 ] where one described a micro-credential [ 40 ] and the other described a fellowship [ 39 ]. Most articles ( n  = 3) were published recently between 2021 and 2023 [ 38 , 39 , 40 ]. Healthcare workforce settings were distributed across the continents of the United States of America [ 36 ], Asia [ 37 ], Africa [ 38 ] and Australia [ 39 , 40 ], with no articles reporting a setting in the European continent. Further study characteristics are available in Table 1 .

Review findings

What are the existing approaches to digital health education and training for rural hcws.

Training and education programs were needed due to identified gaps in knowledge, skills and expertise to support healthcare delivery in rural contexts with digital health [ 36 , 37 , 38 ], [ 40 ]. One article reported the target learners as village doctors, who may have “limited training and inadequate medical knowledge, yet they are generally the mainstay of health services” [ 37 ]. The mode of teaching in the included studies were four modular online learning courses [ 36 , 37 , 38 ], [ 40 ] and one fellowship [ 39 ]. Of the four modular online learning courses, one was supplemented by a facilitator-led train-the-trainer model [ 38 ], informed by an academic framework [ 41 ], with cohort-based discussion via a social media platform. The second was a certification in the form of a self-paced micro-credential completed individually [ 40 ]. Of the four modular online learning courses, the number of modules ranged from three to eight and covered a variety of digital health topics including innovation, commercialization, bioinformatics, technology use, data and information, professionalism, implementation and evaluation. One had a particular focus on information and communication technology tool use [ 37 ] while another focused on remote consulting [ 38 ]. The mode of delivery of the fellowship was not reported in the article.

Four [ 36 , 37 , 39 , 40 ] of the five included articles did not report an evaluation. One article in rural Tanzania described the evaluation of the train-the-trainer digital health training program using a mixed-method design [ 38 ]: (1) questionnaire informed by Kirkpatrick’s model of evaluation to capture knowledge gained and perceived behavior change on a Likert scale, (2) qualitative interviews to explore training experiences and views of remote consulting, and (3) document analysis from texts, emails and training reports [ 38 ]. Of the tier 1 trainees (senior medical figure trainers who were trained to educate their peers) that completed the questionnaire ( n  = 10, 83%), nine (90%) recommended the training program and reported receiving relevant skills and applying learning to daily work, demonstrating satisfaction, learning and perceived behavior change [ 38 ]. Overall, the feasibility study confirmed that remotely delivered training supported by cascade training was technically and pedagogically feasible and well received in rural Tanzania [ 38 ].

What are the barriers and enablers for high quality digital health education and training of the rural healthcare workforce?

Reported enablers and barriers are presented using the macro, meso, micro framework [ 34 ] (Table 2 ).

This scoping review reflects the scarcity of reported digital health education and training programs in existence for rural HCWs globally. This review responds to the World Health Organization (WHO) recommendation to design and enable access to continuing education and professional development programs that meet the needs of rural HCWs [ 7 ], and the Sustainable Development Goal for inclusive and equitable quality education [ 42 ].

Concurrent challenges of people (workforce), setting (rural) and content (digital health) are reported in included articles alongside enablers and barriers to education and training programs. Included studies reported a shortage of doctors and specialists [ 36 ], lack of technical knowledge [ 36 ] (people); higher cost of delivering rural healthcare, high burden of illness [ 40 ], medically underserved population due to rural hospital closures [ 36 ] (setting); and limited use of digital health tools due to coordination challenges among non-government organisations [ 37 ] (content). These additional macro, meso and micro level factors are described by authors firstly as influencing the need for digital health programs in rural settings, and secondly, as contributing to the challenges of implementing effective programs. The rural health workforce challenges in digital health education and training reflect the broader workforce development issues experienced globally [ 7 ]. While this review sought to identify workforce development programs, the WHO model indicates the need for attractiveness, recruitment and retention to enable workforce performance (i.e., appropriate and competent multidisciplinary teams providing care) and health system performance (i.e., improving universal health coverage) [ 7 ].

In low-resource settings such as rural areas, education and training may not be prioritized among other competing workload demands. As the value of digital health transformations are realized for strengthening healthcare systems [ 25 , 43 ], the value of digital health education or training programs may become realized. This value was evidenced in the implementation of the teleconsulting training intervention in rural Tanzania [ 38 ] in rapid response to supporting care delivery during the COVID-19 pandemic period. With evaluations of programs largely absent from an already small number of programs globally, it will be important for future research to focus on implementation evaluation studies. As Table 2 presents only limited enablers and barriers, more evidence is needed to build on the findings from this scoping review to inform strategies for policy and practice.

The interdisciplinarity of digital health presents challenges and opportunities for nurturing digital health expertise across the rural healthcare workforce. Included articles largely described the target learners of education and training programs as clinicians, practitioners and healthcare workforce. Walden et al. further indicated that users of online content may extend beyond rural health clinicians to healthcare administrators, researchers and providers relevant to address the regulatory factors of clinical validation and implementation [ 36 ]. Therefore, for their program of work, the University of Arkansas for Medical Sciences identified and fostered collaboration with an interprofessional team of clinicians, researchers, informaticists, a bioethicist, lawyers, technology investment experts, and educators [ 36 ]. No articles in the review described education or training health informaticians or similar digital health leadership role types, yet building defined career pathways for health informaticians is recommended [ 4 ]. Existing pedagogy shows that the learning principles of interprofessional practice is grounded in understanding one’s own practice as well as the practice of other health professionals and remains aligned to the educational needs of specific professions [ 44 ] (i.e., medicine, nursing, pharmacy). Defining new career pathways for interdisciplinary leaders in digital health within a specific clinical context, like the ‘rural health informatician’, will be important to identify or define the (hidden) specialized workforce.

Local, informal organizational initiatives for digital health learning were discovered alongside formal education or training programs in included studies. Programs were often reported in articles alongside concurrent digital health implementation or healthcare improvement programs, sometimes referred to as ‘outreach’ [ 36 ] activities. These informal initiatives included special interest groups, in-person conferences, networking events, working groups [ 36 ] and seminars [ 37 ]. Current evidence from this scoping review suggests that the efficacy and sustainability of education or training programs are reliant on integrated approaches, like the train-the-trainer [ 38 ] or academic organization approach [ 36 ], that foster translational research for rural healthcare improvement. As illustrated by Walden et al., success in digital health is likely to require a foundational environment where technologies can be discussed, developed and deployed [ 36 ]. Success in rural digital health skills acquisition likely requires a similar, longitudinal and collaborative approach beyond the confines of an online course completed individually. Previous research shows us that blended learning, which merges face-to-face with online learning, translates to better knowledge outcomes [ 44 ]. Blended learning can also overcome the barrier of rural HCWs travelling large distances to attend face-to-face training that comes at a great cost to themselves and the work unit. A key recommendation to improve the digital health training program described by Downie et al. was more face-to-face time with trainers, from the perspective of both trainee and facilitator [ 38 ]. This, however, can only be realized with targeted planning and budgeting of such offerings by involved rural healthcare organizations.

The opportunities to advance digital health education and training for rural HCWs are presented across the macro, meso and micro levels in the socio-institutional framework, with suggested relevant stakeholders suited to actioning the recommendations (Table  3 ). While the context for this is likely to vary across the globe, these recommendations and stakeholders are expected to provide a starting point to initiate a dialogue that can influence change. These recommendations are not meant to be prescriptive or rigid, but rather meant to flag actionable solutions that can be contextualized for any given setting.

Strengths and limitations

It is possible that there is a greater number of published educational and training programs than those reported in this review (i.e., publication bias). To mitigate this, we used a scoping review methodology and stakeholder engagement activity to identify unpublished or emerging programs that answer the review question but may not be discoverable in the academic databases. The review is limited to articles available in the English language. The small number of programs, heterogeneity of programs and limited evaluation of programs significantly limit generalizability of findings. Due to data availability, the barriers and enablers findings summary contain an overrepresentation from a small number of studies limiting conclusions that can be drawn.

Digital health offers the best opportunity for innovative sustainable change to address critical issues in health and care in rural settings. Workforce education and training initiatives in rural healthcare settings are scarce, largely delivered via online training, and are rarely evaluated. Current best practice points to flexible, blended (online and face-to-face) training programs that are suitably embedded with interdisciplinary, collaborative rural healthcare improvement initiatives. More research will expand the evidence base to deliver high-quality digital health education to strengthen rural healthcare delivery. Future work to advance the field could define rural health informatician career pathways, address concurrent rural workforce issues, and conduct implementation evaluations.

Availability of data and materials

No datasets were generated or analysed during the current study.

Abbreviations

Cumulative Index for Nursing and Allied Health Literature

Education Resources Information Centre

Healthcare worker

Joanna Briggs Institute

Preferred Reporting of Systematic Reviews and Meta-analyses for scoping reviews

World Health Organization

Australian Digital Health Agency. The national digital health capability action plan. Australia: Australian Government; 2022. p. 36.

Google Scholar  

World Health Organization. Global strategy on digital health 2020–2025. Geneva: World Health Organization; 2021. p. 60.

Australian Digital Health Agency. National digital health workforce and education roadmap. Sydney; 2020.

Woods L, Janssen A, Robertson S, et al. The typing is on the wall: Australia’s healthcare future needs a digitally capable workforce. Aust Health Rev. September 2023;2023:25. https://doi.org/10.1071/AH23142 .

Article   Google Scholar  

Younge VL, Borycki EM, Kushniruk AW. On-the-job training of health professionals for electronic health record and electronic medical record use: A scoping review. Knowledge Management & E-Learning: An International Journal. 2015;7:436–69.

Thomas EE, Haydon HM, Mehrotra A, et al. Building on the momentum: sustaining telehealth beyond COVID-19. J Telemed Telecare. 2022;28:301–8.

Article   PubMed   Google Scholar  

World Health Organization. WHO guideline on health workforce development, attraction, recruitment and retention in rural and remote areas. 2021.

Woods L, Eden R, Pearce A, et al. Evaluating Digital Health Capability at Scale Using the Digital Health Indicator. Appl Clin Inform. 2022;13:991–1001. https://doi.org/10.1055/s-0042-1757554 .

Article   PubMed   PubMed Central   Google Scholar  

Woods L, Dendere R, Eden R, et al. Perceived Impact of Digital Health Maturity on Patient Experience, Population Health, Health Care Costs, and Provider Experience: Mixed Methods Case Study. J Med Internet Res. 2023;25: e45868. https://doi.org/10.2196/45868 .

Yao R, Zhang W, Evans R, et al. Inequities in health care services caused by the adoption of digital health technologies: scoping review. J Med Internet Res. 2022;24:e34144. https://doi.org/10.2196/34144 .

Clark CR, Akdas Y, Wilkins CH, et al. TechQuity is an imperative for health and technology business: Let’s work together to achieve it. J Am Med Inform Assoc. 2021;28:2013–6. https://doi.org/10.1093/jamia/ocab103 .

Esteban-Navarro M-Á, García-Madurga M-Á, Morte-Nadal T, et al. The rural digital divide in the face of the COVID-19 pandemic in Europe—recommendations from a scoping review. Informatics. 2020;7:54.  https://doi.org/10.3390/informatics7040054 .

Macklin S. Understanding the pathway to consumer centred healthcare information in rural and remote Queensland. Brisbane: The University of Queensland; 2022.

Aungst TD, Patel R. Integrating digital health into the curriculum—considerations on the current landscape and future developments. J Med Educ Curric Dev. 2020;7:2382120519901275.

Edirippulige S, Gong S, Hathurusinghe M, et al. Medical students’ perceptions and expectations regarding digital health education and training: a qualitative study. J Telemed Telecare. 2022;28:258–65.

Veikkolainen P, Tuovinen T, Jarva E, et al. eHealth competence building for future doctors and nurses–Attitudes and capabilities. Int J Med Informatics. 2023;169:104912.

American Medical Informatics Association. Informatics academic programs. 2022 https://amia.org/careers-certifications/informatics-academic-programs . Accessed 21 June 2022.

American Nurses Credentialing Center. Informatics nursing board certification examination. Maryland, USA2018, p. https://www.nursingworld.org/~490a495b/globalassets/certification/certification-specialty-pages/resources/test-content-outlines/427-tco-rds-2016-effective-date-march-2023-2018_100317.pdf .

Topol E. The topol review: preparing the healthcare workforce to deliver the digital future. United Kingdom: Health Education England NHS; 2019. p. 1–48.

McCleery J, Laverty J, Quinn TJ. Diagnostic test accuracy of telehealth assessment for dementia and mild cognitive impairment. Cochrane Database Syst Rev. 2021;7(7):CD013786.

PubMed   Google Scholar  

Janjua S, Carter D, Threapleton CJ, et al. Telehealth interventions: remote monitoring and consultations for people with chronic obstructive pulmonary disease (COPD). Cochrane database of systematic reviews; 2021.

Xyrichis A, Iliopoulou K, Mackintosh NJ, et al. Healthcare stakeholders’ perceptions and experiences of factors affecting the implementation of critical care telemedicine (CCT): qualitative evidence synthesis. Cochrane Database Syst Rev. 2021;2(2):CD012876.

Odendaal WA, Watkins JA, Leon N, et al. Health workers’ perceptions and experiences of using mHealth technologies to deliver primary healthcare services: a qualitative evidence synthesis. Cochrane Database Syst Rev. 2020;3(3):CD011942.

Edirippulige S, Armfield N. Education and training to support the use of clinical telehealth: A review of the literature. J Telemed Telecare. 2017;23:273–82. https://doi.org/10.1177/1357633x16632968 .

Article   CAS   PubMed   Google Scholar  

World Health Organization. Recommendations on digital interventions for health system strengthening. World Health Organization. 2019. 2020–2010.

Gardner RM, Overhage JM, Steen EB, et al. Core content for the subspecialty of clinical informatics. J Am Med Inform Assoc. 2009;16:153–7.

JBI. JBI manual for evidence synthesis. 2020.

Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169:467–73.

Peters MDJ, Marnie C, Tricco AC, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth. 2020;18(10):2119–26. https://doi.org/10.11124/jbies-20-00167 .

Shead DA, Olivier B. Traditional or digital health care education? JBI Evidence Synthesis. 2020;18:861–2.

Tricco AC, Lillie E, Zarin W, et al. A scoping review on the conduct and reporting of scoping reviews. BMC Med Res Methodol 2016; 16: 15. https://doi.org/10.1186/s12874-016-0116-4 .

Falagas ME, Pitsouni EI, Malietzis GA, et al. Comparison of PubMed, Scopus, web of science, and Google scholar: strengths and weaknesses. FASEB J. 2008;22:338–42.

Peters MD, Godfrey CM, Khalil H, et al. Guidance for conducting systematic scoping reviews. JBI Evid Implement. 2015;13:141–6.

Smith T, McNeil K, Mitchell R, et al. A study of macro-, meso-and micro-barriers and enablers affecting extended scopes of practice: the case of rural nurse practitioners in Australia. BMC Nurs. 2019;18:1–12.

Desai D, Mayne C, Bates H, et al. A rapid review of the barriers and enablers of medical student participation in research in health settings. 2022. https://doi.org/10.17605/OSF.IO/5XZWN .

Walden A, Kemp AS, Larson-Prior LJ, et al. Establishing a digital health platform in an academic medical center supporting rural communities. Journal of Clinical and Translational Science. 2020;4:384–8.

Mridha M and Islam M. To improve patient care & safety of rural patients empowering the village doctors. In: EMBEC & NBC 2017: Joint Conference of the European Medical and Biological Engineering Conference (EMBEC) and the Nordic-Baltic Conference on Biomedical Engineering and Medical Physics (NBC), Tampere, Finland, June 2017 2018, pp.502-505. Springer.

Downie A, Mashanya T, Chipwaza B, et al. Remote Consulting in Primary Health Care in Low-and Middle-Income Countries: Feasibility Study of an Online Training Program to Support Care Delivery During the COVID-19 Pandemic. JMIR Formative Research. 2022;6:e32964.

Australian College of Rural & Remote Medicine. Rural Generalist Curriculum - Fellowship. Brisbane, Australia: ACRRM; 2021. p. 174.

Northern Australia Regional Digital Health Collaborative and James Cook University (NARDHC and JCU). Digital Health for the Rural and Remote Health Workforce Micro-credential (Brochure). Townsville: JCU; 2023.

Mormina M, Pinder S. A conceptual framework for training of trainers (ToT) interventions in global health. Glob Health. 2018;14:1–11.

United Nations Department of Economic and Social Affairs Sustainable Development. Sustainable development goals. 2023. https://sdgs.un.org/goals . Accessed 20 Feb 2024

Woods L, Eden R, Canfell OJ, et al. Show me the money: how do we justify spending health care dollars on digital health? Med J Aust 2022 2022/12/12. https://doi.org/10.5694/mja2.51799 .

Dizon JMR. Educating future health professionals to keep pace with changing times. LWW, 2021, p. 2904–2905.

Download references

Acknowledgements

Not applicable.

No external funding.

Author information

Authors and affiliations.

Queensland Digital Health Centre, The University of Queensland, Brisbane, Australia

Leanna Woods, Johnson Khor, Lauren Guthrie & Clair Sullivan

Centre for Health Services Research, The University of Queensland, Brisbane, Australia

Leanna Woods & Clair Sullivan

Rural Clinical School, Faculty of Medicine, The University of Queensland, Toowoomba, Australia

Priya Martin

Ochsner Clinical School, Faculty of Medicine, The University of Queensland, Brisbane, Australia

Johnson Khor

Metro North Hospital and Health Service, Herston, Australia

Clair Sullivan

You can also search for this author in PubMed   Google Scholar

Contributions

LW, PM and CS designed the study. LW, PM, JK and LG acquired data; analyzed and interpreted results and drafted the manuscript and all subsequent drafts. CS read and contributed to manuscript drafts. All authors read and approved the final manuscript draft.

Corresponding author

Correspondence to Leanna Woods .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

12913_2024_11313_moesm1_esm.docx.

Additional File 1. Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) Checklist.

Additional File 2. Full search strategy for each information source.

Additional file 3. data extraction instrument template., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Woods, L., Martin, P., Khor, J. et al. The right care in the right place: a scoping review of digital health education and training for rural healthcare workers. BMC Health Serv Res 24 , 1011 (2024). https://doi.org/10.1186/s12913-024-11313-4

Download citation

Received : 13 March 2024

Accepted : 15 July 2024

Published : 02 September 2024

DOI : https://doi.org/10.1186/s12913-024-11313-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Clinical informatics
• Health informaticians
• Digital health
• Health personnel
• Rural population
• Rural health
• Low-resource settings

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

The University of Chicago The Law School

Civil rights & police accountability clinic—significant achievements for 2023-24.

Our Clinic students continue to make a difference in the community, while learning all that it means to be a lawyer.

The Federal Civil Rights Consent Decree Governing the Chicago Police Department

Years of advocacy by Clinic students and our clients resulted in the 2019 federal civil rights Consent Decree that seeks to remedy the Chicago Police Department’s (CPD’s) pattern and practice of excessive and discriminatory violence targeted disproportionately against Black people. Highlights from our Consent Decree work during the 2023-24 academic year include: (1) our success in remedying CPD’s practice of violent, dehumanizing, and discriminatory home raids that have targeted and traumatized Black and Brown children and families in Chicago; (2) the relief that we won in emergency proceedings that we initiated to prevent unlawful mass arrests and First Amendment violations during the Democratic National Convention in Chicago; (3) our progress toward remedying racially discriminatory practices of targeting Black people for unlawful stop-and-frisks and pretextual traffic stops; and (4) advocacy for critical modifications to strengthen and improve the Decree.

Ending Illegal and Discriminatory Home Raids

Clinic students and our community-based clients won a complete overhaul of the policies that govern residential search warrants in Chicago in months-long court supervised multi-party negotiations, briefs, and court proceedings. CPD’s new policies will seek to restrict home raids to circumstances in which they are necessary and forbid raids whenever the potential harms outweigh the expected benefits. CPD will be required to develop a written plan for the execution of every residential warrant to minimize the harm, trauma, and intrusion to families and their homes. Officers will be evaluated for their success in mitigating harm when executing search warrants. The new policies will protect children and vulnerable people from unnecessary harm, including requiring police to schedule raids at times when they are least likely to be home. They ban high-risk nighttime raids and limit no-knock warrants to circumstances in which people’s lives and physical safety are in jeopardy. They prohibit police from leaving families with broken doors and locks vulnerable to crime. And they will require police to thoroughly document and publicly report on each raid to enhance transparency and accountability. In addition, we won measures that will prevent wrong raids, including requiring CPD to independently investigate and corroborate tips, maintain records of any instance in which the informant provided false or inaccurate information, and provide the prosecutor and court with any information that may undermine the credibility of the informant and tip before seeking a warrant. The proposed new policies will soon be subject to public review and comment in anticipation of full implementation. The public and judicial scrutiny that we brought to bear during our enforcement proceedings has already resulted in a tenfold reduction of home raids and prevented the traumatization of thousands of children.

Protecting First Amendment Rights to Protest and the DNC

In spring 2023, we learned that the CPD intended to implement a new policy to facilitate mass arrests during protests and other First Amendment activities in anticipation of the Democratic National Convention. The proposed new policy would have eviscerated relief that we had won in 2021 that fundamentally transformed Chicago police policies governing the policing of First Amendment activities—relief that requires CPD to protect the rights of people to engage in public protest and dissent rather than to stamp out protests. The Clinic filed an emergency enforcement action to enjoin the proposed mass arrest policy. In the proceedings that followed, we succeeded in preventing the parts of the policy that threatened people’s First Amendment rights from taking effect. The First Amendment policy that we had won in 2021 continues to govern during the DNC and all public demonstrations now and in the future. For example, the revised policy on mass arrests will now prohibit police from arresting people engaged in First Amendment conduct for minor offenses unless they pose an immediate threat to the physical safety or property of others. It also explicitly bans retaliation against people for exercising their First Amendment rights.

Strengthening the Consent Decree and Advancing Racial Justice

Having won our community-based clients’ historic power to enforce the Decree, Clinic students continue to fight to strengthen the Decree to make our clients—people who have been most impacted by CPD’s civil rights violations—full and equal partners in the process. As a result of our advocacy, throughout the 2023-24 academic year, the federal court ordered the City to engage with community representatives when developing policies, procedures, and training—including the recent mass arrests policy that the CPD had initially sought to impose without any meaningful community engagement. Clinic students participated in five full-day public hearings in federal court focusing on potential modifications to the Decree and issues of racial justice. Students presented powerful testimony and legal memoranda that advocated for Consent Decree revisions that (a) require de-escalation and reductions in CPD violence; (b) divert people from the criminal legal system through alternatives to arrest and the elimination of unnecessary negative interactions with police; (c) develop non-criminal responses to people experiencing mental health crises (we won implementation of an historic pilot program on this); (d) prohibit police from pointing guns at people unless they present an immediate threat to serious injury or death to another person; (e) require officers to file a written report each time they point a gun at a community member; (f) provide services to survivors of CPD violence and their family members; and (g) address barriers to police accountability that were erected in the new collective bargaining agreements with the unions representing Chicago police officers. We also succeeded in subjecting CPD’s racially discriminatory stop-and-frisk practices to federal court supervision.

The court is currently deciding whether to also subject CPD traffic stops to federal court oversight under the Consent Decree, as traffic stops have become a flashpoint for unnecessary police violence in Chicago as they have skyrocketed in Black and Brown communities. Eighty-five percent of the instances in which Chicago police used force in traffic stops have been directed at Black people. We presented testimony and briefs that seek to outlaw CPD’s practice of using pretextual stops to harass Black and Brown people, disband police tactical units that have been responsible for unnecessary and disproportionate violence directed toward Black and Brown people, and limit CPD traffic stops to violations that pose genuine threats to public safety.

Ending Incommunicado Detention—A Second Consent Decree

The Clinic continues to advance its historic work in making real the fifty-eight-year-old promise of Miranda v. Arizona in Chicago. Tyler Lawson, ’24 , and Katherine Stanton, ’25, led a team of Clinic students that did outstanding advocacy work with our community-based clients and the Office of the Cook County Public Defender after having won a second consent decree that went into effect in February 2023 in Cook County Circuit Court—a decree that is designed to end the decades-long practice of incommunicado detention in CPD stations that has facilitated torture, coerced confessions, and wrongful convictions. Clinic students produced an empirical report with Professor Kyle Rozema that analyzed data from every arrest in Chicago that took place during the first year of the Decree. The Report found ninety-nine percent of people in CPD custody did not access an attorney and more than half of the people most vulnerable to interrogation did not get prompt access to a phone. Inspections by Clinic students inside Chicago police stations revealed that legible signs required by the Consent Decree that inform people in custody of their rights under the Decree and the Public Defender’s free 24-hour hotline number for legal assistance were routinely missing in the places where CPD detains people who may be subject to interrogation. Clinic students also documented that contrary to the Decree, many of the visiting rooms that CPD is required to maintain in every police station did not allow for private and confidential meetings between people in custody and their attorneys. The Clinic presented the Report and our findings to the court. In response, the Honorable Judge Neil H. Cohen directed CPD to work with the Clinic to ensure the installation of appropriate signs and remedy the documented deficiencies with respect to privacy. We are administering a survey to people at their first court appearance to provide the court with additional information about the reasons why people in CPD custody have not promptly accessed phones and counsel. Our preliminary findings indicate that CPD has failed to offer phones or provided the Public Defender’s 24-hour number to people subject to police interrogation. A quarter of the people surveyed report that CPD interrogated them without access to counsel. In addition, Clinic students have engaged in targeted outreach to people at risk of arrest and criminal defense attorneys in Chicago, created fantastic flyers, social media, and written material to educate people about their rights under the Decree, and developed a long-form interview tool to gain additional insight about barriers to access to counsel and phones.

Individual Cases

While we fight for systemic change, the Clinic has continued its tradition of excellence in serving individuals and families in need.

Clinic students won a stage three post-conviction hearing with our client Christopher Ellis before the Honorable Carol Howard in Cook County Circuit Court that can result in vacating Mr. Ellis’s conviction. Two Chicago police officers pulled Mr. Ellis out of his car, beat and tased him, and then falsely accused Mr. Ellis of aggravated battery against the police officers to cover up their abuse. Mr. Ellis was convicted and sentenced to six years in prison. Based on a phenomenal set of briefs written by Clinic students Hannah V.L. George, ’24, and Becky Marvin, ’24, and Professor Herschella Conyers ’ students Amara Shaikh, ’24 , and Liam Grah, ’25, in the Criminal and Juvenile Justice Clinic and Becky Marvin ’s outstanding oral argument, Judge Howard found that the Clinic has made a substantial showing of Mr. Ellis’s innocence and the ineffective assistance of his trial counsel. Judge Howard offered the highest praise to the students’ work. We expect Mr. Ellis’s case to go to trial in the fall.

 Erin Yonchak,’24, presented Clifton Young’s case before the Illinois Torture and Inquiry Relief Commission. Erin’s presentation and supporting written memorandum were nothing short of superb. As a result of Erin’s scrupulous investigation, factual and legal determinations, and recommendations, the Torture Commission found credible evidence that Mr. Young was tortured by Chicago police and ordered a full evidentiary hearing in Cook County Circuit Court that may result in his freedom after having served more than twenty years in prison.

Amrita Krishnan, ’25, is investigating a novel claim of police torture before the Illinois Torture Commission that is based on Chicago police detectives’ exploitation of a person’s withdrawal symptoms from heroin and denial of medical treatment to obtain a confession. This is the first of a series of claims of torture before the Commission based on deliberate indifference to a person in custody’s severe physical and psychological pain associated with drug withdrawal to leverage an incriminating statement. Amrita’s legal and medical research into whether and under what circumstances drug withdrawal can form a basis for a torture claim is precedential. It has the power to establish the governing legal standards in Illinois for assessing torture claims involving withdrawal.

Gabbie Zook, ’24 , Hannah V.L. George, ’24 , and Becky Marvin, ’24, led an investigation with a client who was repeatedly sexually assaulted by a Chicago police officer in public housing when she was a mere teenager. The Clinic helped to connect our client with the Chicago Torture Justice Center to provide her with critical support as she continues to work through her trauma from the repeated assaults. We face a myriad of legal challenges because of the years that have passed since the assaults and Illinois law that protects municipalities from liability when police officers abuse their state power to sexually assault people, but we remain committed to supporting our client in her fight for a measure of justice and healing. Our students’ work has shined a light on a path forward.

Policy Projects

Chicago police transparency.

Natalie Cohn-Aronoff, ’24 , and Amber Hunter, ’25, have led a critical project to prevent the return to a state of police impunity in Chicago. The Clinic is responding to the Fraternal Order of Police’s (FOP’s) efforts to shroud in secrecy the adjudication of cases in which Chicago police officers have been found to have committed the most serious forms of misconduct to warrant firing or suspension of more than a year. After the FOP won an arbitration award that sought to end a sixty-year history of public hearings before a neutral body to be replaced by secret hearings behind closed doors by a handful of handpicked arbitrators who have a long track record of protecting Chicago police officers from accountability, the Clinic began work with a coalition of community, civil rights, and good government groups organized to stop the FOP from turning back the clock on our progress. We drafted press releases and an op-ed that lifted the threat of Chicago police impunity to visibility. We drafted policy and legal material for City Council to provide the basis for challenging the arbitrator’s award. We provided testimony in public hearings that was widely cited in the media. Our work supporting the organization of community members persuaded the Mayor and City Council to reject the Arbitrator’s award by a 3/5 vote in City Council and challenge the award in court. The Cook County Circuit Court then ruled that the Arbitrator’s award violated fundamental state policy in Chicago police transparency and accountability and ordered that the Chicago police disciplinary cases must remain open to the public. The FOP has filed a notice of appeal. A team of Clinic students led by Ben Postone, ’24, is drafting an amicus brief before the Illinois Court of Appeals on behalf of the broad community-based coalition that will explain the nature and strength of the public interest at stake.

At the same time, Clinic students have conducted extensive research and consulted experts in labor law to draft proposed state legislation that requires the public adjudication of Chicago police misconduct cases. The Clinic is collaborating with stakeholders to devise a path to establish law that will guarantee public transparency on CPD misconduct now and in the future. The Clinic has also drafted potential municipal legislation that would enhance Chicago’s Civilian Office of Police Accountability’s (COPA’s) efforts to promote greater transparency and accountability by enabling COPA to promptly publicly release summaries of completed misconduct investigations, prosecute disciplinary proceedings that result from COPA investigations, and restrict the Police Department’s power to overturn misconduct findings only for clear error and disciplinary recommendations only for abuse of discretion.

Sam Hallam, ’25, and Katherine Stanton, ’25, are leading efforts to remedy other aspects of FOP’s new collective bargaining contract that thwart police accountability and transparency in Chicago, including a provision that prohibits the videotaping of conversations between officers and supervisors after a police officer shoots a community member. The recording and use of such conversations are critical tools to remedy the longstanding code of silence in the CPD—a code that has encouraged officers to manufacture a common narrative when an officer shoots or kills a person or is otherwise accused of misconduct.

Medical-Legal Partnership with University of Chicago Trauma Center

Rosie Gruen, ’25 , and Sam Hallam, ’25, have led a medical-legal project that we launched last year with the Trauma Center at the University of Chicago Medical Center (UCMC) and pro bono attorneys from the Akerman law firm to prevent police from to violating patient civil rights and medical privacy and interfering with critical medical care. We formed this partnership to address reports from the doctors and staff at the Medical Center of police abuse of patients who have suffered gunshot injuries; coercive interrogations of people who are being treated for serious injuries; interference with medical care and patient autonomy over medical decisions; searches and seizures of patients’ personal property; invasions of patient privacy and personal health information; shackling and physical abuse of patients; and forcing medical personal to perform invasive tests on patients. The Clinic team has been conducting and working to publish empirical research on interactions between police and professionals and staff at the Medical Center and patients and their family members. In addition to the conducting approximately fifty long-form interviews, the Clinic has researched the intersection of property law, criminal law and procedure, privacy law, constitutional law, and administrative regulations and practices in medical settings around the United States. Students have also consulted with national medical and legal experts. Based upon our research, the Clinic developed a first draft of recommended UCMC policies for internal feedback to prevent ongoing civil rights violations and interference with patient care. Our research has also taught us that despite similar civil rights violations in hospital settings and interference by law enforcement with medical treatment, there is a lack of model policies or established best practices on the subject. We are hopeful that the publication of our research and the policies that we develop at UCMC will serve as a model for hospitals throughout the country and prevent civil and human rights violations and improve health outcomes in the Trauma Center and beyond.

Partnership with the Cook County Public Defender and Zealous

We also built on our partnership with the Cook County Public Defender’s Office and Zealous, a national non-profit dedicated to supporting public defender offices, to identify and address systemic issues in the criminal legal system that deprive clients of the Public Defender and Clinic access to justice. Darius Diamond, ’24 , Gabbie Zook, ’24 , and Katherine Stanton, ’25, have led our efforts on this project. This year, our focus has been to support the Public Defender’s work to create two holistic community defender offices in Chicago—the first is scheduled to open this fall in the Roseland community on Chicago’s South Side. The second will be in the Austin community on the West Side. Clinic students have been on the ground floor in designing the offices and services with community members, public defenders, and people in jail. We are developing plans for Clinic students to maintain a regular presence in the Community Defender Offices to work with public defenders and their clients in addressing police accountability and other systemic barriers to justice.

In addition, students are working with public defenders in Cook County to achieve greater independence from county prosecutors and judges when advocating with their clients to change and enjoin laws, policies and practices that impair the ability of public defenders to represent their clients and to improve the criminal legal system. For example, Clinic students are currently working with the Public Defender to explore ways to change the law to give the Public Defender the power to retain counsel to bring affirmative civil rights litigation.

Captcha / Bot check:

Posted: 30-Aug-24

Location: Phoenix, Arizona

Type: Full Time

Required Education:

Internal Number: 723005

The Division of Plastic and Reconstructive Surgery at Phoenix Children’s is recruiting for a Section Chief of the Hand and Microsurgical Program.  We are seeking a plastic surgeon who subspecializes in hand surgery, is CAQ certified/eligible and is at the Associate/Full Professor level.  This position requires leadership skills and experience to guide a growing program, and the desire to build a robust clinical practice. He/she would be integrally involved in the academic and educational advancement and training of Mayo plastic surgery residents, the University of Arizona orthopedic hand surgery fellows, and the University of Arizona and Creighton University medical students.  This surgeon will be responsible for hand trauma call coverage.  Microsurgery, including replantation and free tissue reconstruction, is required as well as participation in complex wound reconstruction.  A strong interest in spastic paralysis, arthrogryposis and microvascular surgery is highly desirable. 

The Division of Plastic Surgery at Phoenix Children’s provides surgical treatment for the full spectrum of pediatric congenital and acquired conditions such as cleft lip and palate, craniosynostosis, hemifacial microsomia, vascular anomalies, hand anomalies, and traumatic injuries.  Plastic Surgery participates in several multi-disciplinary programs at Phoenix Children’s. The Phoenix Children’s Plastic Surgery team is fellowship trained in Craniofacial Surgery, Pediatric Plastic Surgery, and Pediatric Hand Surgery.

Phoenix Children’s Hospital is Arizona’s only children’s hospital recognized by U.S. News & World Report’s Best Children’s Hospitals.  Phoenix Children’s provides world-class inpatient, outpatient, trauma, emergency and urgent care to children and families in Arizona and throughout the Southwest.  As one of the largest children’s hospitals in the country, Phoenix Children’s provides care across more than 75 pediatric specialties.  Phoenix Children’s is poised for continued growth in quality patient care, research and medical education.

This position offers competitive compensation, excellent benefits, including employer retirement contribution, generous vacation/meeting time, CME funds, health and dental benefits, disability, and life insurance.

How to Initiate an Application:

Candidates are kindly asked to submit a CV and cover letter to Raye Jean Becker, Physician Talent Acquisition Partner at Phoenix Children’s. Letters of reference may be requested. 

Raye Jean Becker 

Physician Talent Acquisition Partner

Phoenix Children's Hospital

[email protected]

(O) (602)933-1101 - (C) (319)290-2906

Warning: The NCBI web site requires JavaScript to function. more...

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Hughes RG, editor. Patient Safety and Quality: An Evidence-Based Handbook for Nurses. Rockville (MD): Agency for Healthcare Research and Quality (US); 2008 Apr.

Patient Safety and Quality: An Evidence-Based Handbook for Nurses.

Chapter 8 health services research: scope and significance.

Donald M. Steinwachs ; Ronda G. Hughes .

Affiliations

The provision of high-quality, affordable, health care services is an increasingly difficult challenge. Due to the complexities of health care services and systems, investigating and interpreting the use, costs, quality, accessibility, delivery, organization, financing, and outcomes of health care services is key to informing government officials, insurers, providers, consumers, and others making decisions about health-related issues. Health services researchers examine the access to care, health care costs and processes, and the outcomes of health services for individuals and populations.

The field of health services research (HSR) is relied on by decisionmakers and the public to be the primary source of information on how well health systems in the United States and other countries are meeting this challenge. The “goal of HSR is to provide information that will eventually lead to improvements in the health of the citizenry.” 1 Drawing on theories, knowledge, and methods from a range of disciplines, 2 HSR is a multidisciplinary field that moves beyond basic and applied research, drawing on all the health professions and on many academic disciplines, including biostatistics, epidemiology, health economics, medicine, nursing, operations research, psychology, and sociology. 3

In 1979, the Institute of Medicine defined HSR as “inquiry to produce knowledge about the structure, processes, or effects of personal health services” 4 (p. 14). This was expanded upon in 2002 by AcademyHealth, the professional organization of the HSR field, with the following definition, which broadly describes the scope of HSR:

Health services research is the multidisciplinary field of scientific investigation that studies how social factors, financing systems, organizational structures and processes, health technologies, and personal behaviors affect access to health care, the quality and cost of health care, and ultimately our health and well-being. Its research domains are individuals, families, organizations, institutions, communities, and populations. 5

More specifically, HSR informs and evaluates innovations in health policy. These include changes in Medicare and Medicaid coverage, disparities in access and utilization of care, innovations in private health insurance (e.g., consumer-directed health plans), and trends among those without health insurance. 6–10 The health care industry continues to change, and HSR examines the impact of organizational changes on access to care, quality, and efficiency (e.g., growth in for-profit hospital systems). As new diagnostic and treatment technologies are introduced, HSR examines their impact on patient outcomes of care and health care costs.

The definition of HSR also highlights the importance of examining the contribution of services to the health of individuals and broader populations. HSR applied at the population level is particularly important in understanding health system performance and the impact of health policy on the public’s health. In the United States, the National Healthcare Quality Report, 11 National Healthcare Disparities Report, 12 and Healthy People Year 2010 13 exemplify our capacity for monitoring quality and assessing change. These reports tell us that the American quality of care is inconsistent and could be substantially improved. The associated cost of health care services is monitored by the Centers for Medicare & Medicaid Services (CMS). CMS reports tell us that American health care is the most expensive in the world, consuming approximately 16 percent of America’s gross domestic product. 14

Beyond health policy, HSR examines the process of care and the interactions of patients and providers. For example, HSR methods have been developed to describe doctor-patient communication patterns and examine their impact on patient adherence, satisfaction, and outcomes of care. 15–17

Advances in HSR measurement methodologies have made possible policy innovations. Prospective payment of hospitals, nursing homes, and home health care by Medicare became possible with the development of robust case-mix measurement systems. 18 CMS was able to initiate a pay-for-performance demonstration, rewarding hospitals with better quality performance, using valid and robust measures of quality. 14 Innovations in health care policy are frequently made possible by advances in measurement of indicators of health system performance.

• History of Health Services Research

The history of HSR is generally considered to have begun in the 1950s and 1960s with the first funding of grants for health services research focused on the impact of hospital organizations. 19 , 20 On the contrary, HSR began with Florence Nightingale when she collected and analyzed data as the basis for improving the quality of patient care and outcomes. 21 Also significant in the history of HSR was the concern raised about the distribution, quality, and cost of care in the late 1920s that led to one of the first U.S. efforts to examine the need for medical services and their costs, undertaken in 1927 by the Committee on the Costs of Medical Care. 22 The committee published a series of 28 reports and recommendations that have had a significant impact on how medical care is organized and delivered in the United States. 23 Other key reports of historical importance to HSR were, for example, the national health survey in 1935–1936 by the Public Health Service, the inventory of the nation’s hospitals by the American Hospital Association’s Commission on Hospital Care in 1944, and studies by the American Hospital Association’s Commission on Chronic Illness on the prevalence and prevention of chronic illness in the community. 23

In 1968, the National Center for Health Services Research and Development was established as part of the U.S. Public Health Services to address concerns with access to health services, quality of care, and costs. The Center funded demonstration projects to measure quality and investigator-initiated research grants. In 1989, Congress created the Agency for Health Care Policy and Research and broadened its mission to focus attention on variations in medical practice, patient outcomes of care, and the dissemination of evidence-based guidelines for the treatment of common disorders. Later Congress reauthorized and renamed the agency, Agency for Healthcare Research and Quality (AHRQ). AHRQ provides Federal leadership for the field, investing in methods for quality measurement, development of patient safety methods, and health information technology (e.g., electronic health records and decision support systems).

The Federal role in HSR has expanded over time, and investments in HSR are made by multiple Federal agencies. In addition to AHRQ, the U.S. Department of Veterans Affairs, Centers for Disease Control and Prevention, the National Institutes of Health, CMS, and other Federal agencies fund HSR. The diversification of funding comes, in part, from the recognition that HSR is important in managing health care systems, such as the Veterans Health Administration, and provides essential information on the translation of scientific discoveries into clinical practice in American communities, such as those funded by National Institutes of Health. It is estimated that total Federal funding of HSR was $1.5 billion in 2003, of which AHRQ was responsible for approximately 20 percent. 24

Private funding of HSR has also grown over time. Funding by private foundations has a significant role and complements Federal funding. Among the many foundations funding HSR are the Robert Wood Johnson Foundation, Commonwealth Fund, Kaiser Family Foundation, Kellogg Foundation, and Hartford Foundation. Other private funding sources include the health care industry, for example, pharmaceutical companies, health insurers, and health care systems.

• Goals for Health Services and Patient Outcomes

The goal of health services is to protect and improve the health of individuals and populations. In a landmark 2001 report, Crossing the Quality Chasm: A New Health System for the 21 st Century, 25 the Institute of Medicine (IOM) of the National Academy of Sciences proposed that the goals for health services should include six critical elements:

• Patient Safety: Patients should not be harmed by health care services that are intended to help them. The IOM report, To Err Is Human , 26 found that between 46,000 and 98,000 Americans were dying in hospitals each year due to medical errors. Subsequent research has found medical errors common across all health care settings. The problem is not due to the lack of dedication to quality care by health professionals, but due to the lack of systems that prevent errors from occurring and/or prevent medical errors from reaching the patient.
• Effectiveness: Effective care is based on scientific evidence that treatment will increase the likelihood of desired health outcomes. Evidence comes from laboratory experiments, clinical research (usually randomized controlled trials), epidemiological studies, and outcomes research. The availability and strength of evidence varies by disorder and treatment.
• Timeliness: Seeking and receiving health care is frequently associated with delays in obtaining an appointment and waiting in emergency rooms and doctors’ offices. Failure to provide timely care can deny people critically needed services or allow health conditions to progress and outcomes to worsen. Health care needs to be organized to meet the needs of patients in a timely manner.
• Patient Centered: Patient-centered care recognizes that listening to the patient’s needs, values, and preferences is essential to providing high-quality care. Health care services should be personalized for each patient, care should be coordinated, family and friends on whom the patient relies should be involved, and care should provide physical comfort and emotional support.
• Efficiency: The U.S. health care system is the most expensive in the world, yet there is consistent evidence that the United States does not produce the best health outcomes 27–30 or the highest levels of satisfaction. 31 The goal is to continually identify waste and inefficiency in the provision of health care services and eliminate them.
• Equity: The health care system should benefit all people. The evidence is strong and convincing that the current system fails to accomplish this goal. The IOM report, Unequal Treatment , 32 documented pervasive differences in the care received by racial and ethnic minorities. The findings were that racial and ethnic minorities are receiving poorer quality of care than the majority population, even after accounting for differences in access to health services.

Crossing the Quality Chasm concludes that for the American health care system to attain these goals, transformational changes are needed. 25 The field of HSR provides the measurement tools by which progress toward these goals is assessed, as seen in the National Healthcare Quality Report. 11 Equally important, health services researchers are developing and evaluating innovative approaches to improve quality of care, involving innovations in organization, financing, use of technology, and roles of health professionals.

• Evaluating the Quality of Health Care

HSR evaluation of quality of care has proven to be an inexact science and complex, even though its definition is relatively simple: “Quality of care is the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge.” 1 This definition draws attention to the importance of the application of current professional knowledge in the diagnostic and treatment processes of health care. The goal of quality care is to increase the likelihood of achieving desired health outcomes, as expressed by the patient.

The complexity in measuring quality comes from gaps in our knowledge regarding which services, for which patients, will actually improve the likelihood of desired health outcomes. Also, patients need not have the same desired health outcomes and therefore might not receive the same care for an identical health problem, further complicating the measurement of quality of care. Quality measurement has advanced substantially, but it remains early in its development.

The conceptual framework widely applied in evaluating quality comes from years of research and the insightful analysis of Avedis Donabedian. 33 He formalized the conceptual model for describing, analyzing, and evaluating the quality of care using three dimensions: (1) structure, (2) process, and (3) outcome. This model is applied in the evaluation of health services and the accreditation of health care providers and organizations.

Seminal research about variation in the quality of care patients received brought to focus the need to monitor and improve the quality of health care. Wennberg and Gittelsohn 34 , 35 found wide variation in practice patterns among community physicians, surgical procedures, and hospitals. Brook and colleagues 36 found that a small number of physicians were responsible for a large number of improperly administered injections. This was the precursor to research on the appropriateness of procedures and services under specific circumstances 36 , 37 as well as the development of practice guidelines and standards for quality care. 38 Yet the challenge of research on variations in care is the implication of the inappropriateness of care. The challenge is determining whether there is a direct relationship between rates of utilization, variations in appropriateness, and quality of care.

One of the challenges in understanding quality, how to measure it, and how to improve it is the influence of physical, socioeconomic, and work environments. Income, race, and gender—as well as individuals within society and organizations—influence health and risks to health. 40 Researchers have found that differences in internal factors, such as collaborative relationships with physicians, decentralized clinical decisionmaking, and positive administrative support, impact nurse and patient outcomes 41 , 42 and the quality and safety of care. 43 Differences in external factors, such as insurance and geographic location, can influence access to available health care professionals and resources, what type of care is afforded patients, and the impact of care on patients. The structure, process, and outcome dimensions of quality are influenced by both internal and external factors.

Structure of Health Care

The structure of health care broadly includes the facilities (e.g., hospitals and clinics), personnel (e.g., number of nurses and physicians), and technology that create the capacity to provide health services. Structural characteristics are expected to influence the quality of health care services. One component in the accreditation of health care facilities (e.g., hospitals, nursing homes) is the review of the adequacy of structural characteristics, including staffing, on-call resources, technology, and support services (laboratory, pharmacy, radiology). The structural resources of health care facilities and organizations are the foundation upon which quality health care services are provided.

Process of Care

The interactions between the health care providers and patients over time comprise the process of health care. The process of care may be examined from multiple perspectives: the sequence of services received over time, the relationship of health services to a specific patient complaint or diagnosis, and the numbers and types of services received over time or for a specific health problem. Examining the time sequence of health care services provides insights into the timeliness of care, organizational responsiveness, and efficiency. Linking services to a specific patient complaint or diagnosis provides insights into the natural history of problem presentation and the subsequent processes of care, including diagnosis, treatment, management, and recovery. Examining the natural history of a presenting health complaint across patients will reveal variations in patterns of care. For example, presenting complaints for some patients never resolve into a specific diagnosis. An initial diagnosis may change as more information is obtained. Patients may suffer complications in the treatment process. Also, the process of care may provide insights into outcomes of care (e.g., return visit for complications). Generally it is not possible to examine the process of care and determine how fully the patient has recovered prior health status by the end of the episode of treatment. For this reason, special investigations are needed to assess outcomes of care.

Evaluation of the process of care can be done by applying the six goals for health care quality. 25 Was the patient’s safety protected (i.e., were there adverse events due to medical errors or errors of omission)? Was care timely and not delayed or denied? Were the diagnosis and treatments provided consistent with scientific evidence and best professional practice? Was the care patient centered? Were services provided efficiently? Was the care provided equitable? Answers to these questions can help us understand if the process of care needs improvement and where quality improvement efforts should be directed.

Outcomes of Care

The value of health care services lies in their capacity to improve health outcomes for individuals and populations. Health outcomes are broadly conceptualized to include clinical measures of disease progression, patient-reported health status or functional status, satisfaction with health status or quality of life, satisfaction with services, and the costs of health services. Historically, quality assessment has emphasized clinical outcomes, for example, disease-specific measures. However, disease-specific measures may not tell us much about how well the patient is able to function and whether or not desired health outcomes have been achieved. To understand the patients’ outcomes, it is necessary to ask patients about their outcomes, including health status, quality of life, and satisfaction with services. HSR has developed valid and robust standardized questionnaires to obtain patient-reported information on these dimensions of health outcomes. As these are more widely applied, we are learning about the extent to which health care services are improving health.

Public Health Perspective on Health Services

Another perspective on health care services comes from the field of public health in which preventive health services are conceptualized at three levels: primary, secondary, and tertiary prevention. 44 Primary prevention includes immunizations, healthy lifestyles, and working and living in risk-free environments. Primary prevention seeks to prevent disease or delay its onset. Examples of primary prevention include immunizations against infectious disease; smoking prevention or cessation; and promotion of regular exercise, weight control, and a balanced diet. Secondary prevention includes the range of interventions that can reduce the impact of disease morbidity once it occurs and slow its progression. With the increasing burden of chronic diseases, much of the health care provided is directed at secondary prevention. Tertiary prevention is directed at rehabilitation for disabilities resulting from disease and injury. The goal of tertiary prevention is to return individuals to the highest state of functioning (physical, mental, and social) possible. The public health framework expands the structure, process, and outcome conceptual model by identifying the role and value of health services at three stages: prior to onset of disease, disease management, and disease recovery and rehabilitation.

• Methodologies and Data Sources Used in Health Services Research

The interdisciplinary character of HSR draws on methods and data sources common to the many disciplines that form the intellectual underpinnings of the field. This section discusses the measurement of effectiveness and efficacy of health services and some of the methods and data sources used to understand effectiveness. Effectiveness is one of the six goals of health services. Effectiveness is interrelated with the other five goals, and some of these interrelationships are discussed.

Efficacy and Effectiveness

An important distinction is made between efficacy and effectiveness of health services. Efficacy is generally established using randomized controlled trial (RCT) methods to test whether or not clinical interventions make a difference in clinical outcomes. A good example is the series of studies required for Food and Drug Administration approval of a new drug before it is certified as safe and efficacious and allowed to be used in the United States. Efficacy research is generally done with highly select groups of patients where the impact of the drug can be validly measured and results are not confounded by the presence of comorbid conditions and their treatments. The efficacy question is: What impact does a clinical intervention have under ideal conditions?

In contrast, effectiveness research is undertaken in community settings and generally includes the full range of individuals who would be prescribed the clinical intervention. Many of these individuals will have multiple health problems and be taking multiple medications, unlike those who were recruited to the RCT. Effectiveness research is seeking to answer the question: Who will benefit from the clinical intervention among all those people in the community who have a specific health problem(s)?

Both efficacy and effectiveness questions are important. Logically, effectiveness research would be conducted after finding the clinical intervention to be efficacious. However, there are many treatments for which no efficacy information exists; the treatments are accepted as common practice, and it would not be ethical to withhold treatments from a control group in an RCT. As a result, effectiveness research may not have the benefit of efficacy findings.

The routine use of an RCT to evaluate efficacy began in the 1960s and is the accepted procedure for evaluating new medications. However, this standard is not applied across all health care services and treatments. Most surgical procedures are not evaluated using an RCT. Intensive care units have never been evaluated using an RCT, nor are nurse staffing decisions in hospitals or the evaluation of many medical devices. We currently accept different standards of evidence depending on the treatment technology. As a result, the level of evidence guiding clinical and public health decisionmaking varies.

Methods for Effectiveness Research

A variety of methods are used to examine effectiveness of health services. RCT methods are not usually applied in effectiveness research because the intervention being studied has demonstrated efficacy or is acknowledged as accepted clinical practice. When this is true, it would be unethical to randomly assign individuals who would be expected to benefit from the intervention to a control group not receiving an efficacious treatment. We will discuss when RCT methods can be used to test effectiveness and provide several examples. More commonly, effectiveness research uses statistical methods for comparing treatments across nonequivalent groups.

RCT and Policy Research

RCT study methods can be used to compare the effectiveness and costs of services across randomly assigned representative population groups. In an RCT, study participants are randomly assigned to two or more groups to ensure comparability and avoid any selection bias. At least one group receives an intervention (clinical, organizational, and/or financial), and usually one group serves as a control group, receiving a current standard of care, sometimes referred to as “usual care.” Two examples of effectiveness research using an RCT methodology to answer policy questions are described.

Health insurance experiment

Probably the first application of RCT methods in effectiveness research was undertaken in the 1970s as a health insurance experiment. The experiment was designed to test the impact on cost and health outcomes of different levels of insurance deductibles and copayment rates. A total of 3,958 people, ages 14–61, were randomized to a set of insurance plans and followed over 3 to 5 years. 45

The economic impact of receiving free care in one plan versus being in a plan requiring payment out-of-pocket of deductibles and co-insurance had the expected impact on utilization. Those paying a share of their medical bills utilized approximately one-third fewer doctor visits and were hospitalized one-third less frequently.

The impact on 10 health measures of free health insurance versus paying a portion of medical care costs out of pocket was evaluated. The findings were that there was largely no effect on health as measured by physical functioning, role functioning, mental health, social contacts, health perceptions, smoking, weight, serum cholesterol, diastolic blood pressure, vision, and risk of dying. 46 The exceptions were that individuals with poor vision improved under free care, as did low-income persons with high blood pressure.

Medicare preventive services experiment

A more recent example of RCT methods applied in HSR is the Baltimore Medicare Preventive Services Demonstration. The study evaluated the impact on cost and outcomes of offering a defined preventive services package to Medicare beneficiaries. This was compared to usual Medicare coverage, which paid for few preventive services. The preventive services coverage being evaluated included an annual preventive visit with screening tests and health counseling. The physician could request a preventive followup visit during the year, which would also be covered. Medicare beneficiaries (n = 4,195) were randomized to preventive services (the intervention group) or usual care (the control group). Sixty-three percent of those in the intervention group had at least one preventive visit. Significant differences were found in health outcomes between intervention and control groups. Among the 45 percent with declining health status, as measured by the Quality of Well-Being scale, 47 the decline was significantly less in the group offered preventive services. Mortality was also significantly lower in the intervention group. There was no significant impact of preventive services on utilization and cost. 48

Comparative Clinical Effectiveness and RCTs

The passage of the Medicare Prescription Drug, Improvement, and Modernization Act of 2003 (MMA) included provisions for the funding of comparative effectiveness studies. AHRQs’ Effective Health Care Program (authorized under MMA Section 1013) informs comparative clinical effectiveness efforts by conducting and supporting research and evidence syntheses on priority topics to CMS.

Comparative effectiveness studies ask the question: Which of the alternative treatments available is best and for whom? Interest in this question reflects how advances in science have provided multiple treatment options for many conditions. Currently, there is no systematic process by which treatment options are compared and matched to the needs of different types of patients. Frequently, patients are started on one treatment and then may be prescribed alternative treatments if they cannot tolerate the treatment or if it is not as effective as expected. RCT methods can be used to evaluate comparative effectiveness of an intervention in treatment and control populations. This is ethical to do when there is no evidence that the treatments are not equivalent.

An example of a comparative effectiveness study using RCT methods is the CATIE study, testing alternative antipsychotic medications in the treatment of schizophrenia. A study of 1,493 persons with schizophrenia compared five of the newer antipsychotic medications (second generation) and also compared them against one of the first-generation antipsychotic medications. 49 , 50 The findings were surprising to many. The second-generation antipsychotics were no more effective in controlling psychotic symptoms than the first-generation drug. There was one exception, the drug Clozapine. 51 Furthermore, second-generation medications showed significant side effects that can affect health outcomes. These included weight gain, metabolic changes, extrapyramidal symptoms, and sedation effects. Each medication showed a somewhat different side-effect risk profile. From a positive perspective, the findings indicated that the clinician and patient can choose any of these medications as first-line treatment except Clozapine, which is generally used for treatment-resistant cases due to more intensive clinical monitoring requirements. The ultimate choice of treatment will depend on the patient’s ability to tolerate side effects that vary by drug.

The conduct of any RCT is resource intensive, requiring the recruitment of participants, and participants must give informed consent to be randomized. The rationale for making this investment may depend on the importance of the policy or practice issue. As shown, RCT methods can be applied to address policy and clinical care concerns with effectiveness. To the extent that the RCT includes a broad cross-section of people who would be affected by a policy or receive a clinical treatment, this methodology provides robust effectiveness findings.

Comparing Effectiveness and Costs Across Nonequivalent Groups

A range of statistical methods can be used to compare nonequivalent groups (i.e., groups receiving different treatments or exposures when there has been no random assignment to ensure comparability of group membership). It is not practical to review all the specific statistical approaches that can be applied. In general, the statistical methods seek to adjust for nonequivalent characteristics between groups that are expected to influence the outcome of interest (i.e., make the comparisons fair). Statistical adjustment for nonequivalent characteristics is referred to as “risk adjustment.” The foundations for risk adjustment come from multiple disciplines. Epidemiologic methods are routinely used to identify and estimate disease and outcomes risk factors. These methods are applicable in comparative effectiveness evaluations. 52

Operations research uses methods for creating homogeneous groups predictive of cost or disease outcomes. These methods are used to make fair comparisons across provider practices and health plans and to control the cost of health care. They also have been used in designing payment systems, including diagnostically related groups used in Medicare’s Prospective Payment System to reimburse hospitals for care rendered to Medicare beneficiaries, and resource-based relative value scales used in Medicare’s physician payment system. Diagnostically related groups are used to standardize and rationalize patient care in hospitals—provided largely by nurses and other health professionals—and resource-based relative value scales are used to standardize and rationalize patient care in outpatient settings—care provided largely by physicians and nurse practitioners. Other disciplines also contribute to our understanding of risk factors for the range of health outcomes, including mortality, health and functional status, quality of life, and rehabilitation and return to work

The basic form of a nonequivalent group comparison includes adjusting the outcomes of each group for the risk factors that are known to affect the occurrence and/or severity of the outcomes being evaluated. 53 For many disease outcomes, risk factors include demographic characteristics (age, gender), disease-specific risk factors (e.g., health behaviors, environmental exposures, and clinical indicators of risk), and indicators of health status (e.g., presence of comorbid conditions). After adjustment for risks factors, variations in access to care and quality of care (e.g., choice of treatment and adherence to treatment) would be expected to explain the remaining observed variation in outcomes. Ideally, the nonequivalent group comparison makes it possible to compare the effectiveness of alternative treatments and assess the impact of poor access to care. One limitation of this methodology is the limit of current knowledge regarding all relevant disease risk factors. Even when risk factors are known, limits on data availability and accuracy of risk factor measurement have to be considered.

Risk adjustment methods are also used to make cost comparisons across health care providers to determine which providers are more efficient. Instead of adjusting for disease risk factors, adjustments are made for the costliness of the patient mix (case mix) and differences in costs of labor, space, and services in the local area. Comparisons may be made to assess efficiency of providing specific services (e.g., hospitalization, office visit, or laboratory test). These comparisons would use case-mix measures that adjust for the costliness of different mixes of hospital episodes. 18 Comparisons of the total cost of care for insured populations would apply case-mix measures that adjust for disease and health factors that affect total cost of care. 54

Data Sources for Effectiveness Research

A range of data sources is used in effectiveness research, including administrative and billing data, chart reviews and electronic health records, and survey questionnaires. The following discussion identifies major attributes of each category of data source.

Medical records

Medical records document the patient’s presenting problem or condition, tests and physical exam findings, treatment, and followup care. The medical record is generally the most complete source of clinical information on the patient’s care. However, medical records are generally not structured to ensure the physician or other provider records all relevant information. The completeness of medical record information can vary considerably. If the patient does not return for followup care, the medical record may provide no information on outcomes of care. If a patient sees multiple providers during the course of treatment, each with its own separate medical record, complete information on treatment requires access to all the records. Lack of standardization of medical records also can make abstracting records for research very resource intensive.

Administrative and billing data

Health care providers generally have administrative and billing data systems that capture a limited and consistent set of data on every patient and service provided. These systems uniquely identify the patient and link information on insurance coverage and billing. Each service received by the patient is linked to the patient using a unique patient identifier. Services are identified using accepted codes (e.g., ICD9-CM, CPT), together with date of service, provider identifier, and other relevant information for billing or management reporting. Administrative data make it possible to identify all individual patients seen by a provider and produce a profile of all services received by each patient over any defined time period. Administrative data are comprehensive and the data are generally complete (i.e., no problems with missing data). The primary limitation is the data set collected by administrative systems is very limited and lacks the detail of the medical record.

Administrative data systems can provide some insights into quality and outcomes of care. AHRQ has developed software that provides quality indicators and patient safety measures using one administrative data set, hospital discharge abstracts. 55 , 56 Utilization-based indicators of outcome include rehospitalization, return to surgery during a hospitalization, and incidence of complications; some systems include information on death. Administrative data can efficiently provide quality and outcomes indicators for defined populations and for health systems. Other applications of administrative data include assessing efficiency, timeliness, and equity. The limitation is that there are many health conditions and health outcomes that cannot currently be measured using administrative data.

Survey questionnaires

Neither the medical record nor the administrative data capture information on the patient’s experience in health or patient-reported outcomes of care. Survey questionnaires are routinely used to obtain information on patient satisfaction in health plans. A widely used example is the Consumer Assessment of Healthcare Providers and Systems or CAHPS. 57

Information on the impact of health conditions on health and functional status has to come from the patient. This may be obtained at the time of a visit or hospitalization. However, to assess patient outcomes of care, systematic followup of patients after the completion of treatment is generally required. This can be done using mail questionnaires, telephone interviews, or in-person interviews. The HSR field has developed health-status and quality-of-life measures that can be used no matter what health conditions the patient has. 47 , 58–60 Numerous condition-specific measures of outcome are also used. 53

Effectiveness research relies on a range of data sources. Some are routinely collected in the process of medical care and patient billing. Others may require special data collection, including medical record abstracts to obtain detailed clinical data and survey questionnaires to gain information on the patient’s perspective on treatment and outcomes. Efficient strategies for examining effectiveness may use administrative data to examine a limited set of data on all patients, and a statistically representative sample of patients for in-depth analysis using data from chart abstracts and survey questionnaires.

Using HSR Methods To Improve Clinical Practice

HSR research tools can be applied in clinical settings to improve clinical practice and patient outcomes. These tools are used as part of quality improvement programs in hospitals, clinics, and health plans. Two examples illustrate applications to improve quality-of-care performance.

Evidence-based treatment

For many chronic medical conditions, clinical research has evaluated the efficacy of diagnostic methods and treatment interventions. As a result, evidence-based reviews of research literature can provide a basis for establishing quality-of-care criteria against which to judge current practice. In a national study of quality of medical care, it was found that only 55 percent of patients received evidence-based treatments for common disorders and preventive care. 61 The researchers examined treatment for a range of health conditions, using a national sample of medical records abstracts. For each quality criterion, a classification was applied to determine if the quality-of-care deficiency was one of underuse, overuse, or misuse. Greater problems were found with underuse (46 percent) than with overuse (11 percent). Quality of care varied by condition: senile cataracts scored highest, 78 percent of recommended care received, and alcohol dependence scored lowest, 10 percent of recommended care received. Overall, only about half of recommended care was received, frequently due to underuse of services.

Researchers have sought to identify why rates of conformance with evidence-based treatments are low. Frequently cited barriers to evidence-based practice include physician disagreement with the evidence, perception that patients will not accept treatment, low ratings of self-efficacy as a provider of the treatment, and difficulty of integrating the evidence-based treatment into existing practice. 62 More needs to be learned how to assist health care providers to overcome barriers to the adoption of evidence-based practices.

The described data sources and methods can be applied in clinical settings to assess conformance to evidence-based quality criteria and provide feedback to clinicians. If electronic health records are available, the feedback and reminders may be directly incorporated into the medical record and seen by the clinician at the time of a visit. Intermountain Health Care utilizes its electronic health records to monitor adherence to evidence-based quality standards and to provide decision support to clinicians when seeing patients. This strategy has contributed to substantial improvements in their quality performance. 63

Outcomes management system

In 1988, Paul Ellwood proposed the adoption of outcomes management system (OMS) as a method to build clinical intelligence on “what treatments work, for whom, and under what circumstances.” 64 OMS would require linking information on the patient’s experience with outcomes of care and information on diagnosis and treatment that would usually come from the medical record.

In 1991, the Managed Health Care Association, an employer organization, brought together a group of employers and their health plan partners who were interested in testing the OMS concept in health plans. 65 To do so would require a set of methods that could be widely applied across health plans with differing information systems. The methodology chosen was for each of 16 health plans to identify all adult enrollees with at least two diagnoses of asthma over the previous 2 years. A stratified sample was chosen with half of the enrollees having more severe asthma (e.g., hospitalization or emergency room visit in the past 2 years) and the other enrollees having less severe asthma (outpatient visits only). Each adult received a questionnaire asking about their asthma treatment and health status. Followup surveys were done in each of 2 successive years to track changes over time.

The findings were compared to national treatment recommendations for adult asthma. 66 Across the health plans, 26 percent of severe asthmatics did not have a corticosteroid inhaler, and 42 percent used it daily, as recommended. 67 Only 5 percent of patients reported monitoring their asthma using a home peak flow meter. Approximately half of adults with asthma reported having the information they needed to avoid asthma attacks, to take appropriate actions when an asthma flare-up occurs, and to adjust medications when their asthma gets worse. Health plans used the baseline findings to develop quality-improvement interventions, which varied across health plans. Followup surveys of the patient cohort provided feedback to health plans on their success in improving asthma treatment and outcomes over time.

This chapter has provided a definition and history of the field of health services research and discussed how this field is examining quality-of-care issues and seeking to improve quality of care. Comparisons of current practice to evidence-based standards with feedback to clinicians and the integration of patient-reported outcomes are two examples of how HSR tools can be used to provide quality-improvement information for health care organizations. These examples utilize multiple data sources, including medical records, patient surveys, and administrative data. The opportunities for nurse researchers to provide invaluable contributions to the growing field of health services research are innumerable.

• Cite this Page Steinwachs DM, Hughes RG. Health Services Research: Scope and Significance. In: Hughes RG, editor. Patient Safety and Quality: An Evidence-Based Handbook for Nurses. Rockville (MD): Agency for Healthcare Research and Quality (US); 2008 Apr. Chapter 8.
• PDF version of this page (92K)

In this Page

Other titles in this collection.

• Advances in Patient Safety

Related information

• PMC PubMed Central citations
• PubMed Links to PubMed

Similar articles in PubMed

• The whole is greater than the sum of its parts: the anatomy of the Department of Veterans Affairs Medical District 17 Health Services Research and Development Field Program. [Health Serv Res. 1990] The whole is greater than the sum of its parts: the anatomy of the Department of Veterans Affairs Medical District 17 Health Services Research and Development Field Program. Demakis JG, Turpin RS, Conrad KJ, Stiers WM, Weaver FM, Sinacore JM, Cowper DC, Darcy LA, Huck MN, Friedman BS. Health Serv Res. 1990 Apr; 25(1 Pt 2):269-85.
• Telemedicine for the Medicare population: pediatric, obstetric, and clinician-indirect home interventions. [Evid Rep Technol Assess (Summ)...] Telemedicine for the Medicare population: pediatric, obstetric, and clinician-indirect home interventions. Hersh WR, Wallace JA, Patterson PK, Shapiro SE, Kraemer DF, Eilers GM, Chan BK, Greenlick MR, Helfand M. Evid Rep Technol Assess (Summ). 2001 Aug; (24 Suppl):1-32.
• Review Functionality and feedback: a realist synthesis of the collation, interpretation and utilisation of patient-reported outcome measures data to improve patient care [ 2017] Review Functionality and feedback: a realist synthesis of the collation, interpretation and utilisation of patient-reported outcome measures data to improve patient care Greenhalgh J, Dalkin S, Gooding K, Gibbons E, Wright J, Meads D, Black N, Valderas JM, Pawson R. 2017 Jan
• Health systems research in maternal and child health challenges, problems and prescriptions. [Indian J Matern Child Health. ...] Health systems research in maternal and child health challenges, problems and prescriptions. Deodhar NS. Indian J Matern Child Health. 1994 Oct-Dec; 5(4):88-94.
• Review Improving patient experience in primary care: a multimethod programme of research on the measurement and improvement of patient experience [ 2017] Review Improving patient experience in primary care: a multimethod programme of research on the measurement and improvement of patient experience Burt J, Campbell J, Abel G, Aboulghate A, Ahmed F, Asprey A, Barry H, Beckwith J, Benson J, Boiko O, et al. 2017 Apr

Recent Activity

• Health Services Research: Scope and Significance - Patient Safety and Quality Health Services Research: Scope and Significance - Patient Safety and Quality

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

• Yekaterinburg
• Novosibirsk
• Vladivostok

• Tours to Russia
• Practicalities
• Russia in Lists

Rusmania • Deep into Russia

Out of the Centre

Savvino-storozhevsky monastery and museum.

Zvenigorod's most famous sight is the Savvino-Storozhevsky Monastery, which was founded in 1398 by the monk Savva from the Troitse-Sergieva Lavra, at the invitation and with the support of Prince Yury Dmitrievich of Zvenigorod. Savva was later canonised as St Sabbas (Savva) of Storozhev. The monastery late flourished under the reign of Tsar Alexis, who chose the monastery as his family church and often went on pilgrimage there and made lots of donations to it. Most of the monastery’s buildings date from this time. The monastery is heavily fortified with thick walls and six towers, the most impressive of which is the Krasny Tower which also serves as the eastern entrance. The monastery was closed in 1918 and only reopened in 1995. In 1998 Patriarch Alexius II took part in a service to return the relics of St Sabbas to the monastery. Today the monastery has the status of a stauropegic monastery, which is second in status to a lavra. In addition to being a working monastery, it also holds the Zvenigorod Historical, Architectural and Art Museum.

Belfry and Neighbouring Churches

Located near the main entrance is the monastery's belfry which is perhaps the calling card of the monastery due to its uniqueness. It was built in the 1650s and the St Sergius of Radonezh’s Church was opened on the middle tier in the mid-17th century, although it was originally dedicated to the Trinity. The belfry's 35-tonne Great Bladgovestny Bell fell in 1941 and was only restored and returned in 2003. Attached to the belfry is a large refectory and the Transfiguration Church, both of which were built on the orders of Tsar Alexis in the 1650s.  

To the left of the belfry is another, smaller, refectory which is attached to the Trinity Gate-Church, which was also constructed in the 1650s on the orders of Tsar Alexis who made it his own family church. The church is elaborately decorated with colourful trims and underneath the archway is a beautiful 19th century fresco.

Nativity of Virgin Mary Cathedral

The Nativity of Virgin Mary Cathedral is the oldest building in the monastery and among the oldest buildings in the Moscow Region. It was built between 1404 and 1405 during the lifetime of St Sabbas and using the funds of Prince Yury of Zvenigorod. The white-stone cathedral is a standard four-pillar design with a single golden dome. After the death of St Sabbas he was interred in the cathedral and a new altar dedicated to him was added.

Under the reign of Tsar Alexis the cathedral was decorated with frescoes by Stepan Ryazanets, some of which remain today. Tsar Alexis also presented the cathedral with a five-tier iconostasis, the top row of icons have been preserved.

Tsaritsa's Chambers

The Nativity of Virgin Mary Cathedral is located between the Tsaritsa's Chambers of the left and the Palace of Tsar Alexis on the right. The Tsaritsa's Chambers were built in the mid-17th century for the wife of Tsar Alexey - Tsaritsa Maria Ilinichna Miloskavskaya. The design of the building is influenced by the ancient Russian architectural style. Is prettier than the Tsar's chambers opposite, being red in colour with elaborately decorated window frames and entrance.

At present the Tsaritsa's Chambers houses the Zvenigorod Historical, Architectural and Art Museum. Among its displays is an accurate recreation of the interior of a noble lady's chambers including furniture, decorations and a decorated tiled oven, and an exhibition on the history of Zvenigorod and the monastery.

Palace of Tsar Alexis

The Palace of Tsar Alexis was built in the 1650s and is now one of the best surviving examples of non-religious architecture of that era. It was built especially for Tsar Alexis who often visited the monastery on religious pilgrimages. Its most striking feature is its pretty row of nine chimney spouts which resemble towers.

Plan your next trip to Russia

Ready-to-book tours.

Your holiday in Russia starts here. Choose and book your tour to Russia.

REQUEST A CUSTOMISED TRIP

Looking for something unique? Create the trip of your dreams with the help of our experts.