general practitioner research paper

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Front Med (Lausanne)

The Role of General Practice in Complex Health Care Systems

Katharina schmalstieg-bahr.

1 Department of General Practice and Primary Care, University Medical Center Eppendorf, Hamburg, Germany

Uwe Wolfgang Popert

2 Department of General Practice, University Medical Center Göttingen, Göttingen, Germany

Martin Scherer

Associated data.

According to the WHO, in a complex system, “there are so many interacting parts that it is difficult (…), to predict the behavior of the system based on knowledge of its component parts. “In countries without general practitioner (GP)-gatekeeping, the number of possible interactions and therefore the complexity increases. Patients may consult any doctor without contacting their GP. Family medicine core values, e.g., comprehensive care, and core tasks, e.g., care coordination, might be harder to implement and maintain. How are GPs perceived and how do they perceive themselves if no GP-gatekeeping exists? Does the absence of any GP-gatekeeping influence family medicine core values? A PubMed and Cochrane search was performed. The results are summarized in form of a narrative review. Four perspectives regarding the GP's role were identified. The GPs' self-perception regarding family medicine core values and tasks is independent of their function as gatekeepers, but they appreciate this role. Patient satisfaction is also independent of the health care system. Depending on the acquisition of income, specialists have different opinions of GP-gatekeeping. Policymakers want GPs to play a central role within the health care system, but do not commit to full gatekeeping. The GPs and policymakers emphasize the importance of family medicine specialty training. Further international studies are needed to determine if family medicine core values and tasks can be better accomplished by GP-gatekeeping. Specialty training should be mandatory in all countries to enable GPs to fulfill these values and tasks and to act as coordinators and/or gatekeepers.

Introduction

A complex system can be defined in various ways ( 1 – 4 ). According to the WHO, it is a system in which “there are so many interacting parts that it is difficult, if not impossible, to predict the behavior of the system based on knowledge of its component parts.” Delivering health care in general meets this definition due to, for example, the huge number of relationships between patients, caregivers, health care providers, support staff, family, and community members, the diversity of tasks as well as the diversity of care pathways, and organizations involved ( 5 ). In countries in which general practitioners (GP)/family physicians do not function as gatekeepers to the health care system, e.g., in Austria, the Czech Republic, and Greece ( 6 ), the number of possible interactions and therefore the complexity increases. Patients are not required to have a GP and/or may consult any doctor of any specialty without contacting their GP first. This could be resource-intensive, as it may lead to unnecessary patient-doctor encounters, or potentially be harmful if diagnostic tests are doubled, not ordered at all, or drug interactions occur due to a lack of coordination.

Family medicine principles have been described and redefined over the years ( 7 – 9 ). In 2019, van der Horst and Wit identified four core values through a complex discussion and voting process involving more than 1,000 GPs: continuity, medical generalism, person-centeredness, and collaboration (with patients, colleagues, and other health care professionals). Furthermore, they specified medical generalist care, out-of-hour services, palliative- and preventive care as well as coordination of care as core tasks ( 7 , 10 ). Especially, values, such as continuity and medical generalism/comprehensiveness or tasks, such as coordination of care might be harder to implement and maintain in complex medical systems without any gatekeeping.

This article focuses on the challenges for GPs in complex health care systems. How do GPs perceive their own role if no GP-gatekeeping exists? How do others perceive the role of the GP? Does the absence of any GP-gatekeeping influence family medicine core values?

List of Countries

Based the report of the Organisation for Economic Co-operation and Development (OECD) and European Union (EU) ( 6 ), the following countries were identified as countries without GP-gatekeeping ( Table 1 ).

Countries without GP-gatekeeping.

Literature Review

A narrative, rather than a systematic, review was chosen to give an overview, and cover a wide range of issues within the topic at the same time. Although narrative reviews often do not reveal explicit information about the literature search and why studies were found to be relevant ( 11 ), the authors of this article decided to include this information to increase transparency.

A PubMed and Cochrane search was performed. In PubMed, Medical Subject Headings (MeSH) were used to identify relevant literature regarding the role of GPs in health care systems without gatekeeping. Since the MeSH-terms care continuity and patient-centered care were indexed under comprehensive health care, the third and fourth search showed fewer results. The second to fourth search did not yield any additional relevant articles that had not been included before. In Cochrane, an advanced search was performed. The filter was set to “all text” and the same headings as in PubMed were used ( Table 2 ). All full-text articles in English and German that met the topic were included. No filter regarding the publication date was applied.

Search strategy.

Fifteen relevant articles from 2000 to 2017 were found by the PubMed search. The list is shown under Supplementary Table 1 . Further relevant literature was identified while reviewing the articles ( 12 – 24 ).

The GP's Perspective

The GP's self-perceived responsibilities and aspirations regarding family medicine core values and core tasks do not seem to depend on the health care system he or she practices in. Sturm described continuity of care while using the knowledge about the patient's social situation as essential and called for ongoing responsibility during and after the consultation and while receiving secondary care, although German GPs do not generally function as gatekeepers ( 25 ). In Israel, patients may consult any GP or specialist within their health care plan, although the country has tried to move toward a gatekeeping system ( 26 , 27 ). Nonetheless, the study of Tabenkin et al. showed that almost all participating GPs considered “coordination of all patient care” as very important, a third-rated “24 h responsibility for patients” as important ( 15 ). But the traditional role of the GP as the first contact person within the health care system may shift from the individual to the practice, where the GP leads a team that collectively takes responsibility and provides a patient-centered medical home ( 28 ).

Several studies demonstrated that GPs prefer the gatekeeper role ( 24 , 29 – 31 ) due to multiple reasons, e.g., GPs in Iceland thought that mandatory referral increased the flow of information and enhanced the communication with a specialist ( 32 ). Fewer hospital admissions, better quality of care, and lower health care cost were also mentioned ( 24 ). Rosemann et al. showed that not only the patient had a better experience with a referral, but also the GP if he or she was the initiator of the referral ( 30 ). But there are also critical voices among primary-care scientists ( 12 ). Greenfield et al. called for a revision of gatekeeping regulations in the United Kingdom, where GP-gatekeepers are established, to grant patients more choices and by that “facilitate more collaborative work” with other specialties ( 23 ). A Lithuanian study suggested a flexible gatekeeping model regarding adolescents' reproductive health care, as GPs questioned the appropriateness of gatekeeping in this field due to a lack of willingness to provide these services, insufficient training, and inadequately equipped surgeries ( 33 ). GPs considered formal specialty training as essential ( 15 ).

The GP's choice of specialists may depend on the health care system, as they are sometimes required to refer patients within a network ( 34 ).

The Patient's Perspective

A large study that included 17,391 patients in 10 different countries (with and without gatekeeping) evaluated the patient's view on general practice and found no large discrepancies in regards to aspects of care. Minor differences were the relatively positive evaluations given to preventive services in the United Kingdom and the GP's availability (either by an appointment or by phone) in Switzerland, Germany, and Belgium. The relatively negative evaluations were given for service in case of emergencies in the United Kingdom and Slovenia. Slovenian patients also gave relatively negative evaluations regarding the GP's interest in their personal situation. A tendency toward a more positive overall assessment was seen in Switzerland, Germany, and Belgium (countries with no GP-gatekeeping). For the United Kingdom and the Scandinavian countries, a trend toward a less positive assessment was found ( 22 ).

In Germany, multiple attempts have been made to shift to a more family medicine-centered care model. Although it has only been established in voluntary projects, Himmel et al. showed that the majority of over 400 participants from the general population would accept their GP as gatekeeper and appreciated the coordination of secondary care by the GP. Nearly two-thirds wanted to consult their GP during hospitalization. Participants who had a GP at the time of the survey were more likely to accept him or her as gatekeeper compared with participants without a GP ( 35 ). The results are in line with a study from the United States ( 17 ). Another study from Israel reported numbers that were less clear, but trending toward the same direction: a third of all respondents preferred self-referral to a specialist, 40% preferred their GP to act as a gatekeeper, and 19% preferred the GP to coordinate care but to refer themselves to a specialist ( 13 , 15 ). A few years before, 52% of the respondents were in favor of direct access to specialists, but the rate was lower in patients who were older than 45 years and patients whose primary-care physician was a specialist in family medicine ( 16 ). The denial of a referral resulted in lower satisfaction rates ( 20 ). On the contrary, a patient's experience was more positive if the initiative for a referral came from the GP. The authors concluded that this supports the GP's role as gatekeeper, since, in Germany, the patients could have directly scheduled an appointment with a specialist ( 30 ). However, satisfaction rates were not compared to patients who had opted to do so.

Although gatekeeping is often focused on when accessing the patient's perspective on the GP role, van den Brink-Muinen et al. demonstrated that the doctor-patient communication was hardly influenced by it. They compared data from countries with and without gatekeeping and only found that paraphrases, checks for understanding, and requests for clarification and opinion were found more often in consultations of the gatekeeping countries ( 36 ). Two newer reviews conclude that evidence regarding the effect of gatekeeping on quality of care and patient or provider satisfaction is inconsistent and limited ( 18 , 19 ). A large international study using survey data from over 25,000 patients in 17 countries showed that patients were highly satisfied with their GPs, independent of health care system characteristics such as GP density, fee for service reimbursement, gatekeeping, or the GP's role as first contact ( 37 ).

The Specialist's Perspective

Among a U.S. group of nearly 1,500 specialists (cardiology, endocrinology, gastroenterology, general surgery, neurology, ophthalmology, and orthopedics), the attitudes toward primary-care gatekeepers were mixed. Compared with non-salaried physicians, salaried physicians were more in favor of gatekeepers, as did physicians with a greater percentage of practice income derived from capitation ( 21 ). A study from The Netherlands showed that specialists were particularly interested in collaborating with GPs due to their function as gatekeepers. However, an informal network with incidental contacts fulfilled the collaborative needs of the specialists. They did not regard GPs as equal and felt that GPs could learn a lot from them, but that there was nothing to learn vice versa ( 38 ). Specialists were satisfied with the appropriateness and timing of the referral but would have appreciated more information about the patient's medical history or medications ( 30 ). If multiple specialties or even professions address the same medical problem, it is likely that more than one will claim the gatekeeper role. A study among ophthalmologists, GPs, orthoptists, optometrists, and opticians regarding common eye problems was at least able to show a trend toward a medical (ophthalmologist, GP) rather than a non-medical gatekeeper (optometrist) ( 31 ).

The Policymaker's Perspective

Compared with the other three groups, the policy maker's perspective regarding the GP's role in complex medical systems is less well-researched. Mariñoso and Jelovac used a statistical model to identify optimal contracts that would induce the best behavior from a public insurer's point of view and found that gatekeeping was superior wherever GP's incentives matter ( 39 ). Philips et al. performed a multivariate logistic regression analysis using secondary data showing that gatekeeper requirements are associated with higher utilization of widely recommended cancer screening interventions (e.g., mammography). No association was found regarding the use of less uniformly recommended interventions [e.g. prostate-specific antigen (PSA) checks]. The authors concluded that “policymakers should consider the potential benefits of gatekeeper requirements with respect to preventive care when designing health plans and legislation” ( 40 ).

Only two studies from Israel reported data directly displaying the policymaker's perspective ( 14 , 15 ). The members of the Ministry of Health, the Sick Funds' central administrations, and the Israel Medical Association (IMA) central office were interviewed and stated that the highly trained GP should play a central role in the health care system. GPs should be highly accessible coordinators, able to weigh cost considerations. However, only about half of the participants supported a GP-gatekeeper model. The perceived barriers to implement such a model included loss of faith in GPs by the general population, dearth of GPs with adequate training, low stature, lack of 24 h-availability, resistance from specialists, and competition between the sick funds.

Discussions

Most publications illustrate the GP's and patient's point of view. Lesser is known about the view of secondary-care providers, especially addressing other aspects of the GP's role, besides his or her function as a gatekeeper. Studies depicting the policymaker's perspective are lacking. Few studies directly compare countries with and without GP-gatekeeping. Often studies focus on the question whether GP-gatekeeping should be implemented, is beneficial, or leads to satisfaction of patients, GPs, and specialists. Other aspects of the GP's role in complex health care systems are lesser well-researched.

The GP's self-perceived role regarding family medicine core values and core tasks is independent of the health care system. The GPs strive to accomplish these whether they function as gatekeepers or not, but there is evidence that they appreciate or would appreciate this role. Patient satisfaction and doctor-patient communication also seem to be independent of the health care system. Depending on the acquisition of income (salaried vs. non-salaried physicians, capitation vs. no capitation), specialists have different opinions on whether GPs should function as gatekeepers, but do not regard them as equals. Policymakers want GPs to play a central role within the health care system, but do not like to commit to full GP-gatekeeping. The GPs and policymakers emphasize the importance of family medicine specialty training.

Besides the health care system, cultural, religious, economical, and even geographical aspects might influence the role of the GP and implementation of family medicine core values.

Limitations

As this is a narrative review, the typical limitations apply: compared to a systematic review, the literature search was more subjective and less structured. The aim was to give an overview of the GP's role in a complex medical system without gatekeeping. It is possible that a more systematic research and the use of other/further MeSH terms would have identified more articles, although relevant literature was added while reviewing the initial list. The aim of the review is to describe the GP's role in health care systems that are similar to each other (no GP-gatekeeping) without focusing on a national perspective. All authors have worked and lived in other countries. However, it is possible that the article was influenced by their experience in Germany. Despite the fact that the debate of the GP's role is ongoing, most articles are over 10 years old.

Conclusions

Further international studies are needed to compare the role of GPs in countries with and without gatekeeping. These studies should include multiple perspectives (GP, patient, specialist, and policymaker) and go beyond the question of whether GP-gatekeeping should be established or not. There are studies indicating that specialty training needs to be mandatory in all countries to enable GPs to fulfill family medicine core values and tasks and to act as coordinators and/or gatekeepers. However, more studies are needed to prove this.

Author Contributions

KS-B conducted the literature review. All authors were responsible for drafting the manuscript and for the critical revision of the content.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fmed.2021.680695/full#supplementary-material

• Research article
• Open access
• Published: 29 January 2016

Knowledge structure and theme trends analysis on general practitioner research: A Co-word perspective

• Yang Hong 1 ,
• Qiang Yao 2 ,
• Ying Yang 1 ,
• Jun-jian Feng 1 ,
• Shu-de Wu 1 ,
• Wen-xue Ji 1 ,
• Lan Yao 1 &
• Zhi-yong Liu 1  

BMC Family Practice volume  17 , Article number:  10 ( 2016 ) Cite this article

3876 Accesses

32 Citations

1 Altmetric

Metrics details

General practitioners (GPs) are the most important providers of primary health care, as proven by related research published several decades ago. However, the knowledge structure and theme trends of such research remain unclear. Accordingly, this study aimed to provide an overview of the development of research on GPs over the period of 1999 to 2014.

Studies on GPs conducted from 1999 to 2014 were retrieved from PubMed. In this work, co-word, social network analysis, and theme trends analyses were conducted to reveal the knowledge structures and thematic evolution of research on GPs.

The number of conducted studies on GPs increased. However, growth speed slowed down during the past 16 years. A total of 27 high-frequency keywords were identified in 1999 to 2003, and more new and specific high-frequency keywords emerged in the subsequent periods. The dynamic of this field was first divergent and then considered convergent. Specifically, network centralization is 19.77 %, 19.09 %, and 13.04 % in 1999 to 2003, 2004 to 2008 and 2009 to 2014, respectively. The major topics of research on GPs completed from 1999 to 2014 were “physician/family,”“attitude of health personnel,” and “primary health care,” and “general practitioner” communities, and so on.

The research themes on GPs are relatively stable at the beginning of the 21 st century. However, the thematic evolution and research topics of research on GPs are changing dynamically in recent years. Themes related to the roles and competencies of GPs, and the relations between general practitioner and patients/others have become research foci on GPs. In addition, more substantial research especially on comprehensive approaches and holistic modeling, which have been defined in the European Definition of General Practice/Family Medicine, are expected to be accomplished.

Peer Review reports

High-quality primary care is the foundation of effective and efficient health care systems. The essential elements of the practice of primary care include accessibility as the first-contact point of entry to the health care system, continuity, comprehensiveness, coordination of referrals, and understanding of the family and community context of health [ 1 – 3 ]. General practice is a key discipline of primary care, and in many countries, general practitioners (GPs) are physicians who are directly accessible to the public. Thus, strengthening the knowledge structure and analyzing theme trends in GPs will contribute to the provision of better health care for all [ 4 , 5 ].

Historically, the role of a GP was once performed by any doctor qualified in a medical school working in the community [ 6 ]. However, since the 1950s, general practice has become a specialty in its own right, with specific training requirements tailored to each country. The Alma Ata Declaration in 1978 set the intellectual foundation of what primary care and general practice should be [ 7 – 9 ]. Currently, GPs are specialist physicians trained in the principles of the discipline [ 10 ]. They are personal doctors who are responsible primarily for the provision of comprehensive and continuing care to every individual seeking medical care irrespective of age, sex, and illness. GPs care for individuals in the context of their family, their community and their culture, always respecting the autonomy of their patients [ 11 – 13 ]. The core competencies of GPs are primary care management, person-centered care, specific problem solving skills comprehensive approach, community orientation, and holistic modeling. However, the role of a GP can vary greatly between (or even within) countries [ 14 ]. In urban areas of developed countries, their roles tend to be narrower and focused on the care of chronic health problems, treatment of acute non-life-threatening diseases, early detection and referral to specialized care of patients with serious diseases, and preventative care including health education and immunization. However, in developing countries or in the rural areas of developed countries, a GP may be routinely involved in pre-hospital emergency care, the delivery of babies, community hospital care, and performance of low-complexity surgical procedures. Moreover, in some healthcare systems, GPs work in primary care centers where they play a central role in the healthcare team, whereas GPs can work as single-handed practitioners in other models of care [ 15 , 16 ].

Entering the 21 st century, the connotation and role of GPs was also developed with changes of society and health reform all over the world. In particular, GPs not only provide comprehensive and compassionate health care services in the context of individual needs, their families and communities, but also play a vital role in reducing health inequalities and in delivering high-quality and cost-effective care. The role of GPs as primary care physicians in health risk factor interventions has been well introduced in the literature [ 17 – 20 ]. Many researchers have suggested that GPs can contribute to reducing the prevalence of smoking or alcohol misuse. Moreover, GPs encourage lifestyle changes, especially in nutrition and physical activities. Patients primarily obtain worthy information on nutrition or physical activity from GPs [ 21 ]. Hence, to understand the knowledge structure and theme trends on general practitioners further, we use co-word analysis and related technologies in this paper to reveal the research evolution and trends of major themes and knowledge structure on GPs, probe features of the major themes and its development process, and provide an overview of the development trends in the field of GPs during 1999–2014 based on the PubMed database [ 22 – 25 ].

Data source

Data were retrieved and downloaded from PubMed, a biomedical literature database developed by the US National Center for Biotechnology Information. PubMed was selected as the data source for two reasons. First, PubMed is a free authoritative medical literature database consisting of over 25 million citations for biomedical literature from MEDLINE, life science journals, and online books, including the fields of biomedicine and health, covering portions of the life sciences, behavioral sciences, chemical sciences, and bioengineering. Second, the articles from PubMed are indexed with Medical Subject Headings (MeSH) terms, which comprises a set of normalized words that can reflect the contents of articles [ 26 , 27 ]. The PubMed database and MeSH terms provide a good possibility of extracting emerging keywords. The MeSH terms “family, physician” and “general practitioner” are two different terms in the database. The meanings of the two subject terms are also different and reflect the development of GPs. In this study, retrieval strategies employed included “general practitioners” [MeSH] or “physicians, family” [MeSH terms] according to the entry words, which include “general practitioner,” “practitioner, general,” “practitioners, general,” “physicians, general practice,” “general practice physician,” “general practice physicians,” “physician, general practice,” and “practice physicians, general.” The publication scope was limited to within 1999–2014 and a total of 10704 articles were retrieved on August 7, 2015.

Scientometrics

Scientometrics is a discipline of measuring science and the effects of scientific work, and its indicators are equally suitable for macro-analysis and micro studies [ 30 ]. Scientometrics has been widely used in the fields of data mining, machine learning, and information retrieval [ 31 , 32 ].

Co-word analysis is a content analysis technique that is based on the assumptions that a scientific field can mark literature and reflect its core contents by abstracting a set of single words, and that the keywords of scientific publications can be treated as signal-words [ 33 ]. This technique means that the more frequent the co-occurrence of a pair of words in the literature, the more similar the themes they indicate. Moreover, the frequency of word occurrence in the entire body of a selected field can reflect important themes, and co-occurrence of multiple terms in the same literature that reflects the themes to which they refer [ 31 ].

Social network analysis is a method that aims to study the relationship between a set of actors and views social relationships in terms of network theory that consists of nodes (representing words in this study) and ties (represent word relationships in this study) [ 22 , 34 , 35 ]. Centrality is an important index for analyzing the network and determining the influence of a node in the network, including degree centrality, betweenness centrality, and closeness centrality [ 36 ]. Degree represents the number of ties to others, while in a friendship network, degree may translate to gregariousness or popularity. Betweenness indicates how frequently a node lies along the geodesic pathways of other nodes in the network and, therefore is an inherently asymmetric measure. Closeness represents the graph-theoretic distance of a given node to all other nodes, and network centralization reflects the whole network tightness.

Thematic evolution analysis [ 37 , 38 ] was used to detect and visualize the topic evolution of GPs as it can be used to conduct sufficient analyses than co-word networks [ 39 ]. The alluvial diagram based from the geographic domain and proposed by Rosvall and Berstrom [ 40 ] was also employed in this study to visualize the evolution of networks. The overall evolution provides insights on the evolution of different topics.

Science mapping analysis is an important research topic in the field of scientometrics, and is focused on monitoring a scientific field and delimiting research areas to determine its cognitive structure and its evolution, thereby revealing the hidden key relations among documents, authors, institutions, and topics. The workflow of science mapping analysis contains eight aspects: data retrieval, preprocessing, network extraction, normalization, mapping, analysis, visualization, and interpretation [ 41 – 43 ].

Data analysis

The process includes mainly three stages: data processing, theme structure, and theme evolution. First, the retrieved articles were downloaded as XML files and imported into the Bibliographic Item Co-Occurrence Matrix Builder (BICOMB) [ 28 ] software. Next, the major MeSH terms were extracted and assessed with the BICOMB. The high-frequency MeSH terms were identified and divided into three time periods, and a word occurrence matrix was constructed to support further co-word analysis. The high-frequency words were defined by using the following formula proposed by Donohue in 1973: \( N=1/{2}^{\ast}\left(-1+\sqrt{1+8*{I}_1}\right) \) [ 29 ]. In this formula, I 1 represents the number of words that occurred once in the articles, and words with higher frequency than N were considered high-frequency words. Second, the word occurrence matrix was imported into Ucinet software and visualized based on co-word theory, after which the social network analysis method was used to analyze the themes and knowledge structure of GPs in different time periods. Third, the thematic evolution analysis method and NEViewer software were used to analyze and forecast the theme evolution trends by combining the different stages information. Based on the above analysis, BICOMB [ 28 ], Ucinet [ 44 ], and NEViewer [ 39 ] software were used to analyze the publications for knowledge mapping.

Growth rate

The entire data were divided into three periods, namely, 1999 to 2003, 2004 to 2008 and 2009 to 2014, to display the research hotspots and developments of research on GPs. The growth rate of the number of publications was determined through the absolute increase of publications and then measured by using two related parameters: relative growth rate (RGR) and double time (Dt) [ 45 , 46 ].

RGR in the classical growth analysis is defined as

where N 2 and N 1 are the cumulative publications in two years, namely, t 2 and t 1, respectively. In the present analysis t 2– t 1 is taken as one year. Accordingly, RGR can be expressed as RGR = ln ( N 2/ N 1).

Dt is the time required for publications to double in number for a given RGR, and is expressed as

where Dt is a characteristic time for this exponential growth, and a constant value for RGR in each subsequent year indicates that the growth rate is exponential. For example, if the number of articles or pages of a subject doubles in one year then the difference between the logarithms of numbers at the beginning and end of this period must be the logarithm of the number 2. Hence, if the natural logarithm is used, the RGR = 0.693(ln2 ≈ 0.693) and Dt =1.

The entire set of data was divided into three periods, namely, 1999 to 2003, 2004 to 2008 and 2009 to 2014, to display the hotspots and developments of research on GPs clearly. Table  1 indicates that the high-frequency major MeSH terms, with less than 3 % proportion, have a total frequency accounting for approximately 50 % of the total frequency of major MeSH terms. All these high-frequency MeSH terms provide the hot topics analyzed in substantial research on GPs.

Figure  1 reveals the publication trend of research on GPs conducted all over the world from 1999–2014. The figure shows that considerable research on GPs has been well developed since 1999, with 392 records. The number of papers annually increased from 1999–2003 with a period of slow growth. Subsequently, the number of papers continued to increase rapidly and attained a peak in 2008, although a slight decline occurred in 2004. However, a fluctuating decline trend was observed after 2008 (for details, see Table  2 ).

Papers on general practitioners and related disciplines published from 1999–2014 in WoS

As shown in Table  2 , for the last 16 years, the worldwide RGR achieved an average value of 0.22 and an average D t of 4.76. Moreover, RGR dropped from 0.72 in 2000 to 0.16 in 2006, but slightly increased to 0.17 in 2007 and 2008. The RGR continuously dropped until it attained a value of 0.07 in 2014. The D t reflected a similar trend, in which it increased from 0.96 in 2000 to 10.66 in 2014. These findings suggest that the growth speed of GP-related publications has slowed down in the 16 years. This specific result conforms to the outcomes presented in Fig.  1 .

As shown in Fig.  1 , although research on GPs has increased during last decades, compared with other fields (e.g., cardiovascular diseases, digestive system diseases, neoplasms and respiratory tract diseases), the gaps between GPs and others has become much larger. In 1999, GP-related papers accounted for 0.08 % of all papers in PubMed and reached a peak in 2008 (0.12 %). Then, the growth speed of GP-related research became increasingly lower compared with other fields and the entire field of medicine. The results conform to the outcomes presented in Table  2 .

Knowledge structure and research topic analysis

The threshold for generating edges was set as five times of co-occurrences, in order to eliminate the weak relation among the major MeSH terms. Keyword centrality was measured by the degree, betweenness, and closeness centrality. Network centralization was applied to analyze the network structure. Specific data are shown in Table  3 . As shown in Table  3 , the network centralization decreased with the development of GPs, while the mean values of degree and betweenness increased rapidly. The results suggest that the topics in this field are becoming increasingly richer and no longer based on only several words or themes; moreover, he links with topics have become even closer.

Knowledge structure of the time period from 1999–2003

A sum of 27 high-frequency keywords were identified from the research on GPs published from 1999–2003 (Table  4 ). “Physicians, family” with a frequency of 1759 ranks first. This keyword has a degree value of as high as 2160, indicating that it has a direct link with many keywords and is in the central position in the social network of GP-related research. Moreover, keywords “attitude of health personnel,” “family practice,” “physician’s practice patterns,” and “primary health care” contain degree values that rank in the top 5. The betweenness value in Table  4 also shows that these four keywords mentioned above with the same value (11.15), rank top in the list. Meanwhile, these four keywords have the lowest closeness value (26) in the list, indicating that they have the shortest path when they communicate with other keywords. Such a finding indicates that these keywords can transfer information with less dependence on other keywords. Figure  2 demonstrates that the research themes in this period focus mainly on “family practice,” “primary health care,” “attitude of health personnel,” and “physician’s practice patterns.”

Map for keywords in GP research, 1999–2003.The size of nodes indicates the keywords centrality, and the thickness of the lines indicates the co-occurrence frequency of keywords pairs

Knowledge structure of the time period from 2004–2008

A total of 41 high-frequency keywords were identified from research on GPs published from 2004–2008. “physicians, family” still tops the list with a frequency of 2424, followed by “attitude of health personnel,” “family practice,” “primary health care,” and “physician’s patient patterns.” Compared with other keywords illustrated in Table  5 , these keywords have higher degree and betweenness values and lower closeness value. Accordingly, these five keywords above are in the central position of the social network in GPs research, and many paths that diversified from them connect to other keywords. Figure  3 also demonstrates that most research on GPs focus primarily on “physicians, family,” “family practice,” “attitude of health personnel,” and “primary health care.” However, “physician’s practice patterns” and “physician-patient relations” have received more attention based on the increasing frequency of new keywords linked to them.

Map for keywords in GP research, 2004–2008. The size of nodes indicates the keywords centrality, and the thickness of the lines indicates the co-occurrence frequency of keywords pairs

Knowledge structure of the time period from 2009–2014

A total of 64 high-frequency keywords were extracted from the papers published from 2009 to 2014. “General practitioner” replaces “family practice” in the rank of top four. Both “physicians, family” and “general practitioners” are placed exclusively at the central position in the social network of GP-related research; their degree values are similar, but much higher than those of the remaining keywords shown in Table  6 . Moreover, “attitude of health personnel” and “primary health care” also play indispensable roles in the social network in accordance with their relatively high degrees and betweenness values. Because of their lowest closeness centrality values, “general practitioners,” and “physicians, family,” have the shortest commutation path with other keywords, which means that if any of these keywords is curtailed, the knowledge structure shown in Fig.  4 would show that GP-related research has undergone significant changes in recent years compared with former stages.

Map for keywords in GP research, 2009–2014. The size of nodes indicates the keywords centrality, and the thickness of the lines indicates the co-occurrence frequency of keywords pairs

Thematic evolution and trends analysis

Figure  5 shows an overall picture of the topic evolution of research on GPs from 1999–2014, in which different bars of color represent different communities of GPs (themes or topics).

The evolution alluvial diagram of GP research

“Physicians, family” always topped the list of the diagram in all time periods (1999–2003, 2004–2008, and 2009–2014). The keywords “attitude of health personnel,” “family practice,” “general practitioner,” and “primary health care” also topped the diagram, indicating that they are the major topics of research on GPs. The keyword “attitude of health personnel” ranked fourth in 2009–2014 and involved a branch from “health knowledge, attitude, practice” in 2004–2008, which is also composed of partially “physician–patient patterns,” “physician’s role,” and “health knowledge, attitude, practice” communities in 1999–2003. Another branch of studies featuring “health knowledge, attitudes, practice” in 2004–2008 evolved a new community, that is, “mass screening” in 2009–2014. “General practitioner,” an emerging community back in 2009–2014, replaced the “attitude of health personnel” and became the second hot topic in the last period. This particular keyword relates to the evolution of keywords and changes depending on different concerns of considerable research on GPs. “Family practice” had a high spot in 1999–2003 and in 2004–2008. “Rural health service,” which emerged in the first period, merged into “family practice” community in 2004–2008, but finally disappeared and became a “death topic” in 2009–2014. A small branch from “primary health care” formed a new community, that is, “depression” in 2009–2014. Finally, the “physician–patient relations” community is divided into “physician–patient relations” and “patient satisfaction” in 2009–2014. The “referral and consultation” community in 1999–2003 and 2004–2008 introduced a new community, “inter-professional relations,” in 2009–2014. “Practice guidelines” as a topic community in 1999 to 2003 and 2004 to 2008 finally merged into the “physician’s practice patterns” community in 2009–2014.

Discussions

Publication trends of research on gps.

The publication trend of research on GPs is revealed to maintain a fluctuating increase with a slight decline occurring in 2004 before reaching its peak in 2008. The rapid increase of papers in 2004–2008 and the peak in 2008 might be related to the existence of health care reforms in most countries, which focused considerably on general practice and primary health care [ 47 ]. Although a decline trend of papers publication on GPs after 2008 was observed, in general, the total quantity of papers on GPs were still massive. Hence, higher numbers of better papers are expected to be produced in the impending maturity period. Moreover, compared with the fields of cardiovascular diseases, digestive system diseases, neoplasms and respiratory tract diseases, the gaps of absolute number and growth rate are increasing. General practice is a key discipline of primary care, and in many countries, GPs are physicians directly accessible to the public. The increase of research output pertaining to general practice can also promote the development of primary care. Although related research in this discipline shows a fluctuating increase, classical clinical disciplines, such as cardiovascular diseases, digestive system diseases, neoplasms and respiratory tract diseases have consistently been the focus of research. If the field of general practice intends to catch up, it will take many years of work and accumulated experience before this can happen [ 25 ].

The knowledge structures of research on GPs

During the first stage, the knowledge structure could not be shaped if “physician, family” is removed (Fig.  2 ). Therefore, “physician, family” maintains its central position in the social network. The keywords “attitude of health personnel,” “family practice,” “physician’s practice patterns,” and “primary health care” also play vital roles in the social network of GP-related research. Consequently, these keywords have an indispensable place when information is transferred from one keyword to another, and can control information exchange among other keywords. “Physicians, family” is evidently the dominant keyword, but the other keywords related to it, such as “family practice,” “attitude of health personnel,” and “primary health care” are also at the center of the knowledge structure. The knowledge structure exists based on these main keywords. In general, research themes over this period aim mainly to sort and clarify the role/career orientation of GPs in primary care/family practice. The results demonstrate that research themes in this period focus mainly on the categories enumerated below. First, the basic role of GPs is within the scope of family practice/primary health care, such as conducting referrals, consultations, and mass screenings. The equity in the provision in rural health services is also considered a basic role of GPs. Second, the relationship between general practitioners/family physician and patients is a topic widely studied based on the keywords “physician-patient relations,” “communication,” and “patient education”. Third, with the development and changes in primary health and primary care teams, GPs now have opportunities to extend the range of their own skills and interests in clinical practice. Therefore, the development of GPs in terms of their clinical skills and career has received remarkable attention. Keywords “specialization,” “clinical competence,” and “education, medical, continuing” support such observation.

During the second stage, “physicians, family” still plays the central role, and “attitude of health personnel,” “family practice,” “primary health care,” and “physician’s patient patterns” are also in the top 5. This observation is similar with the previous finding in the first period; however, the knowledge structure shown in Fig.  3 is obviously more complex and scattered than the former period. In this period, the basic role of GPs in primary health care/family practice remains the main focus of research on GPs. However, such works have begun to involve the quality and accessibility of primary health services with new emerging keywords (i.e., “health services accessibility,” “delivery of health care,” and “quality of health care”) shown in Table  5 . The role of GPs in managing diseases, especially in managing chronic diseases (e.g., asthma, hypertension, diabetes mellitus type 2, cardiovascular diseases, mental disorders, and depression) is also fast becoming an emergent research topic in this field. This may be because during this period, most Western countries (e.g., Australia, the US, and New Zealand) spent significant amount of time refocusing their health care systems to address the increasing burden of chronic diseases [ 48 , 49 ]. The CP–patient relation and clinical development of GPs (i.e., continuing education and clinical competence improvement) are also considered main research themes from 2004–2008. Moreover, literature on GPs has begun to emphasize the spiritual/psychological experience of both GPs and patients according to the keywords “patient” and “job satisfaction.” Based on the above analysis, substantial amount of research on GPs published in this period are more in-depth and diversified than works published in the last stage. Moreover, during these years, GPs have begun to provide patients with higher-quality, more equitable, and comprehensive health services. A new keyword, “medical records systems, computerized” also emerged during this period, indicating that the use of computers and computerized medical records is becoming more popular in primary health care services. Hence, the topics related to “medical records systems, computerized” and GPs are also widely investigated.

Unlike in the last two periods, during the third stage, “general practitioner” replaces “family practice” in the rank of top four, and both “physicians, family” and “general practitioners” are placed at the central position. Both Table  6 and Fig.  4 indicate that research themes can be clustered into several respects. The first one pertains to the new role changes of GPs in primary/family/general practices that correspond with health care reforms throughout the world. The new keyword “general practice,” which refers mainly to community-orientation, has received wide attention because of the increasing consideration paid to primary health care. Hence, GPs have gradually provided person-centered, continuing, comprehensive, and coordinated whole person health care services to individuals and families in their respective communities [ 47 ]. The second aspect refers to the comprehensive functions of GPs. Other than focusing on managing chronic diseases, in this period, the GPs are more concerned with health promotion and disease follow-up care to meet the challenges that confront the reformed health care system, including the existence of an ageing population accompanied by an increased prevalence of long-term health conditions. The third aspect relates to care coordination with other types of health and social care providers. The whole world is confronted with problems regarding chronic and complex disease management. Thus, closer inter-professional cooperation is urgently needed [ 50 ]. In this context, GPs and other relevant persons involved must work together as part of a healthcare team that provides the best health care practice [ 51 , 52 ].

The thematic evolution of research on GPs

Overview, the majority of the themes maintain stable positions or only undergo slight rank changes during the former two stages. The thematic evolution results show that many research topics do not emerge out of the void and are more or less associated with the presence of other topics that emerged in the past. Many themes are based on previous scientific research and are produced gradually, although different forms of thematic evolution have taken place in recent years as shown in Fig.  5 . For example, the “general practitioners” community in 2009–2014 emerged with the development of primary care and the evolution of general practice. Specifically, with the enhancement and improvement of primary health care, general practice has become a place (both real and virtual) of comprehensive health service, in which individual patients are not only provided with episodic care and ongoing clinical management, but also granted access to preventive care, health education, and other services. The focus of primary health care [ 53 ] has been widely analyzed in the literature, which corresponds to an increase in works that study the roles of GPs in general practice. “Inter-professional relations,” a new community that emerged in the 2009–2014 period, developed from the “referral and consultation” community in the last two periods. Such an occurrence may be because of the improvement of primary health care, inter-professional relations with broader connotation and referral and consultation, which gradually formed a new community in 2009–2014. Pieces of evidence indicate that high-quality community-based palliative care is achieved with effective multidisciplinary teamwork, good inter-professional relationships (i.e., good communication between GPs and district nurses), and early referral of patients to district nurses [ 54 ]. In the last two decades of the 20 th century, the prevalence of depression in the general population has constituted a major health burden among developed countries, and an increase in recognizing the importance of ensuring its identification and treatment in primary health care has been observed [ 55 , 56 ]. Therefore, with thematic evolution, “depression” finally became a new community from the “primary health care” expansion in 2009–2014. In summary, the considerable amount of research on GPs has great potential for further development.

Hot topics found in research on GPs

Multiple roles and competency improvement of gps.

Except for the foundational role of GPs in primary health care and with the represented keywords (e.g., “referral and consultation,” “drug prescriptions,” “delivery of health care,” “mass screening,” and “patient education as topic” shown from 1999–2014), GPs have continuously played important roles in primary care management, improving quality and ensuring equity of primary health care services, which has become a major concern as a research topic. Anne et al. conducted research to assess the geographical equity in the availability and accessibility of GP services for women in Australia, and their analysis results indicated that women living in rural areas gave lower ratings for availability, accessibility, and affordability of GP services than women in urban areas [ 57 ]. The new trends of knowledge structure analysis reveal that GPs have gradually provided person-centered, community-orientation health care services with comprehensive approaches. The management of chronic diseases has also become one of the most important tasks of GPs. The importance of GPs in managing patients with chronic diseases, especially asthma, hypertension, Type 2 diabetes, and mental health issues, include the initial diagnosis, initiating treatment, risk factor interventions to overall continuity of care. Many studies have explored the disease management of patients with specific populations, such as women [ 58 ], older adults [ 59 – 62 ], and children [ 63 ]. The role of GPs in promoting health and preventing disease, whether effective or not, must be investigated further.

As for the competency of GPs, continuous improvements involving education level, specific problem-solving competence, and communication skill with patients have become the focus of studies on GPs in response to the increasing needs of quality improvement in primary care in many countries. In England, some GPs have taken leading roles in their practices for specific clinical areas [ 64 ]. GPs are also gaining more specific problem-solving skills. Joanna et al. defined GPs as physicians who supplement their generalist role by delivering high quality and improved accessibility to services. Dermatology and respiratory diseases are areas that GPs with special interest have chosen to develop in recent years [ 65 , 66 ]. With healthcare system reforms as well as the problem-solving and skills development of GPs, more specific themes on the role and competencies of GPs may become available to meet the challenges that confront disease management. Such challenges include new concepts of patient empowerment and continuous quality improvement, which has been revised in the 2011 edition of European Definition of General Practice/Family Medicine compared with the 2002 and 2005 edition [ 67 ].

Conclusions

Scientometrics, co-word analysis, and social network analysis were combined and used to reveal the knowledge structures and thematic evolution of research on GPs published from 1999–2014. The number of studies on GPs has rapidly increased but the growth rate has decreased to some extent. The gaps between GPs and others (e.g., cardiovascular diseases, digestive system diseases, neoplasms, and respiratory tract diseases) have also been growing in recent years.

The research on GPs varies and develops with the changes in health care reforms, health policies, and functions of GPs in many countries, especially in recent years. The multiple roles and competency improvement of GPs, as well as the relations between GPs and patients/others involved (e.g., health care providers) have reflected the core competencies of GPs, especially in primary care management, person-centered care, specific problem solving skills and community-orientation, in accordance with the European Definition of General Practice/Family Medicine [ 67 ]. More substantial research, especially on comprehensive approaches, olistic modeling, patient empowerment, and continuous quality improvement should be accomplished. This study also anticipates that, owing to the growth of the elderly population, elderly persons shall be the main topics of the major specific groups in GP-related research.

Limitations

First, literature was extracted only from the PubMed database, which may not contain all literature related to GP research, especially non-English articles and some grey literature (e.g., reports or internal materials). Second, only high-frequency words were analyzed; hence, the results could only show the hot topics on GP-related research. Some new emerging topics with low attention may not have been shown in the map. Therefore, analyses combining multiple databases and new emerging topics should be conducted in future studies.

Abbreviations

• General practitioners

Bibliographic item co-occurrence matrix builder

relative growth rate

double time

Starfield B, Shi L, Macinko J. Contribution of primary care to health systems and health. Milbank Q. 2005;83:457–502.

Article   PubMed   PubMed Central   Google Scholar  

van Weel C, Rosser WW. Improving health care globally: a critical review of the necessity of family medicine research and recommendations to build research capacity. Ann Fam Med. 2004;2 Suppl 2:S5–16.

Organisation mondiale de la santé. Bureau régional de l'Europe, Atun R. What are the advantages and disadvantages of restructuring a health care system to be more focused on primary care services?[M]. Copenhagen: WHO Regional Office for Europe; 2004.

Rosser WW, Van Weel C. Research in family/general practice is essential for improving health globally. The Annals of Family Medicine. 2004;2:S2–4.

Article   PubMed   Google Scholar  

Mitchell GK. How well do general practitioners deliver palliative care? A systematic review. Palliative Medicine. 2002;16:457–64.

Glanville J, Kendrick T, McNally R, Campbell J, Hobbs FDR. Research output on primary care in Australia, Canada, Germany, the Netherlands, the United Kingdom, and the United States: bibliometric analysis. British Medical Journal. 2011;342:d1028.

Phillips RL, Brungardt S, Lesko SE. The future role of the family physician in the United States: a rigorous exercise in definition (vol 12, pg 250, 2014). Annals of Family Medicine. 2015;13:311–1.

van Weel C. The impact of research in primary care and family medicine: the Thomson Reuters Web of Science Subject Category ‘Primary Health Care’. Family Practice. 2011;28:239–40.

Watson EK, Shickle D, Qureshi N, Emery J, Austoker J. The ‘new genetics’ and primary care: GPs' views on their role and their educational needs. Family Practice. 1999;16:420–5.

Article   CAS   PubMed   Google Scholar  

McColl A, Smith H, White P, Field J. General practitioners’ perceptions of the route to evidence based medicine: a questionnaire survey. British Medical Journal. 1998;316:361–5.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Grumbach K, Selby JV, Damberg C, Bindman AB, Quesenberry C, et al. Resolving the gatekeeper conundrum - What patients value in primary care and referrals to specialists. Jama-Journal of the American Medical Association. 1999;282:261–6.

Article   CAS   Google Scholar  

Bodenheimer T, Lo B, Casalino L. Primary care physicians should be coordinators, not gatekeepers. Jama-Journal of the American Medical Association. 1999;281:2045–9.

Russell MA, Wilson C, Taylor C, Baker CD. Effect of general practitioners’ advice against smoking. Br Med J. 1979;2:231–5.

Patient care and the general practitioner. Joint Working Party of the Welsh Council of the Royal College of General Practitioners and the Welsh General Medical Services Committee. BMJ. 1994;309:1144–7.

Article   Google Scholar  

McCormick A, Fleming D, Charlton J. Morbidity statistics from general practice: fourth national study 1991-1992[M]. HM Stationery Office, 1995.

Gray DP. History of the Royal College of General Practitioners--the first 40 years. Br J Gen Pract. 1992;42:29–35.

CAS   PubMed   PubMed Central   Google Scholar  

van Weel C. The web of science subject category ‘primary health care’. Family Practice. 2011;28:351–1.

Soler JK, Carelli F, Lionis C, Yaman H. The wind of change: after the European definition--orienting undergraduate medical education towards general practice/family medicine. Eur J Gen Pract. 2007;13:248–51.

McLean R, Mendis K, Harris B, Canalese J. Retrospective bibliometric review of rural health research: Australia’s contribution and other trends. Rural Remote Health. 2007;7:767.

PubMed   Google Scholar  

Van Dormael M. Roles of the general practitioner in different contexts. Ann Soc Belg Med Trop. 1995;75 Suppl 1:79–88.

Stange KC, Nutting PA, Miller WL, Jaen CR, Crabtree BF, Flocke SA, et al. Defining and Measuring the Patient-Centered Medical Home. Journal of General Internal Medicine. 2010;25:601–12.

Zhang CC, Yu Q, Fan QH, Duan ZG. Research collaboration in health management research communities. Bmc Medical Informatics and Decision Making. 2013;13.

Zhang J, Xie J, Hou WL, Tu XC, Xu J,Song FJ, et al. Mapping the Knowledge Structure of Research on Patient Adherence: Knowledge Domain Visualization Based Co-Word Analysis and Social Network Analysis. Plos One. 2012;7(4):e34497.

Kurtz JE, Heitz D, Enderlin P, Imbert F, Nehme H, Bergerat JP, et al. Geriatric oncology, general practitioners and specialists: Current opinions and unmet needs. Critical Reviews in Oncology Hematology. 2010;75:47–57.

Mendis K, Solangaarachchi I. PubMed perspective of family medicine research: where does it stand? Family Practice. 2005;22:570–5.

Li F, Li M, Guan P, Ma S, Cui L. Mapping Publication Trends and Identifying Hot Spots of Research on Internet Health Information Seeking Behavior: A Quantitative and Co-Word Biclustering Analysis. Journal of Medical Internet Research. 2015;17.

Bethesda (MD) National Center for Biotechnology Information (US); 2005-. PubMed Help.

Lei CLLWY, Yingna ZHHYH, Hao Z. Development of a text mining system based on the co-occurrence of bibliographic items in literature databases. New Technology of Library and Information Service. 2008;8:015.

Google Scholar  

Qinglan S. The Demarcation of High-and Low- frequency Terms and Ways of Estimation the Frequency of Terms. China Society of the Library Science. 1992;18:79–81.

Pouris A. Nanoscale research in South Africa: A mapping exercise based on scientometrics. Scientometrics. 2007;70:541–53.

Yao Q, Chen K, Yao L, Lyu PH, Yang TA, Luo F, et al. Scientometric trends and knowledge maps of global health systems research. Health Res Policy Syst. 2014;12:26.

VanRaan AFJ. Scientometrics: State-of-the-art. Scientometrics. 1997;38:205–18.

Ding Y, Chowdhury GG, Foo S. Bibliometric cartography of information retrieval research by using co-word analysis. Information Processing & Management. 2001;37:817–42.

Uddin S, Hossain L, Abbasi A, Rasmussen K. Trend and efficiency analysis of co-authorship network. Scientometrics. 2012;90:687–99.

Lee B, Kwon O, Kim HJ. Identification of dependency patterns in research collaboration environments through cluster analysis. Journal of Information Science. 2011;37:67–85.

Piraveenan M, Prokopenko M, Hossain L. Percolation Centrality: Quantifying Graph-Theoretic Impact of Nodes during Percolation in Networks. Plos One. 2013;8.

Schiebel E, Horlesberger M, Roche I, Francois C, Besagni D. An advanced diffusion model to identify emergent research issues: the case of optoelectronic devices. Scientometrics. 2010;83:765–81.

Wang JL, Xu CF, Li G, Dai ZW, Luo GJ. Understanding research field evolving and trend with dynamic Bayesian networks. Advances in Knowledge Discovery and Data Mining, Proceedings. 2007;4426:320–31.

Wang XG, Cheng QK, Lu W. Analyzing evolution of research topics with NEViewer: a new method based on dynamic co-word networks. Scientometrics. 2014;101:1253–71.

Rosvall M, Bergstrom CT. Mapping Change in Large Networks. Plos One. 2010;5.

Cobo MJ, Lopez-Herrera AG, Herrera-Viedma E, Herrera F. SciMAT: A new science mapping analysis software tool. Journal of the American Society for Information Science and Technology. 2012;63:1609–30.

Boyack KW, Klavans R, Borner K. Mapping the backbone of science. Scientometrics. 2005;64:351–74.

Callon M, Law J, Rip A. Mapping the dynamics of science and technology[J]. Book, 1986.

Yao Q, Lyu PH, Ma FC, Yao L, Zhang SJ. Global informetric perspective studies on translational medical research. Bmc Medical Informatics and Decision Making. 2013;13.

Bajwa RS, Yaldram K, Rafique S. A scientometric assessment of research output in nanoscience and nanotechnology: Pakistan perspective. Scientometrics. 2013;94:333–42.

Bajwa RS, Yaldram K. Bibliometric analysis of biotechnology research in Pakistan. Scientometrics. 2013;95:529–40.

Jackson CL. General practice in Australia 2020: "robust and ready" or "rudderless and reeling"? Medical Journal of Australia. 2006;185:125–7.

McGeoch GRB, Willsman KJ, Dowson CA, Town GI, Frampton CM, McCartin FJ, et al. Self-management plans in the primary care of patients with chronic obstructive pulmonary disease. Respirology. 2006;11:611–8.

Wagner EH. Chronic disease care - Insights from managed care in the United States will help the NHS. British Medical Journal. 2004;328:177–8.

Piterman L, Koritsas S. Part I. General practitioner-specialist relationship. Intern Med J. 2005;35:430–4.

Gillespie U, Alassaad A, Henrohn D, Garmo H, Hammarlund-Udenaes M, Toss H, et al. A comprehensive pharmacist intervention to reduce morbidity in patients 80 years or older: a randomized controlled trial. Arch Intern Med. 2009;169:894–900.

Marshall M N. How well do general practitioners and hospital consultants work together? A qualitative study of cooperation and conflict within the medical profession[J]. British Journal of General Practice. 1998;48(432):1379-1382.

Pullon S. What is the place of general practice within primary health care–in the Aotearoa New Zealand context. New Zealand Family Physician. 2008;35:301–3.

Mahmood-Yousuf K, Munday D, King N, Dale J. Interprofessional relationships and communication in primary palliative care: impact of the Gold Standards Framework. Br J Gen Pract. 2008;58:256–63.

Craven MA, Bland R. Depression in primary care: current and future challenges. Can J Psychiatry. 2013;58:442–8.

Gask L. Educating family physicians to recognize and manage depression: where are we now? Can J Psychiatry. 2013;58:449–55.

Young AF, Dobson AJ, Byles JE. Access and equity in the provision of general practitioner services for women in Australia. Aust N Z J Public Health. 2000;24:474–80.

van Hezewijk M, van den Akker ME, van de Velde CJ, Scholten AN, Hille ET. Costs of different follow-up strategies in early breast cancer: a review of the literature. Breast. 2012;21:693–700.

Hinrichs T, Brach M. The general practitioner's role in promoting physical activity to older adults: a review based on program theory. Curr Aging Sci. 2012;5:41–50.

Peat G, McCarney R, Croft P. Knee pain and osteoarthritis in older adults: a review of community burden and current use of primary health care. Ann Rheum Dis. 2001;60:91–7.

Schoenmakers B, Buntinx F, Delepeleire J. What is the role of the general practitioner towards the family caregiver of a community-dwelling demented relative? Scandinavian Journal of Primary Health Care. 2009;27:31–40.

Cottrell E, Roddy E, Foster NE. The attitudes, beliefs and behaviours of GPs regarding exercise for chronic knee pain: a systematic review. Bmc Family Practice. 2010;11.

Singer S, Gianinazzi ME, Hohn A, Kuehni CE, Michel G. General practitioner involvement in follow-up of childhood cancer survivors: a systematic review. Pediatr Blood Cancer. 2013;60:1565–73.

Nocon A, Leese B. The role of UK general practitioners with special clinical interests: implications for policy and service delivery. Br J Gen Pract. 2004;54:50–6.

PubMed   PubMed Central   Google Scholar  

Coast J, Noble S, Noble A, Horrocks S, Asim O, Peters TJ, et al. Economic evaluation of a general practitioner with special interest led dermatology service in primary care. Bmj-British Medical Journal. 2005;331:1444–8.

Moffat MA, Sheikh A, Price D, Peel A, Williams S, Cleland J, et al. Can a GP be a generalist and a specialist? Stakeholders views on a respiratory General Practitioner with a special interest service in the UK. BMC Health Serv Res. 2006;6:62.

Europe W. The European definition of general practice/family medicine. Barcelona: WONCA Europe; 2002.

Download references

Acknowledgements

The authors are grateful to Lan Yao and Hassan Dib for their helpful discussion and suggestions. The authors would also like to thank the anonymous reviewers for their valuable comments.

This research was sponsored by the China Health Promotion Foundation (no. 0231516031).

Author information

Authors and affiliations.

School of Medicine and Health Management, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430030, Hubei, China

Yang Hong, Ying Yang, Jun-jian Feng, Shu-de Wu, Wen-xue Ji, Lan Yao & Zhi-yong Liu

School of Political Science and Public Administration, Wuhan University, Wuhan, 430072, Hubei, China

You can also search for this author in PubMed   Google Scholar

Corresponding authors

Correspondence to Qiang Yao or Zhi-yong Liu .

Additional information

Competing interests.

The authors declare that they have no competing interests.

Authors’ contributions

YH and ZYL conceived the study. QY and WXJ designed the study and carried out the data analyses. LY, YY, JJF, and SDW participated in the interpretation of data. All authors read and approved the final manuscript.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Hong, Y., Yao, Q., Yang, Y. et al. Knowledge structure and theme trends analysis on general practitioner research: A Co-word perspective. BMC Fam Pract 17 , 10 (2016). https://doi.org/10.1186/s12875-016-0403-5

Download citation

Received : 06 December 2014

Accepted : 15 January 2016

Published : 29 January 2016

DOI : https://doi.org/10.1186/s12875-016-0403-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Research trends
• Co-word analysis
• Social network analysis
• Knowledge map

BMC Primary Care

ISSN: 2731-4553

• General enquiries: [email protected]

• DOI: 10.1007/S12186-016-9153-8
• Corpus ID: 147601994

What is (good) practitioner research?

• Hannu L. T. Heikkinen , F. D. Jong , R. Vanderlinde
• Published 11 March 2016
• Vocations and Learning

53 Citations

Teacher-researchers’ quality concerns for practice-oriented educational research, complexities of practitioner research: seeking hallmarks of quality.

• Highly Influenced

Developing pedagogical practices under umbrellas of different colours

Quality criteria for conceptual technology integration models in education: bridging research and practice, vocationalising specialized concepts: appropriating meanings through feedback, teachers changing higher education : from coping with change to embracing change, how do student teachers experience the development of teacher identity during a three-year professional education degree.

• 11 Excerpts

Guided play as a pedagogical tool for the early grades

Acting upon student voice-based teaching assessment initiatives: an account of participatory action research for teacher professional learning, interlude: towards a better understanding of teacher educators’ professional development: teacher educators’ researcherly disposition as a promising concept, 57 references, what is practitioner based enquiry, the gap between educational research and practice: a literature review, symposium, and questionnaire, the gap between educational research and practice: views of teachers, school leaders, intermediaries and researchers., beyond locality: the creation of public practice‐based knowledge through practitioner research in professional learning communities and communities of practice. a review of three books on practitioner research and professional communities, doing practitioner research, the role of theory in the professional development of an educational theorist, constructing practitioner research, educational research as a practical science, the new paradigm wars: is there room for rigorous practitioner knowledge in schools and universities, self-study in teacher education, related papers.

Showing 1 through 3 of 0 Related Papers

Research Methods for General Practitioners and Developing Small Scale General Practice Research

FMCH Editorial Office

1. The gap between general practice research and general practitioners

In March 2019, editors of Family Medicine and Community Health (FMCH) took part in the annual Cross-Strait Conference of General Medicine in Zhengzhou, China, with a participation of over 3,500 general practitioners from all over China. When discussing general practice research at the conference, many community doctors said: “The general practitioner is a doctor who understands all aspects of medicine, but is not expert in any field. They are just grassroots doctors dealing with daily medical problems in the community.” “We want to do research, but we can’t do it. Some leaders (of hospital/clinics) support us to do research, but we don’t know how to do it.”

In fact, family medicine and community health is a highly complex discipline that includes clinical medicine, epidemiology, public health, preventive medicine, education, medical economics, and even sociology (Table 1). In 2009, the European General Practice Research Network (EGPRN) developed a guideline Research agenda for general practice/family medicine and primary health care in Europe , in which general medical research is divided into six categories. It systematically summarizes the content and applicable methods of various general practice research categories based on a comprehensive literature search. (Table 2) [1]

Although it is one of the wealthiest regions in the world, Europe is also a continent of vast ethnic and institutional diversity. Through decades of efforts, “small world” European general practitioners have set up a theoretical research framework for general practice research in Europe. This framework of EGPRN offers great value for promotion and further improvement of general practice research. However, the vast majority of general practitioners in family medicine-emerging countries such as China know very little about it.Indeed, the gap between general practice research and general practitioners does not only exist in China. A number of studies in Europe, North America, Australia, and China have shown that a large number of general practitioners are enthusiastic about research, but do not understand research methods. [2] [3] [4] [5] [6] Faced with the dilemma of diminishingly little time for research, their enthusiasm and ideals are constantly eroded away and even can become a source of distress. Lack of time and resources results in irreparable loss for general practice research and for the development of the academic enterprise.General practice is a very distinct discipline and specialty. As the foundation of the national public health system in most countries, general practice has the largest number of doctors of all medical specialties. However, the understanding and learning of developments in medical science among many general practitioners often arrests at the level of clinical experience through inductive reasoning. General practitioners may lack skills in using critical thinking to make evidence-based decisions about quality improvement, about the clinical services they provide, about the relation of experiences and outcomes of patients. The lack of skills in the development of organizations and even industries, may become obstacles for personal and career development. When these obstacles accumulate across the population of general practitioners, they hinder the development of the discipline of general practice.Therefore, organized family medicine must bring the theoretical and applied tools of scientific research methodology to every general practitioner in order to mitigate the dilemma of choosing between completing their daily clinical work and conducting scientific research. While there will remain a role for many of the highly resource-intensive studies exemplified in the EGPRN framework in Table 2, many research studies do not require intensive resources that may only be possible in the academic institutions with active general practice research departments. FMCH also recognizes that that small scale research can contribute significantly to the science and methods of our discipline.  What has long been needed is a series of “how to” research methods papers to guide small-scale projects that are possible not only to GPs in research resource rich institutions, but also to GPs in practice outside of academic institutions.

2. Significance of the special issue on methodology in Family Medicine and Community Health

In this context, in late 2018, the Editorial Board of Family Medicine and Community Health（FMCH）invited Michael D. Fetters and Timothy C. Guetterman, two well-known professors in the field of methodology from the Mixed Methods Program in the Department of Family Medicine at the University of Michigan to develop a special issue on research methods that can be practically conducted on a small scale and in settings with limited research resources. They organized more than ten research methodologists and general practitioners from the United States, Japan, and Spain to compile the 2 nd issue of 2019 in FMCH—a special issue on methodological guidance for doing family medicine and community health research in resource limited settings. This issue illustrating the melding of respected research methodologies and family medicine research may serve as a model for the research setting in the future.This issue has an editorial and ten methodological papers with clear, rigorous, logical, step-by-step guidance, and detailed examples. The Special Issue first examines the significance and necessity of family medicine and community health research, how practitioners can transition their daily work to include research, and how to choose among research methods when just starting with a topic. Then they introduce six research methods suitable for practitioners aspiring to do research with specific examples and steps to ensure rigorous quality. Finally, two articles introduce fundamental steps for analysis of quantitative and qualitative data. Already these papers have become an effective collection of systematic methodological guides that can be directly applied by family doctors for learning and conducting research (Table 3).

Table 3 Order and content of methodological articles in the special issue of FMCH

One of the major challenges for conducting research in general practice is the gap between scientific theory and the content of the general practitioner’s daily work. By integrating the theoretical concepts of classification, research paradigms, confidence intervals, bias, and rigorous exploration using the scientific method into examining aspects of routine daily practice, medical records, consultations, prescriptions and health education approaches, general practitioners can combine theory and practice together and bring about practical improvements.Professor Fetters and Professor Guetterman, as well as the other methodology, medical education and clinical experts involved in writing this issue, provide a solution for overcoming the obstacle of linking practice with research. The six research methods recommended by them are easy for general practitioners actually to use, and achieve meaningful research results in one to two years. In particular, the mixed methods procedures illustrate with clear explanation how learning and using the approach can be achieved by linking the theory, philosophy and science. Mixed methods procedures helps break down the traditional pyramidal hierarchy of evidence-based medicine, and opens up a path for general practice research.The development of modern general practice is not only directly related to preventing and overcoming the causes of human suffering, but also faces arduous challenges. In order to overcome the current difficulties and promote the development of general practice, general practitioners can work together and solve the problems one by one through hard work in clinical, educational, scientific research and management. One of the top priorities is to promote and popularize the production and consumption of scientific research knowledge. As long as general practitioners possess critical thinking skills in science and grasp appropriate methods, they will “elevate themselves” and the discipline. They will accelerate their influence for the betterment of health, and write the history of advancing family medicine research and the development of the discipline.Professors Fetters and Guetterman wrote in their editorial that, “We dedicate this issue to aspiring family medicine and community health researchers. We include as our audience students, residents and fellows who are still learning the craft of clinical care, clinician educators making innovative strides in teaching, and experienced clinical practitioners who are inquisitive and want to contribute to the science of family medicine and community health. We hope this special issue will serve as a single, online and open-access resource with strategies for taking project ideas to researchable questions or evaluations. We look forward to seeing your original research and evaluation articles in Family Medicine and Community Health .” [7] The staff of FMCH Editorial Board would like to express our heartfelt respect and gratitude to Professor Fetters, Professor Guetterman and other methodologists, educators and clinicians who created this special issue. We will do our best to promote this issue freely and provide ready access to every general practitioner who is committed to improving the health of patients and communities, by constantly transcending their ideals, and pursuing a scientific spirit of excellence. Although they face the threat of burnout in their daily grind, general practitioners may rejuvenate by continuing to learn and study the science of research after hours. We believe this special issue can help them carry out research and will be a powerful tool for equipping them to do research that will ensure patient safety, overcome disease and minimize health risk factors.

References:

[1] Hummers-Pradier E, Beyer M, Chevallier P, et al. Research agenda for general practice/family medicine and primary health care in Europe[J]. Maastricht (Holland): European General Practice Research Network EGPRN, 2009.

[2] Huas C, Petek D, Diaz E, et al. Strategies to improve research capacity across European general practice: The views of members of EGPRN and Wonca Europe[J]. European Journal of General Practice, 2018: 1-7. DOI: 10.1080/13814788.2018.1546282

[3] Ryan B L, Thorpe C, Zwarenstein M, et al. Building research culture and capacity in academic family medicine departments: Insights from a simulation workshop[J]. Canadian Family Physician, 2019, 65(1): e38-e44. [4] YANG H. Publications in area of General Practice: five year’s review (2013-17) and suggestions for general practice research[J]. 2019.

[5] 刘蕊,李乐园,石建伟, 等.上海市杨浦区社区全科医生的科研需求分析[J].医学与社会,2017,30(1):21-23. DOI:10.13723/j.yxysh.2017.01.007.

[6] Howe A, Kidd M. Challenges for family medicine research: a global perspective[J]. Family practice, 2018, 36(2): 99-101.

[7] Fetters MD, Guetterman TC Discovering and doing family medicine and community health research Family Medicine and Community Health 2019;7:e000084. doi: 10.1136/fmch-2018-000084

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

•  We're Hiring!
•  Help Center

General practitioner

• Most Cited Papers
• Most Downloaded Papers
• Newest Papers
• Save to Library
• Last »
• Tetarudi Follow Following
• Orthopedic Surgery Follow Following
• General Practice Follow Following
• Cardiovascular disease Follow Following
• Dental Surgery Follow Following
• Neurosurgery Follow Following
• General Surgery Follow Following
• Erectile dysfunction Follow Following
• Natural Science Follow Following
• Mahasiswa Follow Following

Enter the email address you signed up with and we'll email you a reset link.

• Academia.edu Publishing
•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

• - Google Chrome

Intended for healthcare professionals

• My email alerts
• BMA member login
• Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Search form

• Advanced search
• Search responses
• Search blogs
• General practitioners'...

General practitioners' perceptions of effective health care

• Related content
• Peer review
• Zelda Tomlin , research fellow ,
• Charlotte Humphrey , senior lecturer in sociology ( charlot{at}rfhom.ac.uk ) ,
• Stephen Rogers , senior lecturer in primary health care.
• Department of Primary Care and Population Sciences, Royal Free and University College Medical School, University College London, London NW3 2PF
• Correspondence to: Dr Humphrey
• Accepted 29 January 1999

Objectives: To explore general practitioners' perceptions of effective health care and its application in their own practice; to examine how these perceptions relate to assumptions about clinicians' values and behaviour implicit in the evidence based medicine approach.

Design: A qualitative study using semistructured interviews.

Setting: Eight general practices in North Thames region that were part of the Medical Research Council General Practice Research Framework.

Participants: 24 general practitioners, three from each practice

Main outcome measures: Respondents' definitions of effective health care, reasons for not practising effectively according to their own criteria, sources of information used to answer clinical questions about patients, reasons for making changes in clinical practice.

Results: Three categories of definitions emerged: clinical, patient related, and resource related. Patient factors were the main reason given for not practising effectively; others were lack of time, doctors' lack of knowledge and skills, lack of resources, and “human failings.” Main sources of information used in situations of clinical uncertainty were general practitioner partners and hospital doctors. Contact with hospital doctors and observation of hospital practice were just as likely as information from medical and scientific literature to bring about changes in clinical practice.

Conclusions: The findings suggest that the central assumptions of the evidence based medicine paradigm may not be shared by many general practitioners, making its application in general practice problematic. The promotion of effective care in general practice requires a broader vision and a more pragmatic approach which takes account of practitioners'concerns and is compatible with the complex nature of their work.

Key messages

Evidence based medicine has emerged as a new paradigm to prevent inappropriate variations in clinical practice

This study explored the extent to which evidence based medicine's emphasis on clinical effectiveness, self analysis, and information seeking is congruent with the modes of thinking and behaviour of general practitioners

General practitioners' definitions of effective health care fell into three categories of clinical, patient related, and resource related; their main reason for not practising effectively was patient factors, and others were lack of time, lack of knowledge and skills, lack of resources, and “human failings”; and their main sources of information in cases of clinical uncertainty were general practitioner partners and hospital doctors

The central assumptions of the evidence based medicine paradigm may not be shared by many general practitioners, making its application in general practice problematic

Promotion of effective care in general practice requires a broader vision and a more pragmatic approach that takes account of practitioners' concerns and is compatible with the complex nature of their work

Introduction

The concept of effectiveness has come to dominate the healthcare debate. The emergence of evidence of variations in practice, with accompanying doubts on the clinical effectiveness of some of those practices, 1 2 has shown the need for a fundamental questioning of the way in which clinical decisions are made, identifying the reasons for such variation, and finding ways of addressing inappropriate variations. 3 4

Awareness of the latest scientific evidence and the ability to critically appraise and assess the applicability of this evidence have been identified as crucial ingredients that are missing in everyday medical practice, and evidence based medicine has emerged as a new paradigm. 5 Evidence based medicine is defined as the “conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.” 6 This approach is underpinned by the assumption that practitioners regard clinical effectiveness as a priority and will be keen to act in a way that optimises this—that is, to question their clinical practice systematically and, where shortcomings are identified, change that practice in line with scientifically valid evidence.

Our research, carried out in general practice, explores the extent to which this emphasis on clinical effectiveness, self analysis, and information seeking is congruent with the modes of thinking and behaviour of a service general practitioner. We asked doctors to describe how they defined effectiveness in health care, whether they thought they always practised effectively and, if not, why not. We also asked them how they sought answers to clinical questions about individual patients and about recent changes in their own clinical practice and how these had come about.

Participants and methods

The study was carried out in the summer of 1997 at eight practices in the North Thames region that were members of the Medical Research Council General Practice Research Framework, a network of about 900 practices that have expressed a willingness to participate in MRC research projects. 7 Framework practices are reimbursed for the time that practice nurses spend on the projects, have lockable filing facilities, and out of hours surgeries for study patients. The practices were recruited from those that responded to an invitation to participate in a feasibility study for a randomised controlled trial of implementation strategies to promote use of evidence based guidelines.

The data presented in this paper were collected as part of baseline interviews for the feasibility study, which were undertaken with 24 doctors, three from each practice. For each practice, those interviewed included at least one general practitioner with close links with the MRC, one woman general practitioner, and one general practitioner who was sceptical about joining the study, as identified during preliminary meetings. The interviews were semistructured and lasted about an hour. As well as the issues reported here, the interview covered the general practitioners' views on implementing research findings in clinical practice, practice guidelines and quality in health care, the characteristics of their practices, and their expectations of the feasibility study.

All the interviews were undertaken by the same researcher and were audio taped and subsequently transcribed. The transcripts were read to identify themes from individual responses, and these were then grouped into categories to produce typologies. To ensure reliability, two researchers analysed the data independently and the results were compared. Both generated similar systems of categorisation, although these were initially described in slightly different terms.

All 24 respondents were general practitioner principals, and 10 were women. Their mean age was 43 (range 29-61). The eight practices had between three and eight partners, all but one were fundholding, and two were training practices.

Definitions

Responses to the question on how general practitioners defined effective health care fell into three broad categories: clinical, patient related, and resource related. Just over half the respondents (13) offered more than one type of definition.

Clinical definitions

Most of the respondents (18) offered definitions that were centred around appropriate investigation, treatment, referral, and follow up; improving morbidity and mortality; and curing and preventing disease. The definitions in this category included two distinct groups: firstly, definitions that were strictly focused on disease—such as, “We ought to be able to see that the disease process is treated”—and, secondly, definitions that incorporated a sense of the patient—such as, “That the patient has got better from whatever it was or that you've alleviated the suffering in some way.”

Patient related definitions

Just under half the respondents (10) offered definitions that were more patient oriented. The most common theme was educating patients and giving them relevant information so that they were able to participate in the decision making process. For example, effectiveness was defined as “helping [patients] to come to a level of understanding such that they can personally make the decision about what happens to them.” Three respondents referred to patient satisfaction as a condition of effectiveness. Another theme, offered by one respondent, was temporarily acquiescing to patients' expectations in order to secure compliance: “You might give somebody a big dose of something to make them better quickly so that they then believe you later when you want to change things.”

Resource related definitions

Eight respondents gave definitions related to resources. These included ideas about cost effective care for individual patients and about a population perspective—for example: “All our decision making before used to be [that] a patient would come to you, and you just made a decision on the basis of that patient, but now I think there is a need for it to be made in a wider context.”

Reasons for not practising effectively

Only one respondent thought he always practised effectively. The rest admitted departing from their own models of effective health care in everyday practice. Four categories of reasons for this emerged, with 15 respondents citing more than one category.

Doctor related reasons

Fourteen respondents mentioned factors that originated from the doctor, either as a professional or as an individual. Nine respondents cited self perceived shortcomings in knowledge, experience, and skills and how well these were applied in practice—for example: “I don't pretend to be up to date all the time” and “Pressure from conflicting ideas so that you don't really know if you are right or wrong.” In addition, feelings of being tired, stressed, or unmotivated (what one doctor called “human failings”) were referred to by five respondents—“[In] periods when I've been under enormous personal stress … my referral patterns shoot up.”

Patient related reasons

Seven respondents mentioned factors that emanated from patients. It was suggested that when patients presented with more than one problem, sometimes acknowledged and sometimes hidden, it was necessary to prioritise even if this meant ignoring some of the problems. One example given was that of an overweight smoker who had had a heart attack; the general practitioner thought that the smoking should be tackled first and the obesity left for later consideration. Alternatively, the effective treatment of one condition might exacerbate the symptoms of another. The example was given of a patient with severe coronary artery disease, gastrointestinal disease, and severe osteoarthritis, for whom appropriate treatment of the osteoarthritis with non-steroidal anti-inflammatory drugs might exacerbate the gastrointestinal problems. Two respondents also mentioned patients' non-compliance as an obstacle to practising effectively.

Doctor and patient related reasons

Eighteen respondents referred to factors associated with the interaction between doctor and patient. There was a pervading feeling that patients' cultural backgrounds, beliefs and attitudes, and levels of understanding resulted in certain expectations that sometimes clashed with the requirements of clinical effectiveness. A commonly cited example was patients demanding to be prescribed antibiotics for respiratory tract infections that were likely to be viral. Other examples were requesting investigations when the results were likely to be negative and requesting inappropriate referrals.

Various reasons were given for bowing to such demands. Firstly, respondents wanted to avoid conflict with their patients—“You might agree to investigate someone because you just can't stand this person nagging you on and on about their complaints.” Secondly, they were keen to keep the “custom” of their patients. As one respondent observed, “Some doctors frighten their patients away because they're so blooming effective. So nobody goes round to see them, and they think they've got everything beautifully organised.” Thirdly, respondents thought that they should respect patients' views and that patients could sometimes be “right” even if their views were not corroborated by scientific evidence. Fourthly, the placebo effect was mentioned as a reason for providing “ineffective” treatments, because it “helped the healing process.” Finally, feelings of sympathy for patients led respondents to provide treatments of doubtful effectiveness—“Say it was my child, if he was suffering as much as that child is suffering, then I would certainly say, ‘Well, look, I would much rather give him the benefit of having an antibiotic,’ which is not the right thing to do, but I would do it.”

In addition, problems were identified in providing effective care for patients who were felt to be “difficult.” These were people who, for one reason or another, “defied” diagnosis and treatment—“You try every single approach and nothing has worked …. I think it's partly the personality of the patient.” It was also acknowledged that personal prejudices could result in the doctor “ignoring” certain patients or devoting less time and effort to them than was necessary.

Environmental reasons

Factors extraneous to both doctor and patient were mentioned by 19 respondents. The concern most commonly referred to (by 13 respondents) was that of time, and the strength of feeling about this was considerable. Time was seen as hindering effectiveness across all its dimensions—“Time influences everything. It influences getting a history correctly, engaging with the patient if you don't know them well, building up some sort of rapport, discussing treatment options, examining them properly.” Lack of time was felt by many respondents to result in ineffective practice because it led them to bow to inappropriate patient demands—“It may be a Friday afternoon, I want to rush off. I want to prevent this patient calling us back on Saturday afternoon, and I would prescribe antibiotics.”

Lack of resources was the other main issue, mentioned by six respondents. This was thought to adversely affect various aspects of care, such as the doctor:patient ratio, the repeat prescribing system, district nursing services, hospital referrals, and operations. One respondent pointed out that a patient waiting for more than a year for a cataract operation may experience a fall and a fracture, but “we don't say, ‘This is ascribed to the cataract or the delay in waiting lists.’ “

Questioning behaviour

We asked the general practitioners to indicate the sources of information they use when they have unanswered clinical questions about particular patients. All 24 respondents mentioned a practice partner or a hospital doctor, or both. Recourse to literature (books, journals, use of a library) was mentioned by 10 respondents, referral to outpatient clinics by four, the internet by two, and the Medline database by one doctor only.

Changes in clinical practice

We also asked what changes individual respondents had made to their own clinical practice over the past few years. A total of 17 respondents recalled 39 changes, most recalling up to three changes. These ranged from switching to a different drug or using an investigative test for the first time to changing management (such as using a lower treatment threshold). Three main categories of reasons accounted for 25 of the changes, either alone or in combination. Contact with a hospital doctor or observation of hospital practice through seeing patients after their hospital visit was given as the sole reason for six changes, journal articles were cited as the sole reason for five changes, and scientific meetings were given as the reason for four. Four changes were attributed to literature combined with hospital contact, four to scientific meetings and hospital contact and two to literature and scientific meetings. Reference to journal articles was commonly along the lines of, “I remember reading something about it,” and did not indicate a literature search or a critical appraisal process. Various reasons were given for the remainder of the changes. Five respondents spoke of a “crystallisation” or an “evolving process” incorporating several sources when elaborating on how the changes had come about.

The findings of our study suggest that the central assumptions of the evidence based medicine paradigm may not be shared by many general practitioners, making its application in general practice problematic

Limitations of study

The 24 general practitioners who participated in our study formed a small sample that was neither random nor representative and came from only one NHS region. However, given the basis on which they joined the study, it seems reasonable to assume that their interest in clinical effectiveness would, if anything, be greater than that of most general practitioners.

Doctors' concerns

The respondents seemed to be acutely aware of, and sensitive to, patients' expectations and were inclined to judge their practice in terms not only of clinical outcome but also of a patient centred interpretation of quality. Thus, in situations where the requirements of clinical effectiveness openly clash with the preferences or circumstances of individual patients, the latter might take precedence in shaping general practitioners' actions. This concurs with theories on the “holistic” nature of general practice, in which biomedical, personal, and contextual perspectives converge in the decision making process. 8 The linear decision making suggested by the model of evidence based medicine, informed chiefly by normative standards of clinical effectiveness, sits uneasily within this framework.

In general practice the doctor-patient encounter is a dynamic phenomenon underpinned by negotiation that takes account of the preoccupations of both parties. The fact that the doctor sometimes chooses to place more weight on the patient's agenda than on clinical evidence seems to be a rational strategy aimed at maintaining an important relationship. The maintenance of this relationship—which is likely to impact on the “healing” process 9 —may be more important to general practitioners than staying within the bounds of a statistically defined consensus on clinical effectiveness.

Doctors' information seeking

When faced with clinical uncertainty the respondents in this study seemed to make more use of their colleagues or hospital doctors than of scientific literature. This finding is supported by those of Barrie and Ward, who found that “desktop” and human sources were used to answer most of the questions that general practitioners generated during consultations and that literature was little used. 10

Contact with hospital colleagues and observation of their practice seems to have been as influential as literature in prompting the respondents to change their clinical practice. Allery et al also found that most changes in practice reported by the clinicians in their study (which included consultants and general practitioners) had no educational basis, and literature was mentioned as a reason for change in less than 10% of instances of change. 11

Limitations of practising evidence based medicine

There is a growing literature on the shortcomings of the evidence based medicine model in general practice, including the scope and nature of the evidence available and its limited applicability in this aspect of patient care. 12 – 14 The difficulties in disseminating evidence, identifying the best format for it, and overcoming organisational barriers to implementing it have also been examined. 15 Proponents of evidence based medicine have identified a number of problems and suggested ways of addressing these. 5

Our findings are based on a limited investigation that formed a small part of a study primarily undertaken for another purpose. Nevertheless, they suggest that the applicability of the evidence based medicine approach in general practice may be limited for more fundamental reasons associated with the assumptions that the model makes about doctors' ways of thinking and behaviour. Firstly, general practitioners may not share evidence based medicine's overarching concern with clinical effectiveness but instead see it as one consideration in a wider framework that also takes account of service oriented concerns such as patient satisfaction and time management. Secondly, even when the concept of clinical effectiveness does come to the fore and leads to self evaluation and the identification of gaps in knowledge, practitioners may prefer to turn to human rather than written sources. Thirdly, even if more evidence based sources are sought and information obtained, it is unlikely to be applied to practice if it proves unacceptable to patients or incompatible with their other needs. Some of these factors have also emerged in other studies. 16 Furthermore, “diagnosis by prognosis” and “diagnosis by therapeutic response”, both common in the uncertain environment of general practice, 17 may preclude the formulation of clear clinical questions demanded by the evidence based medicine model.

The suggested routes to practising evidence based medicine in a clinical setting—acquiring and using critical appraisal skills in everyday patient encounters or, to save time, using evidence based databases and guidelines 18 —fail to adequately comprehend the complex nature of general practice. There is doubtless a need to improve clinical quality in general practice, as in hospital medicine. But policies aimed at this objective need to take account of the concerns of practitioners and should be compatible with the nature of their work; furthermore, they need to be built on an empirical understanding of how knowledge comes to underpin practice, which may, for good reason, be far from any rationalist ideal.

Acknowledgments

We thank other members of the research group, especially Professor A Haines, Dr I Nazareth, andS Lister for their comments on the paper. The study was carried out in collaboration with the MRC General Practice Research Framework and we are grateful to all the general practitioners who participated.

Contributors: A Haines had the original idea for the feasibility study, initiated the research, and discussed core aspects of its design and execution. SR was overall coordinator of both design and execution of the feasibility study, organised recruitment of the study practices, and developed packs of evidence based guidelines used in the study. ZT and CH designed the qualitative component of the study, including the interviews reported on here. I Nazareth and S Lister developed and implemented the interventions strategies used in the feasibility study. ZT, CH, and SR participated in writing this paper. A Haines, I Nazareth, and S Lister commented on drafts of the paper. CH is guarantor for the paper.

Funding The study was supported by a grant from the NHS Implementation Methods Programme.

Competing interest None declared.

• D'Annuo T ,
• Coulter A ,
• Klassen A ,
• MacKenzie IZ ,
• McPherson K
• Dunning M ,
• MacDonald J
• Evidence-Based Medicine Working Group
• Sackett DL ,
• Rosenberg WMC ,
• Haynes RB ,
• Richardson WS
• Medical Research Council General Practice Research Framework
• Jacobson LD ,
• Edwards AGK ,
• Granier SK ,
• Sullivan FM ,
• MacNaughton RJ
• Barrie AR ,
• Allery LA ,
• Greenhalgh T
• Rosenberg WMC

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

General Practitioners' Participation in a Large, Multicountry Combined General Practitioner-Patient Survey: Recruitment Procedures and Participation Rate

Affiliations.

• 1 Netherlands Institute for Health Services Research (NIVEL), P.O. Box 1568, 3500 BN Utrecht, Netherlands; Department of Sociology and Department of Human Geography, Utrecht University, Heidelberglaan 2, 3584 CS Utrecht, Netherlands.
• 2 Hochschule Fulda University of Applied Sciences, Leipziger Straße 123, 36037 Fulda, Germany.
• 3 Netherlands Institute for Health Services Research (NIVEL), P.O. Box 1568, 3500 BN Utrecht, Netherlands.
• PMID: 27047689
• PMCID: PMC4800081
• DOI: 10.1155/2016/4929432

Background. The participation of general practitioners (GPs) is essential in research on the performance of primary care. This paper describes the implementation of a large, multicountry study in primary care that combines a survey among GPs and a linked survey among patients that visited their practice (the QUALICOPC study). The aim is to describe the recruitment procedure and explore differences between countries in the participation rate of the GPs. Methods. Descriptive analyses were used to document recruitment procedures and to assess hypotheses potentially explaining variation in participation rates between countries. Results. The survey was implemented in 31 European countries. GPs were mainly selected through random sampling. The actual implementation of the study differed between countries. The median participation rate was 30%. Both material (such as the payment system of GPs in a country) and immaterial influences (such as estimated survey pressure) are related to differences between countries. Conclusion. This study shows that the participation of GPs may indeed be influenced by the context of the country. The implementation of complex data collection is difficult to realize in a completely uniform way. Procedures have to be tuned to the context of the country.

PubMed Disclaimer

Similar articles

• Differences in medical services in Nordic general practice: a comparative survey from the QUALICOPC study. Eide TB, Straand J, Björkelund C, Kosunen E, Thorgeirsson O, Vedsted P, Rosvold EO. Eide TB, et al. Scand J Prim Health Care. 2017 Aug 3:1-10. doi: 10.1080/02813432.2017.1358856. Online ahead of print. Scand J Prim Health Care. 2017. PMID: 28768442
• Two decades of change in European general practice service profiles: conditions associated with the developments in 28 countries between 1993 and 2012. Schäfer WL, Boerma WG, Spreeuwenberg P, Schellevis FG, Groenewegen PP. Schäfer WL, et al. Scand J Prim Health Care. 2016;34(1):97-110. doi: 10.3109/02813432.2015.1132887. Epub 2016 Feb 10. Scand J Prim Health Care. 2016. PMID: 26862927 Free PMC article.
• Implementation of evidence-based knowledge in general practice. Le JV. Le JV. Dan Med J. 2017 Dec;64(12):B5405. Dan Med J. 2017. PMID: 29206099
• General Practitioners' intention to prescribe and prescribing patterns in selected European settings: The OTCSOCIOMED project. Tsiantou V, Moschandreas J, Bertsias A, Papadakaki M, Saridaki A, Agius D, Alper Z, Faresjo T, Klimkova M, Martinez L, Samoutis G, Vlček J, Lionis C. Tsiantou V, et al. Health Policy. 2015 Sep;119(9):1265-74. doi: 10.1016/j.healthpol.2015.06.006. Epub 2015 Jul 4. Health Policy. 2015. PMID: 26188356
• The role of the general practitioner in the screening and clinical management of chronic viral hepatitis in six EU countries. Bechini A, Levi M, Falla A, Ahmad A, Veldhuijzen I, Tiscione E, Bonanni P. Bechini A, et al. J Prev Med Hyg. 2016;57(2):E51-60. J Prev Med Hyg. 2016. PMID: 27582629 Free PMC article. Review.
• Social concordance and patient reported experiences in countries with different gender equality: a multinational survey. Eggermont D, Kunst AE, Groenewegen PP, Verheij RA. Eggermont D, et al. BMC Prim Care. 2024 Mar 23;25(1):97. doi: 10.1186/s12875-024-02339-y. BMC Prim Care. 2024. PMID: 38521895 Free PMC article.
• Primary care involvement in clinical research - prerequisites, motivators, and barriers: results from a study series. Wangler J, Jansky M. Wangler J, et al. Arch Public Health. 2024 Mar 20;82(1):41. doi: 10.1186/s13690-024-01272-x. Arch Public Health. 2024. PMID: 38504310 Free PMC article.
• Case-mix adjustments for patient reported experience and outcome measures in primary care: an empirical approach to identify patient characteristics as case-mix adjusters based on a secondary analysis of an international survey among patients and their general practitioners in 34 countries. Groenewegen PP, Spreeuwenberg P, Leyland AH, de Boer D, Boerma W. Groenewegen PP, et al. J Patient Rep Outcomes. 2023 Dec 4;7(1):127. doi: 10.1186/s41687-023-00667-8. J Patient Rep Outcomes. 2023. PMID: 38048040 Free PMC article.
• Quick adaptation of the organisation of general practices during the COVID-19 pandemic in the Netherlands. Groenewegen PP, van den Muijsenbergh M, Batenburg R, Van Poel E, van den Broek S, Bussche PV, Willems S. Groenewegen PP, et al. BMC Prim Care. 2023 Aug 31;24(Suppl 1):170. doi: 10.1186/s12875-023-02114-5. BMC Prim Care. 2023. PMID: 37653405 Free PMC article.
• GPs' involvement in specialised palliative home care: A mixed methods study in Germany. Peter S, Volkert AM, Radbruch L, Rolke R, Voltz R, Pfaff H, Scholten N; APVEL-consortium. Peter S, et al. Eur J Gen Pract. 2022 Dec;28(1):224-233. doi: 10.1080/13814788.2022.2139824. Eur J Gen Pract. 2022. PMID: 36369760 Free PMC article.
• Cook J. V., Dickinson H. O., Eccles M. P. Response rates in postal surveys of healthcare professionals between 1996 and 2005: an observational study. BMC Health Services Research. 2009;9, article 160 doi: 10.1186/1472-6963-9-160. - DOI - PMC - PubMed
• Pit S. W., Vo T., Pyakurel S. The effectiveness of recruitment strategies on general practitioner's survey response rates—a systematic review. BMC Medical Research Methodology. 2014;14, article 76 doi: 10.1186/1471-2288-14-76. - DOI - PMC - PubMed
• PHAMEU database. 2014, http://www.nivel.nl/en/dossier/country-information-primary-care .
• Schäfer W. L. A., Boerma W. G. W., Kringos D. S., et al. QUALICOPC, a multi-country study evaluating quality, costs and equity in primary care. BMC Family Practice. 2011;12, article 115 doi: 10.1186/1471-2296-12-115. - DOI - PMC - PubMed
• Schäfer W. L. A., Boerma W. G. W., Kringos D. S., et al. Measures of quality, costs and equity in primary health care instruments developed to analyse and compare primary care in 35 countries. Quality in Primary Care. 2013;21(2):67–79. - PubMed

Related information

Linkout - more resources, full text sources.

• Europe PubMed Central
• Hindawi Limited
• PubMed Central

Other Literature Sources

• scite Smart Citations
• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

• Advanced search

Advanced Search

How the RCGP Research Paper of the Year 2020 reflects our motto ‘Cum Scientia Caritas’

• Find this author on Google Scholar
• Find this author on PubMed
• Search for this author on this site
• For correspondence: [email protected]
• Figures & Data

The Research Paper of the Year (RPY), awarded by the Royal College of General Practitioners (RCGP), gives recognition to an individual or group of researchers who have undertaken and published an exceptional piece of research relating to general practice or primary care.

The three categories are Clinical Research, Health Services Research (including Implementation and Public Health), and Medical Education related to Primary Care. Papers are scored on the criteria of originality, impact, contribution to the reputation of general practice, scientific approach, and presentation. This year, we invited submissions reporting COVID-19 research to each category.

The RCGP’s motto Cum Scientia Caritas means ‘scientific knowledge applied with compassion’. We feel the winning papers for RPY 2020 really reflect this ethos.

• THE OVERALL WINNER

The overall winner of the RPY 2021 award, from 53 submissions, came from Sohal and colleagues from London and Bristol: ‘Improving the healthcare response to domestic violence and abuse in UK primary care: interrupted time series evaluation of a system-level training and support programme’ . 1

The reviewing panel felt that this paper, submitted to Category 2 (Health Services Research) was particularly relevant in the light of COVID-19 restrictions and widespread reports of increased domestic violence during lockdowns.

This programme of work, building on the IRIS (Identification and Referral to Improve Safety) trial, 2 provides evidence that a system-level programme that embeds direct referral pathways to specialist domestic violence and abuse (DVA) agencies within health services, underpinned by face-to-face training of clinicians and their teams, including on-going reinforcement strategies, improves the case identification and referrals for DVA.

This study exemplifies the need for recognition, support, and compassion for this vulnerable group of patients.

• HIGHLY COMMENDED OVERALL

The panel judged Funston and colleagues’ paper, winner of Category 1, ‘The diagnostic performance of CA125 for the detection of ovarian and non-ovarian cancer in primary care: a population-based cohort study’ , 3 should also be ‘highly commended’ overall.

This research concluded that CA125 is a useful test for ovarian cancer detection in primary care, particularly in women over 50 years old. The research team found that almost a third of women with CA125 levels above the suggested cut-off were diagnosed with some form of cancer, suggesting that clinicians should also consider non-ovarian cancers in these women, especially if ovarian cancer has been excluded, in order to prevent diagnostic delay.

These results will enable clinicians and patients to determine the estimated probability of ovarian cancer and all cancers at any CA125 level and age, which can be used to guide discussions with women and individual decisions on the need for further investigation or referral.

This research highlights the contribution of primary care research to science and to the evidence-base that guides our practice, and can be of immediate benefit to patients.

• COVID-19 RESEARCH PAPER OF THE YEAR

The RPY reviewing panel decided to name Clift and colleagues’ paper, ‘Living risk prediction algorithm (QCOVID) for risk of hospital admission and mortality from coronavirus 19 in adults: national derivation and validation cohort study’ , 4 as COVID-19 RPY. The panel were impressed by the speed of this work completed during the pandemic to develop an algorithm based on a number of factors to determine risk of death and hospital admission.

The study is an excellent example of the value of routine primary care records and will have impact on clinician decision making and patient care.

• THIRD CATEGORY WINNER

The winner of our third category, Medical Education related to Primary Care, was published in the BJGP : ‘Revealing the reality of undergraduate GP teaching in UK medical curricula: a cross-sectional questionnaire study’ , submitted by Cottrell and colleagues. 5 The stark conclusion of this study, that undergraduate teaching provision in general practice has plateaued since 2000 and falls short of national recommendations, should be a warning to us all. The authors make important recommendations about the need for an adequate primary care teaching tariff and state that without sufficient funding, medical schools are unlikely to influence GP recruitment problems positively or be able to promote generalism for all future doctors. How can we deliver science-informed care with compassion if education in general practice does not begin in medical schools?

• PROMOTING AND FUNDING THE EXPERT GENERALIST ROLE

We hope that you will (re-)read the winning papers and reflect on how research in general practice contributes to the science, informs our clinical decision making, and supports us to deliver evidence-based care to our patients.

Cottrell’s paper reminds us that education and preparation for general practice begins as students enter medical school, and needs to be funded to ensure high-quality undergraduate GP teaching is delivered and the expert medical generalist role is promoted.

Only then can we ensure that we can encourage the next generation of doctors to enter general practice.

• Acknowledgments

The RPY award is well supported by RCGP staff in Policy, Research and Campaigns. Thank you to all panel members and chairs involved in the process for giving of their time, particularly with all the competing demands on us.

• © British Journal of General Practice 2021
• Davies RA ,
• Funston G ,
• Hamilton W ,
• Coupland CC ,
• Cottrell E ,
• Alberti H ,
• Rosenthal J ,

In this issue

• Table of Contents
• Index by author

Thank you for recommending British Journal of General Practice.

NOTE: We only request your email address so that the person to whom you are recommending the page knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Citation Manager Formats

• EndNote (tagged)
• EndNote 8 (xml)
• RefWorks Tagged
• Ref Manager

• Tweet Widget
• Facebook Like
• Google Plus One

Jump to section

More in this toc section.

• The media, healthcare messaging, and the doctor–patient relationship
• Books: Follow the Money: How Much Does Britain Cost?
• A tale of two doctors: casualties of an inverse education law?

Related Articles

Cited by....

British Journal of General Practice

McKinsey Technology Trends Outlook 2023

After a tumultuous 2022 for technology investment and talent, the first half of 2023 has seen a resurgence of enthusiasm about technology’s potential to catalyze progress in business and society. Generative AI deserves much of the credit for ushering in this revival, but it stands as just one of many advances on the horizon that could drive sustainable, inclusive growth and solve complex global challenges.

To help executives track the latest developments, the McKinsey Technology Council  has once again identified and interpreted the most significant technology trends unfolding today. While many trends are in the early stages of adoption and scale, executives can use this research to plan ahead by developing an understanding of potential use cases and pinpointing the critical skills needed as they hire or upskill talent to bring these opportunities to fruition.

Our analysis examines quantitative measures of interest, innovation, and investment to gauge the momentum of each trend. Recognizing the long-term nature and interdependence of these trends, we also delve into underlying technologies, uncertainties, and questions surrounding each trend. This year, we added an important new dimension for analysis—talent. We provide data on talent supply-and-demand dynamics for the roles of most relevance to each trend. (For more, please see the sidebar, “Research methodology.”)

New and notable

All of last year’s 14 trends remain on our list, though some experienced accelerating momentum and investment, while others saw a downshift. One new trend, generative AI, made a loud entrance and has already shown potential for transformative business impact.

Research methodology

To assess the development of each technology trend, our team collected data on five tangible measures of activity: search engine queries, news publications, patents, research publications, and investment. For each measure, we used a defined set of data sources to find occurrences of keywords associated with each of the 15 trends, screened those occurrences for valid mentions of activity, and indexed the resulting numbers of mentions on a 0–1 scoring scale that is relative to the trends studied. The innovation score combines the patents and research scores; the interest score combines the news and search scores. (While we recognize that an interest score can be inflated by deliberate efforts to stimulate news and search activity, we believe that each score fairly reflects the extent of discussion and debate about a given trend.) Investment measures the flows of funding from the capital markets into companies linked with the trend. Data sources for the scores include the following:

• Patents. Data on patent filings are sourced from Google Patents.
• Research. Data on research publications are sourced from the Lens (www.lens.org).
• News. Data on news publications are sourced from Factiva.
• Searches. Data on search engine queries are sourced from Google Trends.
• Investment. Data on private-market and public-market capital raises are sourced from PitchBook.
• Talent demand. Number of job postings is sourced from McKinsey’s proprietary Organizational Data Platform, which stores licensed, de-identified data on professional profiles and job postings. Data is drawn primarily from English-speaking countries.

In addition, we updated the selection and definition of trends from last year’s study to reflect the evolution of technology trends:

• The generative-AI trend was added since last year’s study.
• We adjusted the definitions of electrification and renewables (previously called future of clean energy) and climate technologies beyond electrification and renewables (previously called future of sustainable consumption).
• Data sources were updated. This year, we included only closed deals in PitchBook data, which revised downward the investment numbers for 2018–22. For future of space technologies investments, we used research from McKinsey’s Aerospace & Defense Practice.

This new entrant represents the next frontier of AI. Building upon existing technologies such as applied AI and industrializing machine learning, generative AI has high potential and applicability across most industries. Interest in the topic (as gauged by news and internet searches) increased threefold from 2021 to 2022. As we recently wrote, generative AI and other foundational models  change the AI game by taking assistive technology to a new level, reducing application development time, and bringing powerful capabilities to nontechnical users. Generative AI is poised to add as much as $4.4 trillion in economic value from a combination of specific use cases and more diffuse uses—such as assisting with email drafts—that increase productivity. Still, while generative AI can unlock significant value, firms should not underestimate the economic significance and the growth potential that underlying AI technologies and industrializing machine learning can bring to various industries.

Investment in most tech trends tightened year over year, but the potential for future growth remains high, as further indicated by the recent rebound in tech valuations. Indeed, absolute investments remained strong in 2022, at more than $1 trillion combined, indicating great faith in the value potential of these trends. Trust architectures and digital identity grew the most out of last year’s 14 trends, increasing by nearly 50 percent as security, privacy, and resilience become increasingly critical across industries. Investment in other trends—such as applied AI, advanced connectivity, and cloud and edge computing—declined, but that is likely due, at least in part, to their maturity. More mature technologies can be more sensitive to short-term budget dynamics than more nascent technologies with longer investment time horizons, such as climate and mobility technologies. Also, as some technologies become more profitable, they can often scale further with lower marginal investment. Given that these technologies have applications in most industries, we have little doubt that mainstream adoption will continue to grow.

Organizations shouldn’t focus too heavily on the trends that are garnering the most attention. By focusing on only the most hyped trends, they may miss out on the significant value potential of other technologies and hinder the chance for purposeful capability building. Instead, companies seeking longer-term growth should focus on a portfolio-oriented investment across the tech trends most important to their business. Technologies such as cloud and edge computing and the future of bioengineering have shown steady increases in innovation and continue to have expanded use cases across industries. In fact, more than 400 edge use cases across various industries have been identified, and edge computing is projected to win double-digit growth globally over the next five years. Additionally, nascent technologies, such as quantum, continue to evolve and show significant potential for value creation. Our updated analysis for 2023 shows that the four industries likely to see the earliest economic impact from quantum computing—automotive, chemicals, financial services, and life sciences—stand to potentially gain up to $1.3 trillion in value by 2035. By carefully assessing the evolving landscape and considering a balanced approach, businesses can capitalize on both established and emerging technologies to propel innovation and achieve sustainable growth.

Tech talent dynamics

We can’t overstate the importance of talent as a key source in developing a competitive edge. A lack of talent is a top issue constraining growth. There’s a wide gap between the demand for people with the skills needed to capture value from the tech trends and available talent: our survey of 3.5 million job postings in these tech trends found that many of the skills in greatest demand have less than half as many qualified practitioners per posting as the global average. Companies should be on top of the talent market, ready to respond to notable shifts and to deliver a strong value proposition to the technologists they hope to hire and retain. For instance, recent layoffs in the tech sector may present a silver lining for other industries that have struggled to win the attention of attractive candidates and retain senior tech talent. In addition, some of these technologies will accelerate the pace of workforce transformation. In the coming decade, 20 to 30 percent of the time that workers spend on the job could be transformed by automation technologies, leading to significant shifts in the skills required to be successful. And companies should continue to look at how they can adjust roles or upskill individuals to meet their tailored job requirements. Job postings in fields related to tech trends grew at a very healthy 15 percent between 2021 and 2022, even though global job postings overall decreased by 13 percent. Applied AI and next-generation software development together posted nearly one million jobs between 2018 and 2022. Next-generation software development saw the most significant growth in number of jobs (exhibit).

Image description:

Small multiples of 15 slope charts show the number of job postings in different fields related to tech trends from 2021 to 2022. Overall growth of all fields combined was about 400,000 jobs, with applied AI having the most job postings in 2022 and experiencing a 6% increase from 2021. Next-generation software development had the second-highest number of job postings in 2022 and had 29% growth from 2021. Other categories shown, from most job postings to least in 2022, are as follows: cloud and edge computing, trust architecture and digital identity, future of mobility, electrification and renewables, climate tech beyond electrification and renewables, advanced connectivity, immersive-reality technologies, industrializing machine learning, Web3, future of bioengineering, future of space technologies, generative AI, and quantum technologies.

End of image description.

This bright outlook for practitioners in most fields highlights the challenge facing employers who are struggling to find enough talent to keep up with their demands. The shortage of qualified talent has been a persistent limiting factor in the growth of many high-tech fields, including AI, quantum technologies, space technologies, and electrification and renewables. The talent crunch is particularly pronounced for trends such as cloud computing and industrializing machine learning, which are required across most industries. It’s also a major challenge in areas that employ highly specialized professionals, such as the future of mobility and quantum computing (see interactive).

Michael Chui is a McKinsey Global Institute partner in McKinsey’s Bay Area office, where Mena Issler is an associate partner, Roger Roberts  is a partner, and Lareina Yee  is a senior partner.

The authors wish to thank the following McKinsey colleagues for their contributions to this research: Bharat Bahl, Soumya Banerjee, Arjita Bhan, Tanmay Bhatnagar, Jim Boehm, Andreas Breiter, Tom Brennan, Ryan Brukardt, Kevin Buehler, Zina Cole, Santiago Comella-Dorda, Brian Constantine, Daniela Cuneo, Wendy Cyffka, Chris Daehnick, Ian De Bode, Andrea Del Miglio, Jonathan DePrizio, Ivan Dyakonov, Torgyn Erland, Robin Giesbrecht, Carlo Giovine, Liz Grennan, Ferry Grijpink, Harsh Gupta, Martin Harrysson, David Harvey, Kersten Heineke, Matt Higginson, Alharith Hussin, Tore Johnston, Philipp Kampshoff, Hamza Khan, Nayur Khan, Naomi Kim, Jesse Klempner, Kelly Kochanski, Matej Macak, Stephanie Madner, Aishwarya Mohapatra, Timo Möller, Matt Mrozek, Evan Nazareth, Peter Noteboom, Anna Orthofer, Katherine Ottenbreit, Eric Parsonnet, Mark Patel, Bruce Philp, Fabian Queder, Robin Riedel, Tanya Rodchenko, Lucy Shenton, Henning Soller, Naveen Srikakulam, Shivam Srivastava, Bhargs Srivathsan, Erika Stanzl, Brooke Stokes, Malin Strandell-Jansson, Daniel Wallance, Allen Weinberg, Olivia White, Martin Wrulich, Perez Yeptho, Matija Zesko, Felix Ziegler, and Delphine Zurkiya.

They also wish to thank the external members of the McKinsey Technology Council.

This interactive was designed, developed, and edited by McKinsey Global Publishing’s Nayomi Chibana, Victor Cuevas, Richard Johnson, Stephanie Jones, Stephen Landau, LaShon Malone, Kanika Punwani, Katie Shearer, Rick Tetzeli, Sneha Vats, and Jessica Wang.

Explore a career with us

Related articles.

McKinsey Technology Trends Outlook 2022

Value creation in the metaverse

Quantum computing funding remains strong, but talent gap raises concern

Artificial Intelligence in Education: Implications for Policymakers, Researchers, and Practitioners

• Original research
• Open access
• Published: 04 June 2024

Cite this article

You have full access to this open access article

• Dirk Ifenthaler   ORCID: orcid.org/0000-0002-2446-6548 1 , 2 ,
• Rwitajit Majumdar 3 ,
• Pierre Gorissen 4 ,
• Miriam Judge 5 ,
• Shitanshu Mishra 6 ,
• Juliana Raffaghelli 7 &
• Atsushi Shimada 8  

288 Accesses

Explore all metrics

One trending theme within research on learning and teaching is an emphasis on artificial intelligence (AI). While AI offers opportunities in the educational arena, blindly replacing human involvement is not the answer. Instead, current research suggests that the key lies in harnessing the strengths of both humans and AI to create a more effective and beneficial learning and teaching experience. Thus, the importance of ‘humans in the loop’ is becoming a central tenet of educational AI. As AI technology advances at breakneck speed, every area of society, including education, needs to engage with and explore the implications of this phenomenon. Therefore, this paper aims to assist in this process by examining the impact of AI on education from researchers’ and practitioners' perspectives. The authors conducted a Delphi study involving a survey administered to N  = 33 international professionals followed by in-depth face-to-face discussions with a panel of international researchers to identify key trends and challenges for deploying AI in education. The results indicate that the three most important and impactful trends were (1) privacy and ethical use of AI; (2) the importance of trustworthy algorithms; and (3) equity and fairness. Unsurprisingly, these were also identified as the three key challenges. Based on these findings, the paper outlines policy recommendations for AI in education and suggests a research agenda for closing identified research gaps.

Similar content being viewed by others

Ethics of AI in Education: Towards a Community-Wide Framework

Ethical principles for artificial intelligence in education

Uncovering Blind Spots in Education Ethics: Insights from a Systematic Literature Review on Artificial Intelligence in Education

Avoid common mistakes on your manuscript.

1 Introduction

Artificial intelligence (AI) is finding its way into people's everyday lives at breathtaking speed and with almost unlimited possibilities. Typical points of contact with AI include pattern, image and speech recognition, auto-completion or correction suggestions for digital search queries. Since the 1950s, AI has been recognised in computer science and interdisciplinary fields such as philosophy, cognitive science, neuroscience, and economics (Tegmark, 2018 ). AI refers to the attempt to develop machines that can do things that were previously only possible using human cognition (Zeide, 2019 ). In contrast to humans, however, AI systems can process much more data in real-time (De Laat et al., 2020 ).

AI in education represents a generic term to describe a wide collection of different technologies, algorithms, and related multimodal data applied in education's formal, non-formal, and informal contexts. It involves techniques such as data mining, machine learning, natural language processing, large language models (LLMs), generative models, and neural networks. The still-emerging field of AI in education has introduced new frameworks, methodological approaches, and empirical investigations into educational research; for example, novel methods in academic research include machine learning, network analyses, and empirical approaches based on computational modelling experiments (Bozkurt et al., 2021 ).

With the emerging opportunities of AI, learning and teaching may be supported in situ and in real-time for more efficient and valid solutions (Ifenthaler & Schumacher, 2023 ). Hence, AI has the potential to further revolutionise the integration of human and artificial intelligence and impact human and machine collaboration in learning and teaching (De Laat et al., 2020 ). The discourse around the utilization of AI in education shifted from being narrowly focused on automation-based tasks to the augmentation of human capabilities linked to learning and teaching (Chatti et al., 2020 ). Notably, the concept of ‘humans in the loop’ (U.S. Department of Education, 2023 ) has gained more traction in recent education discourse as concerns about ethics, risks, and equity emerge.

Due to the remaining challenges of implementing meaningful AI in educational contexts, especially for more sophisticated tasks, the reciprocal collaboration of humans and AI might be a suitable approach for enhancing the capacities of both (Baker, 2016 ). However, the importance of understanding how AI, as a stakeholder among humans, selects and acquires data in the process of learning and knowledge creation, learns to process and forget information, and shares knowledge with collaborators is yet to be empirically investigated (Al-Mahmood, 2020 ; Zawacki-Richter et al., 2019 ).

This paper is based on (a) a literature review focussing on the impact of AI in the context of education, (b) a Delphi study (Scheibe et al., 1975 ) involving N  = 33 international professionals and a focus discussion on current opportunities and challenges of AI as well as (c) outlining policy recommendations and (d) a research agenda for closing identified research gaps.

2 Background

2.1 artificial intelligence.

From a conceptual point of view, AI refers to the sequence and application of algorithms that enable specific commands to transform a data input into a data output. Following Graf Ballestrem et al. ( 2020 ), among several definitions related to AI (Sheikh et al., 2023 ), AI refers to a system that exhibits intelligent behaviour by analysing the environment and taking targeted measures to achieve specific goals using certain degrees of freedom. In this context, intelligent behaviour is associated with human cognition. The focus here is on human cognitive functions such as decision-making, problem-solving and learning (Bellman, 1978 ). AI is, therefore, a machine developed by humans that can achieve complex goals (partially) autonomously. By applying machine learning techniques, these machines can increasingly analyse the application environment and its context and adapt to changing conditions (De Laat et al., 2020 ).

Daugherty and Wilson ( 2018 ) analyse the interaction between humans and AI. They identified three fields of activity: (a) Human activities, such as leading teams, clarifying points of view, creating things, or assessing situations. The human activities remain an advantage for humans when compared to AI. (b) Activities performed by machines, such as carrying out processes and repeating them as required, forecasting target states, or adapting processes. The machine activities are regarded as an advantage when compared to humans. In between are the (c) human–machine alliances. In this alliance, people must develop, train, and manage AI systems—to empower them. In this alliance, machines extend the capabilities of humans to analyse large amounts of data from countless sources in (near) real time. In these alliances, humans and machines are not competitors. Instead, they become symbiotic partners that drive each other to higher performance levels. The paradigm shift from computers as tools to computers as partners is becoming increasingly differentiated in various fields of application (Wesche & Sonderegger, 2019 ), including in the context of education.

2.2 Artificial Intelligence in Education

Since the early 2010s, data and algorithms have been increasingly used in the context of higher education to support learning and teaching, for assessments, to develop curricula further, and to optimize university services (Pinkwart & Liu, 2020 ). A systematic review by Zawacki-Richter et al. ( 2019 ) identifies various fields of application for AI in the context of education: (a) modelling student data to make predictions about academic success, (b) intelligent tutoring systems that present learning artifacts or provide assistance and feedback, (c) adaptive systems that support learning processes and, if necessary, offer suggestions for learning support, and (d) automated examination systems for classifying learning achievements. In addition, (e) support functions are implemented in the area of pedagogical decisions by teachers (Arthars et al., 2019 ), and the (f) further development of course content and curricula (Ifenthaler, Gibson, et al., 2018 ).

However, there are only a few reliable empirical studies on the potential of AI in the context of education concerning its impact (Zawacki-Richter et al., 2019 ). System-wide implementations of the various AI application fields in the education context are also still pending (Gibson & Ifenthaler, 2020 ). According to analyses by Bates et al. ( 2020 ), AI remains a sleeping giant in the context of education. Despite the great attention paid to the topic of AI in educational organizations, the practical application of AI lags far behind the anticipated potential (Buckingham Shum & McKay, 2018 ). Deficits in organizational structures and a lack of personnel and technological equipment at educational organizations have been documented as reasons for this (Ifenthaler, 2017 ).

Despite its hesitant implementation, AI has far more potential to transform the education arena than any technology before it. Potentials for educational organizations made possible by AI include expanding access to education, increasing student success, improving student retention, lowering costs and reducing the duration of studies. The application of AI systems in the context of education can be categorized on various levels (Bates et al., 2020 ).

The first level is aimed at institutional processes. These include scalable applications for managing application and admission procedures (Adekitan & Noma-Osaghae, 2019 ) and AI-based support for student counselling and services (Jones, 2019 ). Another field of application is aimed at identifying at-risk students and preventing students from dropping out (Azcona et al., 2019 ; Hinkelmann & Jordine, 2019 ; Russell et al., 2020 ). For example, Hinkelmann and Jordine ( 2019 ) report an implementation of a machine learning algorithm to identify students-at-risk, based on their study behaviour. This information triggered a student counselling process, offering support for students toward meeting their study goals or understanding personal needs for continuing the study programme.

The second level aims to support learning and teaching processes. This includes the recommendation of relevant next learning steps and learning materials (Schumacher & Ifenthaler, 2021 ; Shimada et al., 2018 ), the automation of assessments and feedback (Ifenthaler, Grieff, et al., 2018 ), the promotion of reflection and awareness of the learning process (Schumacher & Ifenthaler, 2018 ), supporting social learning (Gašević et al., 2019 ), detecting undesirable learning behaviour and difficulties (Nespereira et al., 2015 ), identifying the current emotional state of learners (Taub et al., 2020 ), and predicting learning success (Glick et al., 2019 ). For instance, Schumacher and Ifenthaler ( 2021 ) successfully utilised different types of prompts related to their current learning process to support student self-regulation.

Furthermore, a third level, which encompasses learning about AI and related technologies, has also been identified (U.S. Department of Education, 2023 ). AI systems are also used for the quality assurance of curricula and the associated didactic arrangements (Ifenthaler, Gibson, et al., 2018 ) and to support teachers (Arthars et al., 2019 ). For example, Ifenthale, Gibson, et al. ( 2018 ) applied graph-network analysis to identify study patterns that supported re-designing learning tasks, materials, and assessments.

2.3 Ethics Related to Artificial Intelligence in Education

The tension between AI's potential and ethical principles in education was recognized early on (Slade & Prinsloo, 2013 ). Ifenthaler and Tracey ( 2016 ) continued the discourse on ethical issues, data protection, and privacy of data in the context of AI applications. The present conceptual and empirical contributions on ethics and AI in the context of education show that data protection and privacy rights are a central problem area in the implementation of AI (Li et al., 2023 ).

AI systems in the context of education are characterised by their autonomy, interactivity and adaptability. These properties enable effective management of the dynamic and often incompletely understood learning and teaching processes. However, AI systems with these characteristics are difficult to assess, and their predictions or recommendations can lead to unexpected behaviour or unwanted activities (i.e., black box). Richards and Dignum ( 2019 ) propose a value-centred design approach that considers ethical principles at every stage of developing and using AI systems for education. Following this approach, AI systems in the context of education must (a) identify relevant stakeholders; (b) identify stakeholders' values and requirements; (c) provide opportunities to aggregate the values and value interpretation of all stakeholders; (d) ensure linkage of values and system functionalities to support implementation decisions and sustainable use; (e) provide support in the selection of system components (from within or outside the organisation) against the background of ethical principles. Dignum ( 2017 ) integrates a multitude of ethical criteria into the so-called ART principles (Accountability, Responsibility, Transparency).

Education organisations must embrace the ART principles while implementing AI systems to ensure responsible, transparent and explainable use of AI systems. Initial study results indicate (Howell et al., 2018 ; Viberg et al., 2022 ; West, Heath, et al., 2016 ; West, Huijser, et al., 2016a , 2016b ) that students are not willing to disclose all data for AI applications despite anticipated benefits. Although a willingness to share learning-related data is signalled, personal information or social user paths are not. This remains a critical aspect, especially when implementing the many adaptive AI systems that rely on a large amount of data.

Future AI systems may take over decision-making responsibilities if they are integrated into education organisations' decision-making processes. For instance, this could happen if AI systems are used in automated examination or admissions processes (Prinsloo & Slade, 2014 ; Willis & Strunk, 2015 ; Willis et al., 2016 ). Education organisations and their stakeholders will, therefore, decide against the background of ethical principles whether this responsibility can be delegated to AI. At the same time, those involved in the respective education organisations must assess the extent to which AI systems can take responsibility (if any) for the decisions made.

2.4 Context and Research Questions

EDUsummIT is a UNESCO (United Nations Educational, Scientific and Cultural Organization; https://www.unesco.org ) endorsed global community of researchers, policy-makers, and practitioners committed to supporting the effective integration of Information Technology (IT) in education by promoting active dissemination and use of research. Approximately 90 leading researchers, policymakers, and practitioners from all continents and over 30 countries gathered in Kyoto, Japan, from 29 May to 01 June 2023, to discuss emerging themes and to define corresponding action items. Previous to the meeting, thematic working groups (TWGs) conducted research related to current challenges in educational technologies with a global impact. This paper is based on the work of the TWG, which focuses on ‘Artificial Intelligence for Learning and Teaching’. The authors of this article constituted the TWG.

The research questions addressed by the researchers of TWG ‘Artificial Intelligence for Learning and Teaching’ are as follows:

What recent research and innovations in artificial intelligence in education are linked to supporting learning, teaching, and educational decision-making?

What recommendations for artificial intelligence in education can be proposed for policy, practice, and research?

3 Delphi Study

This study aimed to uncover global trends and educational practices pertaining to AI in education. A panel of multinational specialists from industry and research institutions reached a consensus on a set of current trends using the Delphi method.

3.1 Methodology

The Delphi method is a robust approach for determining forecasts or policy positions considered to be the most essential (Scheibe et al., 1975 ). A Delphi study can be conducted using paper-and-pencil instruments, computer- or web-based approaches, as well as face-to-face communication processes. For this study, the researchers applied a mixed Delphi design, including (a) computer-based and (b) face-to-face discussion methods. In order to assure the reliability and validity of the current study, we closely followed the guidelines proposed by Beiderbeck et al. ( 2021 ), including the general phases of preparing, conducting, and analysing the Delphi study.

In the first phase, using the computer-based method, a panel of international researchers in artificial intelligence in education were invited to submit trends and institutional practices related to AI in the educational arena. The initial list consisted of N  = 70 trends. This initial list was then aggregated through agreement, eliminating duplicates and trends with similar meanings. Agreement on aggregated constructs was met through in-depth research debriefing and discussion among the involved researchers. The final consolidated list included N  = 20 topics of AI in education. In an additional step of the computer-based method, the list was disseminated to global specialists in AI in education. Each participant was asked to rate the 20 topics on the list concerning (1) importance, (2) impact, and (3) key challenges on a scale of 1–10 (with 10 being the highest). The instructions for the ratings were as follows:

Please rate the IMPORTANCE of each of the trends (on a scale of 10, where 10 is the highest IMPORTANCE) for learning and teaching related to AI in organizations within the next 3 years.

Please rate the IMPACT of each of the trends (on a scale of 10, where 10 is the highest IMPACT) on learning and teaching related to AI and how organizations will utilize them.

Please rate the KEY CHALLENGES of each of the trends in AI in education (on a scale of 10, where 10 is the highest CHALLENGE) that organizations will face within the next 3 years.

In preparation for the second phase, face-to-face discussion , the panel of international researchers were asked to provide three relevant scientific literature resources related to the identified key areas in the first phase and explain their contribution to the respective development area. Next, the panel of international researchers met face-to-face for a 3-day workshop. During the face-to-face meeting, the panel of international researchers and policymakers followed a discussion protocol made available before the meeting (Beiderbeck et al., 2021 ). Discussion questions included but were not limited to: (1) What new educational realities have you identified in AI in education so far? (2) What are recommendations for future educational realities in AI in education for practice, policy, and research? The panel of international researchers discussed and agreed on several trends, challenges, and recommendations concerning research gaps and important implications for educational stakeholders, including policymakers and practitioners.

3.2 Participants

The research team sent open invitations to recruit participants through relevant professional networks, conferences, and personal invitations. As a result, a convenience sample of N  = 33 participants (14 = female; 17 = male; 2 = undecided) with an average age of M  = 46.64 years ( SD  = 9.83) took part in the study. The global specialists were from research institutions ( n ri  = 26), industry ( n in  = 5), and government organizations ( n in  = 2). They had an average of M  = 17.8 years ( SD  = 9.4) of experience in research and development in educational technology and are currently focused on artificial intelligence. Participants were based in Argentina ( n  = 1), Australia ( n  = 3), Canada ( n  = 2), China ( n  = 1), Croatia ( n  = 1), Finland ( n  = 1), France ( n  = 1), Germany ( n  = 1), India ( n  = 1), Ireland ( n  = 3), Japan ( n  = 2), Philippines ( n  = 1), Spain ( n  = 2), Sweden ( n  = 1), The Netherlands ( n  = 6), UK ( n  = 4), and USA ( n  = 2).

3.3 Data Analysis

All data were saved and analysed using an anonymized process as per conventional research data protection procedures. Data were cleaned and combined for descriptive and inferential statistics using r Statistics ( https://www.r-project.org ). All effects were tested at the 0.05 significance level, and effect size measures were computed where relevant. Further, discussion protocols of the face-to-face discussion were transcribed and analysed using QCAmap, a software for qualitative content analysis (Mayring & Fenzl, 2022 ). Both inductive and deductive coding techniques were used (Mayring, 2015 ). Regular researcher debriefing was conducted during data analysis to enhance the reliability and validity of the quantitative and qualitative analysis. The deductive coding followed pre-established categories derived from theory and existing research findings as well as the initial list of trends (e.g., ethics and AI, diversity and inclusion). The inductive process included critical reflections on new realities that emerged since the project's initial phase (e.g., generative AI, LLMs).

4.1 Phase 1: Global Trends in Artificial Intelligence in Education

The first phase (i.e., computer-based method) resulted in a preliminary list of trends in AI in education. These trends were rated concerning importance (see Table  1 ), impact (see Table  2 ), and challenges (see Table  3 ).

As shown in Table  1 , the most important trends included (1) Privacy and ethical use of AI and big data in education ( M  = 8.7; SD  = 1.286), (2) Trustworthy algorithms for supporting education ( M  = 8.3; SD  = 1.608), and Fairness & equity of AI in education ( M  = 8.2; SD  = 1.674). Less important trends included (18) Generalization of AI models in education ( M  = 6.2; SD  = 2.018), (19) Intelligent and social robotics for education ( M  = 5.8; SD  = 2.335), and (20) Blockchain technology in education ( M  = 4.9; SD  = 2.482) (see Table  1 ).

Table 2 shows the most impactful trends, including (1) Privacy and ethical use of AI and big data in education ( M  = 8.2; SD  = 1.608), (2) Trustworthy algorithms for supporting education ( M  = 7.7; SD  = 2.268), and (3) Fairness & equity of AI in education ( M  = 7.7; SD  = 1.736). Less impactful trends included (18) Generalization of AI models in education ( M  = 6.4; SD  = 2.115), (19) Intelligent and social robotics for education ( M  = 5.5; SD  = 2.298), and (20) Blockchain technology in education ( M  = 5.0; SD  = 2.650) (see Table  2 ).

Challenges related to the trends in AI in education are presented in Table  3 . Key challenges included (1) Privacy and ethical use of AI and big data in education ( M  = 8.8; SD  = 1.455), (2) Trustworthy algorithms for supporting education ( M  = 8.3; SD  = 1.804), and (3) Fairness & equity of AI in education ( M  = 8.3; SD  = 1.855). Even the weakest challenges received ratings above the mean (18) Intelligent and social robotics for education ( M  = 7.0; SD  = 1.941), (19) Multimodal learning analytics in education ( M  = 6.9; SD  = 2.187), and (20) Blockchain technology in education ( M  = 6.6; SD  = 2.599) (see Table  3 ).

Overall, the challenges ( M  = 7.68, SD  = 0.315) of AI in education have been rated significantly higher than impact ( M  = 7.05, SD  = 0.593) and importance ( M  = 7.28, SD  = 0.829), F (2, 57) = 3.512, p  < 0.05, Eta2  = 0.110 (medium effect).

4.2 Phase 2: Consensus Related to Identified Areas of Artificial Intelligence in Education

For the second phase, the top three trends for importance, impact, and challenges of AI in education were critically reflected and linked with an in-depth and research-informed group discussion. However, all other trends have been recognized during the consensus phase and for developing recommendations toward strategies and actions. As shown in Table  4 , the panel of international researchers and policymakers agreed that (a) privacy and ethical use of AI and big data in education, (b) trustworthy algorithms for supporting education, and (c) fairness and equity of AI in education remain the key drivers of AI in education. Further, the panel of international researchers and policymakers identified emerging educational realities with AI, including (d) new roles of stakeholders in education, (e) human-AI-alliance in education, and (f) precautionary pre-emptive policies preceding practice for AI in education.

5 Discussion

This Delphi study included global specialists from research institutions, industry, and policymaking. The primary goal of the Delphi method is to structure a group discussion systematically. However, reaching a consensus in the discussion may also lead to a biased perspective on the research topic (Beiderbeck et al., 2021 ). Another limitation of the current study is the limited sample size. Hence, our convenience sample could have included more participants and further differentiated the various experience levels in AI in education. Hence, future studies may increase the empirical basis as well as the experience of participants related to AI in education. Further, a limitation may be seen in possible overlaps between the identified constructs during the Delphi study. However, through the in-depth face-to-face discussion of the panel of international researchers, the constructs were constantly monitored concerning their content validity and refined accordingly.

In summary, the highest-rated trends in AI in education regarding importance, impact, and challenges included privacy and ethical use of AI and big data in education, trustworthy algorithms for supporting education, and fairness and equity of AI in education. In addition, new roles of stakeholders in education, human-AI-alliance in education, and precautionary pre-emptive policies precede practice for AI in education have been identified as emerging realities of AI in education.

5.1 Trends Identified for AI in Education

Privacy and ethical use of AI and big data in education emphasise the importance of data privacy (data ownership, data access, and data protection) concerning the development, implementation, and use of AI systems in education. Inevitably, the handling of these data privacy issues has significant ethical implications for the stakeholders involved. For instance, Adejo and Connolly ( 2017 ) discuss ethical issues related to using learning analytics tools and technologies, focusing on privacy, accuracy, property, and accessibility concerns. Further, a survey study by Ifenthaler and Schumacher ( 2016 ) examined student perceptions of privacy principles in learning analytics systems. The findings show that students remained conservative in sharing personal data, and it was recommended that all stakeholders be involved in implementing learning analytics systems. Thus, the sustainable involvement of stakeholders increases trust and establishes transparency regarding the need for and use of data.

More recently, Celik ( 2023 ) focused on teachers' professional knowledge and ethical integration of AI-based tools in education and suggested that teachers with higher knowledge of interacting with AI tools have a better understanding of their pedagogical contributions. Accordingly, AI literacy among all stakeholders appears to be inevitable, including understanding AI capabilities, utilizing AI, and applying AI (Papamitsiou et al., 2021 ; Wang & Lester, 2023 ).

Trustworthy algorithms for supporting education focus on trustworthiness, which is defined as the security, reliability, validity, transparency, and accuracy of AI algorithms and the interpretability of the AI outputs used in education. It particularly focuses on the impact of algorithmic bias (systematic and repeated errors resulting in unfair outcomes) on different stakeholders and stages of algorithm development. Research has demonstrated that algorithmic bias is a problem for algorithms used in education (OECD, 2023 ). Bias, which can occur at all stages of the machine learning life cycle, is a multilayered phenomenon encompassing historical bias, representation bias, measurement bias, aggregation bias, evaluation bias and deployment bias (Suresh & Guttag, 2021 ). For instance, Baker and Hawn ( 2021 ) review algorithmic bias in education, discussing its causes and empirical evidence of its manifestation, focusing on the impacts of algorithmic bias on different groups and stages of algorithm development and deployment in education. Alexandron et al. ( 2019 ) raise concerns about reliability issues, identify the presence of fake learners who manipulate data, and demonstrate how their activity can bias analytics results. Li et al. ( 2023 ) also mention the inhibition of predictive fairness due to data bias in their systematic review of existing research on prediction bias in education. Minn et al. ( 2022 ) argue that it is challenging to extract psychologically meaningful explanations that are relevant to cognitive theory from large-scale models such as Deep Knowledge Tracing (DKT) and Dynamic Key-Value Memory Network (DKVMN), which have useful performance in knowledge tracking, and mention the necessity for simpler models to improve interpretability. On the contrary, such simplifications may result in limited validity and accuracy of the underlying models.

Fairness and equity of AI in education emphasises the need for explainability and accountability in the design of AI in education. It requires lawful, ethical, and robust AI systems to address technical and social perspectives. Current research related to the three trends overlaps and emphasises the importance of considering stakeholder involvement, professional knowledge, ethical guidelines, as well as the impact on learners, teachers, and organizations. For instance, Webb et al. ( 2021 ) conducted a comprehensive review of machine learning in education, highlighting the need for explainability and accountability in machine learning system design. They emphasised the importance of integrating ethical considerations into school curricula and providing recommendations for various stakeholders. Further, Bogina et al. ( 2021 ) focused on educating stakeholders about algorithmic fairness, accountability, transparency, and ethics in AI systems. They highlight the need for educational resources to address fairness concerns and provide recommendations for educational initiatives.

New roles of stakeholders in education is related to the phenomena that AI will be omnipresent in education, which inevitably involves stakeholders interacting with AI systems in an educational context. New roles and profiles are emerging beyond traditional ones. For instance, Buckingham Shum ( 2023 ) emphasises the need for enterprise-wide deployment of AI in education, which is accompanied by extensive staff training and support. Further, new forms of imagining AI and of deciding its integration into socio-cultural systems will have to be discussed by all stakeholders, particularly minority or excluded collectives. Hence, AI deployment reflects different levels of influence, partnership and adaptation that are required to introduce and sustain novel technologies in the complex system that constitutes an educational organisation. Further, Andrews et al. ( 2022 ) recommend appointing a Digital Ethics Officer (DEO) in educational organisations who would be responsible for overseeing ethical guidelines, controlling AI activities, ethics training, as well as creating an ethical awareness culture and advising management.

Human-AI-alliance in education emphasises that AI in education shifted from being narrowly focused on automation-based tasks to augmenting human capabilities linked to learning and teaching. Seeber et al. ( 2020 ) propose a research agenda to develop interrelated programs to explore the philosophical and pragmatic implications of integrating humans and AI in augmenting human collaboration. Similarly, De Laat et al. ( 2020 ) and Joksimovic et al. ( 2023 ) highlight the challenge of bringing human and artificial intelligence together so that learning in situ and in real-time will be supported. Multiple opportunities and challenges arise from the human-AI-alliances in education for educators, learners, and researchers. For instance, Kasneci et al. ( 2023 ) suggest educational content creation, improving student engagement and interaction, as well as personalized learning and teaching experiences.

Precautionary pre-emptive policies precede practice for AI in education, underlining that, overwhelmed by the rapid change in the technology landscape, decision-makers tend to introduce restrictive policies in reaction to initial societal concerns with emerging AI developments. Jimerson and Childs ( 2017 ) highlight the issue of educational data use and how state and local policies fail to align with the broader evidence base of educational organisations. As a reaction toward uninformed actions in educational organisations, Tsai et al. ( 2018 ) introduced a policy and strategy framework that may support large-scale implementation involving multi-stakeholder engagement and approaches toward needs analysis. This framework suggests various dimensions, including mapping the political context, identifying the key stakeholders, identifying the desired behaviour changes, developing an engagement strategy, analysing the capacity to effect change, and establishing monitoring and learning opportunities.

5.2 Strategies and Actions

Based on the findings of the Delphi study as well as current work by other researchers, we recommend the following actions for policymakers (PM), researchers (RE), and practitioners (PR), each strategy linked to the corresponding challenges identified above. A detailed implementation plan for the strategies and related stakeholders can be found in a related paper published during EDUsummIT ( https://www.let.media.kyoto-u.ac.jp/edusummit2022/ ):

In order to support the new roles of stakeholders in education

Identify the elements involved in the new roles (RE)

Identify and implement pedagogical practices for AI in education (PR, RE)

Develop policies to support AI and data literacies through curriculum development (PM)

In order to support the Human-AI-Alliance in education

Encourage and support collaborative interaction between stakeholders and AI systems in education (RE)

Take control of available AI systems and optimize teaching and learning strategies (PR)

Promote institutional strategies and actions in order to support teachers’ agency and avoid teachers’ de-professionalization (PM, PR)

In order to support evidence-informed practices of AI in education

Use both the results of fundamental research into AI and the results of live case studies to build a robust body of knowledge and evidence about AI in education (RE)

Support open science and research on AI in education (PM)

Implement evidence-informed development of AI applications (RE, PR)

Implement evidence-informed pedagogical practices (PR, RE)

In order to support ethical considerations of AI in education

Forefront privacy and ethical considerations utilizing a multi-perspective and interdisciplinary approach as the core of AI in education (PM, RE, PR)

Consider the context, situatedness, and complexity of AI in education’s impacts at the time of exploring ethical implications (PR)

Continuously study the effects of AI systems in the context of education (RE)

6 Conclusion

The evolution of Artificial Intelligence (AI) in education has witnessed a profound transformation over recent years, holding tremendous promise for the future of learning (Bozkurt et al., 2021 ). As we stand at the convergence of technology and education, the potential impact of AI is poised to reshape traditional educational paradigms in multifaceted ways. Through supporting personalised learning experiences, AI has showcased its ability to cater to individual student needs, offering tailored curricula and adaptive assessments (Brusilovsky, 1996 ; Hemmler & Ifenthaler, 2022 ; Jones & Winne, 1992 ; Martin et al., 2020 ). This customisability of education fosters a more inclusive and effective learning environment, accommodating diverse learning needs and regulations. Moreover, AI tools augment the role of educators by automating administrative tasks, enabling them to allocate more time to mentoring, fostering creativity, and critical thinking (Ames et al., 2021 ). However, the proliferation of AI in education also raises pertinent ethical concerns, including data privacy, algorithmic biases, and the digital divide (Baker & Hawn, 2021 ; Ifenthaler, 2023 ). Addressing these concerns requires a conscientious approach, emphasising transparency, equity, and responsible AI development and deployment. In addition, in recent years, the emergence of generative AI, such as ChatGPT, is expected to facilitate interactive learning and assist instructors, while concerns such as the generation of incorrect information and privacy issues are also being addressed (Baidoo-Anu & Owusu Ansah, 2023 ; Lo, 2023 ).

Looking forward, the future of AI in education holds tremendous potential for transformation of learning and teaching. Yet, realising the full potential of AI in education necessitates concerted efforts from stakeholders—educators, policymakers, technologists, and researchers—to collaborate, innovate, and navigate the evolving ethical and pedagogical considerations. Embracing AI's potential while safeguarding against its pitfalls will be crucial in harnessing its power to create a more equitable, accessible, and effective educational arena.

Data availability

The data that support the findings of this study are available from the authors upon reasonable request.

Adejo, O., & Connolly, T. (2017). Learning analytics in a shared-network educational environment: ethical issues and countermeasures. International Journal of Advanced Computer Science and Applications , 8 (4). https://doi.org/10.14569/IJACSA.2017.080404

Adekitan, A. I., & Noma-Osaghae, E. (2019). Data mining approach to predicting the performance of first year student in a university using the admission requirements. Education and Information Technologies, 24 , 1527–1543. https://doi.org/10.1007/s10639-018-9839-7

Article   Google Scholar  

Alexandron, G., Yoo, L., Ruipérez-Valiente, J. A., Lee, S., & Pritchard, D. (2019). Are MOOC learning analytics results trustworthy? With fake learners, they might not be! International Journal of Artificial Intelligence in Education, 29 , 484–506. https://doi.org/10.1007/s40593-019-00183-1

Al-Mahmood, R. (2020). The politics of learning analytics. In D. Ifenthaler & D. C. Gibson (Eds.), Adoption of data analytics in higher education learning and teaching (pp. 20–38). Springer.

Google Scholar  

Ames, K., Harris, L. R., Dargusch, J., & Bloomfield, C. (2021). ‘So you can make it fast or make it up’: K–12 teachers’ perspectives on technology’s affordances and constraints when supporting distance education learning. The Australian Educational Researcher, 48 , 359–376. https://doi.org/10.1007/s13384-020-00395-8

Andrews, D., Leitner, P., Schön, S., & Ebner, M. (2022). Developing a professional profile of a digital ethics officer in an educational technology unit in higher education. In P. Zaphiris & A. Ioannou (Eds.), Learning and collaboration technologies. Designing the learner and teacher experience. HCII 2022. Lecture notes in computer science (Vol. 13328, pp. 157–175). Springer. https://doi.org/10.1007/978-3-031-05657-4_12

Arthars, N., Dollinger, M., Vigentini, L., Liu, D. Y., Kondo, E., & King, D. M. (2019). Empowering teachers to personalize learning support. In D. Ifenthaler, D.-K. Mah, & J. Y.-K. Yau (Eds.), Utilizing learning analytics to support study success (pp. 223–248). Springer. https://doi.org/10.1007/978-3-319-64792-0_13

Azcona, D., Hsiao, I., & Smeaton, A. F. (2019). Detecting students-at-risk in computer programming classes with learning analytics from students’ digital footprints. User Modeling and User-Adapted Interaction, 29 , 759–788. https://doi.org/10.1007/s11257-019-09234-7

Baidoo-Anu, D., & Owusu Ansah, L. (2023). Education in the era of generative artificial intelligence (AI): understanding the potential benefits of chatgpt in promoting teaching and learning. Journal of AI , 7 (1), 52–62. https://doi.org/10.61969/jai.1337500

Baker, R. S. (2016). Stupid tutoring systems, intelligent humans. International Journal of Artificial Intelligence in Education, 26 , 60–6140. https://doi.org/10.1007/s40593-016-0105-0

Baker, R. S., & Hawn, A. (2021). Algorithmic bias in education. International Journal of Artificial Intelligence in Education, 32 (4), 1052–1092. https://doi.org/10.1007/s40593-021-00285-9

Bates, T., Cobo, C., Mariño, O., & Wheeler, S. (2020). Can artificial intelligence transform higher education? International Journal of Educational Technology in Higher Education, 17 (42), 1–12. https://doi.org/10.1186/s41239-020-00218-x

Beiderbeck, D., Frevel, N., von der Gracht, H. A., Schmidt, S. L., & Schweitzer, V. M. (2021). Preparing, conducting, and analyzing Delphi surveys: Cross-disciplinary practices, new directions, and advancements. MethodsX, 8 , 101401. https://doi.org/10.1016/j.mex.2021.101401

Bellman, R. (1978). An introduction to artificial intelligence: can computers think? . Boyd & Fraser.

Bogina, V., Hartman, A., Kuflik, T., & Shulner-Tal, A. (2021). Educating software and AI stakeholders about algorithmic fairness, accountability, transparency and ethics. International Journal of Artificial Intelligence in Education . https://doi.org/10.1007/s40593-021-00248-0

Bozkurt, A., Karadeniz, A., Bañeres, D., Guerrero-Roldán, A., & Rodríguez, M. E. (2021). Artificial intelligence and reflections from educational landscape: A review of ai studies in half a century. Sustainability, 13 , 800. https://doi.org/10.3390/su13020800

Brusilovsky, P. (1996). Methods and techniques of adaptive hypermedia. User Modeling and User-Adapted Interaction, 6 (2–3), 87–129. https://doi.org/10.1007/BF00143964

Buckingham Shum, S., & McKay, T. A. (2018). Architecting for learning analytics. Innovating for sustainable impact. EDUCAUSE Review , 53 (2), 25–37. https://er.educause.edu/articles/2018/3/architecting-for-learning-analytics-innovating-for-sustainable-impact

Buckingham Shum, S. (2023). Embedding learning analytics in a university: Boardroom, staff room, server room, classroom. In O. Viberg & Å. Grönlund (Eds.), Practicable learning analytics (pp. 17–33). Springer. https://doi.org/10.1007/978-3-031-27646-0_2

Celik, I. (2023). Towards Intelligent-TPACK: An empirical study on teachers’ professional knowledge to ethically integrate artificial intelligence (AI)-based tools into education. Computers in Human Behavior, 138 , 107468. https://doi.org/10.1016/j.chb.2022.107468

Chatti, M. A., Muslim, A., Guesmi, M., Richtscheid, F., Nasimi, D., Shahin, A., & Damera, R. (2020). How to design effective learning analytics indicators? a human-centered design approach. In C. Alario-Hoyos, M. J. Rodríguez-Triana, M. Scheffel, I. Arnedillo-Sánchez, & S. M. Dennerlein (Eds.), Addressing global challenges and quality education. EC-TEL 2020 (Vol. 12315, pp. 303–317). Springer. https://doi.org/10.1007/978-3-030-57717-9_22

Daugherty, P. R., & Wilson, H. J. (2018). Human + machine: Reimagining work in the age of AI . Harvard Business Review Press.

De Laat, M., Joksimovic, S., & Ifenthaler, D. (2020). Artificial intelligence, real-time feedback and workplace learning analytics to support in situ complex problem-solving: A commentary. International Journal of Information and Learning Technology, 37 (5), 267–277. https://doi.org/10.1108/IJILT-03-2020-0026

Dignum, V. (2017). Responsible autonomy. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence , Melbourne, VIC, AUS.

Gašević, D., Joksimović, S., Eagan, B. R., & Shaffer, D. W. (2019). SENS: Network analytics to combine social and cognitive perspectives of collaborative learning. Computers in Human Behavior, 92 , 562–577. https://doi.org/10.1016/j.chb.2018.07.003

Gibson, D. C., & Ifenthaler, D. (2020). Adoption of learning analytics. In D. Ifenthaler & D. C. Gibson (Eds.), Adoption of data analytics in higher education learning and teaching (pp. 3–20). Springer. https://doi.org/10.1007/978-3-030-47392-1_1

Glick, D., Cohen, A., Festinger, E., Xu, D., Li, Q., & Warschauer, M. (2019). Predicting success, preventing failure. In D. Ifenthaler, D.-K. Mah, & J. Y.-K. Yau (Eds.), Utilizing learning analytics to support study success (pp. 249–273). Springer. https://doi.org/10.1007/978-3-319-64792-0_14

Graf Ballestrem, J., Bär, U., Gausling, T., Hack, S., & von Oelffen, S. (2020). Künstliche Intelligenz. Rechtsgrundlagen und Strategien in der Praxis . Springer Gabler.

Hemmler, Y., & Ifenthaler, D. (2022). Four perspectives on personalized and adaptive learning environments for workplace learning. In D. Ifenthaler & S. Seufert (Eds.), Artificial intelligence education in the context of work (pp. 27–39). Springer. https://doi.org/10.1007/978-3-031-14489-9_2

Hinkelmann, M., & Jordine, T. (2019). The LAPS project: Using machine learning techniques for early student support. In D. Ifenthaler, J.Y.-K. Yau, & D.-K. Mah (Eds.), Utilizing learning analytics to support study success (pp. 105–117). Springer.

Chapter   Google Scholar  

Howell, J. A., Roberts, L. D., Seaman, K., & Gibson, D. C. (2018). Are we on our way to becoming a “helicopter university”? Academics’ views on learning analytics. Technology, Knowledge and Learning, 23 (1), 1–20. https://doi.org/10.1007/s10758-017-9329-9

Ifenthaler, D. (2017). Are higher education institutions prepared for learning analytics? TechTrends, 61 (4), 366–371. https://doi.org/10.1007/s11528-016-0154-0

Ifenthaler, D., & Schumacher, C. (2016). Student perceptions of privacy principles for learning analytics. Educational Technology Research and Development, 64 (5), 923–938. https://doi.org/10.1007/s11423-016-9477-y

Ifenthaler, D., & Schumacher, C. (2023). Reciprocal issues of artificial and human intelligence in education. Journal of Research on Technology in Education, 55 (1), 1–6. https://doi.org/10.1080/15391523.2022.2154511

Ifenthaler, D., & Tracey, M. W. (2016). Exploring the relationship of ethics and privacy in learning analytics and design: Implications for the field of educational technology. Educational Technology Research and Development, 64 (5), 877–880. https://doi.org/10.1007/s11423-016-9480-3

Ifenthaler, D., Gibson, D. C., & Dobozy, E. (2018). Informing learning design through analytics: Applying network graph analysis. Australasian Journal of Educational Technology , 34 (2), 117–132. https://doi.org/10.14742/ajet.3767

Ifenthaler, D., Greiff, S., & Gibson, D. C. (2018). Making use of data for assessments: harnessing analytics and data science. In J. Voogt, G. Knezek, R. Christensen, & K.-W. Lai (Eds.), International handbook of IT in primary and secondary education (2 ed., pp. 649–663). Springer. https://doi.org/10.1007/978-3-319-71054-9_41

Ifenthaler, D. (2023). Ethische Perspektiven auf künstliche Intelligenz im Kontext der Hochschule. In T. Schmohl, A. Watanabe, & K. Schelling (Eds.), Künstliche Intelligenz in der Hochschulbildung. hancen und Grenzen des KI-gestützten Lernens und Lehrens (pp. 71–86). Transcript-Verlag. https://doi.org/10.14361/9783839457696

Jimerson, J. B., & Childs, J. (2017). Signal and symbol: How state and local policies address data-informed practice. Educational Policy, 31 (5), 584–614. https://doi.org/10.1177/089590481561344

Joksimovic, S., Ifenthaler, D., De Laat, M., Siemens, G., & Marronne, R. (2023). Opportunities of artificial intelligence for supporting complex problem-solving: Findings from a scoping review. Computers & Education: Artificial Intelligence, 4 , 100138. https://doi.org/10.1016/j.caeai.2023.100138

Jones, M., & Winne, P. H. (Eds.). (1992). Adaptive learning environments . Springer.

Jones, K. M. L. (2019). Advising the whole student: EAdvising analytics and the contextual suppression of advisor values. Education and Information Technologies, 24 , 437–458. https://doi.org/10.1007/s10639-018-9781-8

Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., Stadler, M., Weller, J., Kuhn, J., & Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences , 103 , 102274. https://doi.org/10.1016/j.lindif.2023.102274

Li, F., Ruijs, R., & Lu, Y. (2023). Ethics & AI: a systematic review on ethical concerns and related strategies for designing with ai in healthcare. AI , 4 (1), 28–53. https://doi.org/10.3390/ai4010003

Lo, C. K. (2023). What Is the impact of ChatGPT on education? A rapid review of the literature. Education Sciences, 13 (4), 410. https://doi.org/10.3390/educsci13040410

Martin, F., Chen, Y., Moore, R. L., & Westine, C. D. (2020). Systematic review of adaptive learning research designs, context, strategies, and technologies from 2009 to 2018. Educational Technology Research and Development, 68 , 1903–1929. https://doi.org/10.1007/s11423-020-09793-2

Mayring, P. (2015). Qualitative content analysis: Theoretical background and procedures. In A. Bikner-Ahsbahs, C. Knipping, & N. Presmeg (Eds.), Approaches to qualitative research in mathematics education (pp. 365–380). Springer.

Mayring, P., & Fenzl, T. (2022). QCAmap: A software for qualitative content analysis [Computer software]. https://www.qcamap.org/ui/en/home

Minn, S., Vie, J.-J., Takeuchi, K., Kashima, H., & Zhu, F. (2022). Interpretable knowledge tracing: Simple and efficient student modeling with causal relations. Proceedings of the AAAI Conference on Artificial Intelligence, 36 (11), 12810–12818. https://doi.org/10.1609/aaai.v36i11.21560

Nespereira, C., Vilas, A., & Redondo, R. (2015). Am I failing this course?: risk prediction using e-learning data Conference on Technological Ecosystems for enhancing Multiculturality,

OECD. (2023). OECD digital education outlook 2023: Towards an effective digital education ecosystem. OECD Publishing . https://doi.org/10.1787/c74f03de-en

Papamitsiou, Z., Filippakis, M., Poulou, M., Sampson, D. G., Ifenthaler, D., & Giannakos, M. (2021). Towards an educational data literacy framework: Enhancing the profiles of instructional designers and e-tutors of online and blended courses with new competences. Smart Learning Environments, 8 , 18. https://doi.org/10.1186/s40561-021-00163-w

Pinkwart, N., & Liu, S. (Eds.). (2020). Artificial intelligence supported educational technologies . Springer.

Prinsloo, P., & Slade, S. (2014). Student data privacy and institutional accountability in an age of surveillance. In M. E. Menon, D. G. Terkla, & P. Gibbs (Eds.), Using data to improve higher education. Research, policy and practice (pp. 197–214). Sense Publishers.

Richards, D., & Dignum, V. (2019). Supporting and challenging learners through pedagogical agents: Addressing ethical issues through designing for values. British Journal of Educational Technology, 50 (6), 2885–2901. https://doi.org/10.1111/bjet.12863

Russell, J.-E., Smith, A., & Larsen, R. (2020). Elements of Success: Supporting at-risk student resilience through learning analytics. Computers & Education , 152 . https://doi.org/10.1016/j.compedu.2020.103890

Scheibe, M., Skutsch, M., & Schofer, J. (1975). Experiments in Delphi methodology. In H. A. Linestone & M. Turoff (Eds.), The Delphi method - techniques and applications (pp. 262–287). Addison-Wesley.

Schumacher, C., & Ifenthaler, D. (2018). The importance of students’ motivational dispositions for designing learning analytics. Journal of Computing in Higher Education, 30 (3), 599–619. https://doi.org/10.1007/s12528-018-9188-y

Schumacher, C., & Ifenthaler, D. (2021). Investigating prompts for supporting students’ self-regulation—A remaining challenge for learning analytics approaches? The Internet and Higher Education, 49 , 100791. https://doi.org/10.1016/j.iheduc.2020.100791

Seeber, I., Bittner, E., Briggs, R. O., Vreede, T., de Vreede, G.-J., de Elkins, A., Maier, R., Merz, A. B., Oeste-Reiß, S., Randrup, N., Schwabe, G., & Söllner, M. (2020). Machines as teammates: A research agenda on AI in team collaboration. Information & Management, 57 (2), 103174. https://doi.org/10.1016/j.im.2019.103174

Sheikh, H., Prins, C., & Schrijvers, E. (2023). Artificial intelligence: Definition and background. In H. Sheikh, C. Prins, & E. Schrijvers (Eds.), Mission AI. Research for policy (pp. 15–41). Springer. https://doi.org/10.1007/978-3-031-21448-6_2

Shimada, A., Okubo, F., Yin, C., & Ogata, H. (2018). Automatic summarization of lecture slides for enhanced student preview-technical report and user study. IEEE Transaction of Learning Technologies, 11 (2), 165–178. https://doi.org/10.1109/TLT.2017.2682086

Slade, S., & Prinsloo, P. (2013). Learning analytics: Ethical issues and dilemmas. American Behavioral Scientist, 57 (10), 1510–1529. https://doi.org/10.1177/0002764213479366

Suresh, H., & Guttag, J. (2021). A framework for understanding sources of harm throughout the machine learning life cycle. In EAAMO '21: Proceedings of the 1st ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization, Article 17 (pp. 1–9). ACM. https://doi.org/10.1145/3465416.3483305

Taub, M., Azevedo, R., Rajendran, R., Cloude, E. B., Biswas, G., & Price, M. J. (2020). How are students’ emotions related to the accuracy of cognitive and metacognitive processes during learning with an intelligent tutoring system? Learning and Instruction . https://doi.org/10.1016/j.learninstruc.2019.04.001

Tegmark, M. (2018). Life 3.0: Being human in the age of artificial intelligence . Penguin Books.

Tsai, Y.-S., Moreno-Marcos, P. M., Jivet, I., Scheffel, M., Tammets, K., Kollom, K., & Gašević, D. (2018). The SHEILA framework: informing institutional strategies and policy processes of learning analytics. Journal of Learning Analytics , 5 (3), 5–20. https://doi.org/10.18608/jla.2018.53.2

U.S. Department of Education. (2023). Artificial intelligence and future of teaching and learning: insights and recommendations . https://tech.ed.gov

Viberg, O., Engström, L., Saqr, M., & Hrastinski, S. (2022). Exploring students’ expectations of learning analytics: A person-centered approach. Education and Information Technologies, 27 , 8561–8581. https://doi.org/10.1007/s10639-022-10980-2

Wang, N., & Lester, J. (2023). K-12 education in the age of AI: A call to action for K-12 AI literacy. International Journal of Artificial Intelligence in Education, 33 , 228–232. https://doi.org/10.1007/s40593-023-00358-x

Webb, M., Fluck, A., Magenheim, J., Malyn-Smith, J., Waters, J., Deschênes, M., & Zagami, J. (2021). Machine learning for human learners: Opportunities, issues, tensions and threats. Educational Technology Research & Development, 69 (4), 2109–2130. https://doi.org/10.1007/s11423-020-09858-2

Wesche, J. S., & Sonderegger, A. (2019). When computers take the lead: The automation of leadership. Computers in Human Behavior, 101 , 197–209. https://doi.org/10.1016/j.chb.2019.07.027

West, D., Huijser, H., & Heath, D. (2016b). Putting an ethical lens on learning analytics. Educational Technology Research and Development, 64 (5), 903–922. https://doi.org/10.1007/s11423-016-9464-3

West, D., Heath, D., & Huijser, H. (2016). Let’s talk learning analytics: A framework for implementation in relation to student retention. Online Learning , 20 (2), 1–21. https://doi.org/10.24059/olj.v20i2.792

Willis, I. J. E., & Strunk, V. A. (2015). Ethical responsibilities of preserving academecians in an age of mechanized learning: Balancing the demands of educating at capacity and preserving human interactivity. In J. White & R. Searle (Eds.), Rethinking machine ethics in the age of ubiquitous technology (pp. 166–195). IGI Global.

Willis, I. J. E., Slade, S., & Prinsloo, P. (2016). Ethical oversight of student data in learning analytics: A typology derived from a cross-continental, cross-institutional perspective. Educational Technology Research and Development, 64 (5), 881–901. https://doi.org/10.1007/s11423-016-9463-4

Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education—where are the educators? International Journal of Educational Technology in Higher Education, 16 (39), 1–27. https://doi.org/10.1186/s41239-019-0171-0

Zeide, E. (2019). Artificial intelligence in higher education: Applications, promise and perils, and ethical questions. EDUCAUSE Review, 54 (3), 21–39.

Download references

Open Access funding enabled and organized by Projekt DEAL. The authors declare that no funds, grants, or other support were received during the preparation of this manuscript.

Author information

Authors and affiliations.

University of Mannheim L4, 1, 68131, Mannheim, Germany

Dirk Ifenthaler

Curtin University, Perth, Australia

Kumamoto University, Kumanoto, Japan

Rwitajit Majumdar

AN University of Applied Sciences, Arnhem, The Netherlands

Pierre Gorissen

Dublin City University, Dublin, Ireland

Miriam Judge

UNESCO MGEIP, New Delhi, India

Shitanshu Mishra

University of Padua, Padua, Italy

Juliana Raffaghelli

Kyushu University, Fukuoka, Japan

Atsushi Shimada

You can also search for this author in PubMed   Google Scholar

Contributions

All authors contributed to the study conception, design, data collection, and analysis, as well as draft writing and commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Dirk Ifenthaler .

Ethics declarations

Conflict of interests.

The authors have no relevant financial or non-financial interests to disclose.

Ethical Approval

All procedures performed in studies involving human participants were under the ethical standards of the institutional and national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. This article does not contain any studies with animals performed by any of the authors.

Informed Consent

Informed consent was obtained from all individual participants included in the study. Additional informed consent was obtained from all individual participants for whom identifying information is included in this article.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Ifenthaler, D., Majumdar, R., Gorissen, P. et al. Artificial Intelligence in Education: Implications for Policymakers, Researchers, and Practitioners. Tech Know Learn (2024). https://doi.org/10.1007/s10758-024-09747-0

Download citation

Accepted : 21 May 2024

Published : 04 June 2024

DOI : https://doi.org/10.1007/s10758-024-09747-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Artificial intelligence
• Adaptive learning
• Data protection
• Policy recommendation
• Algorithmic bias
• Stakeholders
• Human-AI-Alliance
• Delphi study
• Find a journal
• Publish with us
• Track your research

• En Español

• Medical Devices
• Radiation-Emitting Products
• Vaccines, Blood & Biologics
• Animal & Veterinary
• Tobacco Products

CFR - Code of Federal Regulations Title 21

This information is current as of Dec 22, 2023 .

This online reference for CFR Title 21 is updated once a year. For the most up-to-date version of CFR Title 21, go to the Electronic Code of Federal Regulations (eCFR).

Learn More...

• Medical Device Reports (MAUDE)
• CDRH Export Certificate Validation (CECV)
• CDRH FOIA Electronic Reading Room
• Device Classification
• FDA Guidance Documents
• Humanitarian Device Exemption
• Medsun Reports
• Premarket Approvals (PMAs)
• Post-Approval Studies
• Postmarket Surveillance Studies
• Radiation-Emitting Electronic Products Corrective Actions
• Registration & Listing
• Total Product Life Cycle
• X-Ray Assembler

This website has been translated to Spanish from English, and is updated often. It is possible that some links will connect you to content only available in English or some of the words on the page will appear in English until translation has been completed (usually within 24 hours). We appreciate your patience with the translation process. In the case of any discrepancy in meaning, the English version is considered official. Thank you for visiting esp.fda.gov/tabaco.