databases for medical literature review

Systematic Reviews: Medical Literature Databases to search

• Types of literature review, methods, & resources
• Protocol and registration
• Search strategy
• Medical Literature Databases to search
• Study selection and appraisal
• Data Extraction/Coding/Study characteristics/Results
• Reporting the quality/risk of bias
• Manage citations using RefWorks This link opens in a new window
• GW Box file storage for PDF's This link opens in a new window

How to document your literature search

You should always  document how you have searched each database, what keywords or index terms were used, the date on which the search was performed, how many results you retrieved, and if you use RefWorks to deduplicate results record how many were removed as duplicates and the final number of discrete studies you subjected to your first sift through of study selection.  Here is an example of how to document a literature search on an Excel spreadsheet , this example records a search of the hematology literature for articles about sickle cell disease. Here is another example of  how to document a literature search, this time on one page of a Word document , this example records a search of the medical literature for a poster on Emergency Department throughput.  The numbers recorded can then be used to populate the PRISMA flow diagram summarizing the literature search.

In the final report add as an appendix the full electronic search strategy for each database searched for the literature review e.g. MEDLINE with MeSH terms, keywords & limits

In the final report in the methods section:

PRISMA checklist Item 7 information sources will be reported as:

• What databases/websites you searched, the name of the database search platform and the start/end dates the index covers if relevant e.g. OVID MEDLINE (1950-present, or just PubMed
• Who developed & conducted the searches
• Date each database/website was last searched
• Supplementary sources - what other websites did you search? What journal titles were hand searched, whether reference lists were checked, what trial registries or regulatory agency websites were searched, were manufacturers or other authors contacted to obtain unpublished or missing information on study methods or results.

PRISMA checklist Item 8 search will be reported as:

• In text: describe the principal keywords used to search databases, websites & trials registers

What databases/indexes should you search?

At a minimum you need to search MEDLINE ,  EMBASE , and the  Cochrane CENTRAL  trials register .  This is the recommendation of three medical and public health research organizations: the U.S.  Agency for Healthcare Research and Quality ( AHRQ ), the U.K. Centre for Reviews and Dissemination ( CRD ), and the International Cochrane Collaboration (Source:  Institute of Medicine (2011) Finding What Works in Healthcare: Standards for Systematic Reviews  Table E-1, page 267).  Some databases have an alternate version, linked in parentheses below, that search the same records sets, ie the content of MEDLINE is in PubMed and Scopus, while the content of EMBASE is in Scopus. You should reformat your search for each database as appropriate, contact your librarian if you want help on how to search each database.  

Begin by searching:

1.        MEDLINE  (or  PubMed )

2.       EMBASE (or  Scopus )  Please note Himmelfarb Library does not have a subscription to EMBASE. The content is in the Scopus  database that you can search using keywords, but it is not possible to perform an EMTREE theasaurus search in Scopus.

3.        Cochrane Central Trials Register  (or  Cochrane Library ). In addition Cochrane researchers recommend you search the clinicaltrials.gov and ICTRP clinical trial registries due to the low sensitivity of the Cochrane CENTRAL index because according to Hunter et al (2022) "register records as they appear in CENTRAL are less comprehensive than the original register entry, and thus are at a greater risk than other systems of being missed in a search."

The Polyglot Search Translator is a very useful tool for translating search strings from PubMed or Medline via Ovid across multiple databases, developed by the Institute for Evidence-Based Healthcare at Bond University. But please note Polyglot does not automatically map subject terms across databases (e.g. MeSH terms to Emtree terms) so you will need to manually edit the search syntax in a text editor to change to the actual subject terms used by another database.

The Yale Mesh Analyzer is another very useful tool you can copy and paste in a list of up to 20 PMID numbers for records in the PubMed database, the Yale Mesh Analyzer will then display the Mesh Medical Subject Headings for those 20 articles as a table so you can identify and compare what Mesh headings they have in common, this can suggest additional search terms for your PubMed search.

The MedSyntax tool is another useful tool, for parsing out very long searches with many levels of brackets. This would be useful if you are trying to edit a pre-existing search strategy with many levels of parentheses.

Some sources for pre-existing database search filters or "hedges" include:

• CADTH Search Filters Database ,
• McMaster University Health Information Research Unit ,
• University of York Centre for Reviews and Dissemination InterTASC Information Specialists' Sub-Group ,
• InterTASC Population Specific search filters  (particularly useful for identifying Latinx, Indigenous people's, LGBTQ, Black & Minority ethnic)
• CareSearch Palliative Care PubMed search filters  (bereavement, dementia, heart failure, lung cancer, cost of care, and Palliative Care)
• Low and Middle Income countries filter at https://epoc.cochrane.org/lmic-filters . 
• Search Pubmed for another validated search filter using some variation of a search like this, possibly adding your discipline or search topic keywords: ("Databases, Bibliographic"[Mesh] OR "Search Engine"[Mesh]) AND ("Reproducibility of Results"[Mesh] OR "Sensitivity and Specificity"[Mesh] OR validat*) AND (filter OR hedge) .
• Search MEDLINE (or PubMed), preferably using a peer reviewed search strategy per protocol and apply any relevant methodology filters.
• Search EMBASE (or Scopus) and the Cochrane Central trials register using appropriately reformatted search versions for those databases, and any other online resources. 
• You should also search other subject specific databases that index the literature in your field.  Use our Himmelfarb Library  research guides  to identify other  subject specific databases . 
• Save citations in Covidence to deduplicate citations prior to screening.
• After screening export citations to  RefWorks database when you are ready to write up your manuscript. The Covidence and Refworks databases should be shared with all members of the investigative team.

Supplementary resources to search

Other member of your investigative team may have ideas about databases, websites, and journals they think you should search. Searching these sources is not required to perform a systematic review. You may need to reformat your search keywords.

Researchers at GW should check our subject research guides for suggestions, or check the libguides community for a guide on your subject.

In addition you may wish to search one or more of the following resources:

• Google Scholar
• BASE  academic search engine is useful for searching in University Institutional Repositories
• Cochrane Database of Systematic Reviews  to search for a pre-existing systematic review on your topic
• Epistemonikos database, has a matrix of evidence table so you can see what citations are shared in common across existing systematic reviews of the same topic. This feature might help identify sentinel or 'don't miss' articles.

You might also consider searching one or more of the following websites depending on your topic:

Clinical trial registers. The Cochrane Collaboration recommends for a systematic review to search both clinicaltrials.gov and the WHO ICTRP (See http://handbook.cochrane.org/ section 4.3):

• ClinicalTrials.gov  - also contains study population characteristics and results data of FDA regulated drugs and medical devices in NIH funded studies produced after January 18, 2017.
• WHO ICTRP  - trials register
• TRIP  - searchable index of clinical trials, guidelines,and regulatory guidance
• CenterWatch
• Current Controlled Trials
• European Clinical Trials Register
• ISRCTN Register
• COMPARE - tracks outcome switching in clinical trials
• OpenTrials - aims to match published trials with the underlying data where this is publicly available in an open source 
• ECRI Guidelines Trust

Grey literature resources:

• WONDER - CDC data and reports
• FDSys - search federal government publications
• Science.gov
• NRR Archive
• NIH Reporter
• re3data registry of data repositories
• Data Repositories (listed by the Simmons Open Access Directory)
• OpenDOAR  search academic open access research repositories
• f1000research search open access repositories of articles, slides, and research posters, in the life sciences, public health, education, and communication.
• RAND Health Reports
• National Academy of Medicine Publications
• Kaiser Family Foundation 
• Robert Wood Johnson Foundation health and medical care data archive
• Milbank Memorial Fund reports and issue briefs
• Also search the resources listed in the CADTH (2019) Grey Matters checklist.

Preprints 

• See our Himmelfarb preprints guide page on finding preprints , a useful database for searching Health Sciences preprints is  Europe PMC

Dissertations and Theses:

• Proquest Dissertations and Theses Online 
• Networked Digital Library of Theses and Dissertations
• Open Access Theses and Dissertations
• WorldCat and change Content: from Any Content to Thesis/dissertations

Conference proceedings:

Most conference proceedings are difficult to find because they may or may not be published. Only select individual papers may be made available in print as a book, journal, or series, rather than all of the presented items. Societies and Associations may only publish abstracts, or extended abstracts, from a conference, often in an annual supplement to an issue of the journal of record of that professional society.  Often posters are not published, if they are they may be made available only to other conference registrants at that meeting or online. Authors may "publish" their conference papers or posters on personal or institutional websites.  A limited set of conference proceedings databases include the following:

• BASE  academic search engine, has an Advanced Search feature with a Limit by Type to 'Conference Objects', this is useful for searching for conference posters and submissions stored in University Institutional Repositories.
• Web of Science - click All Databases and select Core Collection - under More Settings limit to the Conference Proceedings Citation Index (CPCI) - searches a limited set of conferences on Science, Social Science and Humanities from 1990-present.
• Scopus - Limit Document Type to Conference Paper or Conference Review.
• Proquest  - Limit search results to conference papers &/or proceedings under Advanced Search.
• BioMed Central Proceedings  - searches a limited set of biomedical conference proceedings, including bioinformatics, genetics, medical students, and data visualization.
• F1000 Research - browse by subject and click the tabs for articles, posters, and slides - which searches a limited number of biology and medical society meetings/conferences. This is a voluntary self-archive repository.

Individual Journals 

• You may choose to "hand search" select journals where the research team reads the Table of Contents of each issue for a chosen period of time.  You can look for the names of high impact journal titles in a particular field indexed in Journal Citation Reports  (JCR). Please note as of August 2021 ISI are linking to a new version of JCR that currently does not have the particularly helpful 'Browse by Category' link working, so I recommend you click the Products link in the top right corner and select Journal Citation Reports (Classic) to switch back to the old version to get that functionality back.
• The AllTrials petition aims to motivate health care researchers to petition regulators and research bodies to require the results and data of all clinical trials be published.
• << Previous: Search strategy
• Next: Study selection and appraisal >>

• Last Updated: May 8, 2024 11:07 AM
• URL: https://guides.himmelfarb.gwu.edu/systematic_review

• Himmelfarb Intranet
• Privacy Notice
• Terms of Use
• GW is committed to digital accessibility. If you experience a barrier that affects your ability to access content on this page, let us know via the Accessibility Feedback Form .
• Himmelfarb Health Sciences Library
• 2300 Eye St., NW, Washington, DC 20037
• Phone: (202) 994-2850
• [email protected]
• https://himmelfarb.gwu.edu

• En español – ExME
• Em português – EME

Literature searches: what databases are available?

Posted on 6th April 2021 by Izabel de Oliveira

Many types of research require a search of the medical literature as part of the process of understanding the current evidence or knowledge base. This can be done using one or more biomedical bibliographic databases. [1]

Bibliographic databases make the information contained in the papers more visible to the scientific community and facilitate locating the desired literature.

This blog describes some of the main bibliographic databases which index medical journals.

PubMed was launched in 1996 and, since June 1997, provides free and unlimited access for all users through the internet. PubMed database contains more than 30 million references of biomedical literature from approximately 7,000 journals. The largest percentage of records in PubMed comes from MEDLINE (95%), which contains 25 million records from over 5,600 journals. Other records derive from other sources such as In-process citations, ‘Ahead of Print’ citations, NCBI Bookshelf, etc.

The second largest component of PubMed is PubMed Central (PMC) . Launched in 2000, PMC is a permanent collection of full-text life sciences and biomedical journal articles. PMC also includes articles deposited by journal publishers and author manuscripts, published articles that are submitted in compliance with the public access policies of the National Institutes of Health (NIH) and other research funding agencies. PMC contains approximately 4.5 million articles.

Some National Library of Medicine (NLM) resources associated with PubMed are the NLM Catalog and MedlinePlus. The NLM Catalog contains bibliographic records for over 1.4 million journals, books, audiovisuals, electronic resources, and other materials. It also includes detailed indexing information for journals in PubMed and other NCBI databases, although not all materials in the NLM Catalog are part of NLM’s collection. MedlinePlus is a consumer health website providing information on various health topics, drugs, dietary supplements, and health tools.

MeSH (Medical Subject Headings) is the NLM controlled vocabulary used for indexing articles in PubMed. It is used by indexers who analyze and maintain the PubMed database to reflect the subject content of journal articles as they are published. Indexers typically select 10–12 MeSH terms to describe every paper.

Embase is considered the second most popular database after MEDLINE. More than 32 million records from over 8,200 journals from more than 95 countries, and ‘grey literature’ from over 2.4 million conference abstracts, are estimated to be in the Embase content.

Embase contains subtopics in health care such as complementary and alternative medicine, prognostic studies, telemedicine, psychiatry, and health technology. Besides that, it is also widely used for research on drug-related topics as it offers better coverage than MEDLINE on pharmaceutics-related literature.

In 2010, Embase began to include all MEDLINE citations. MEDLINE records are delivered to Elsevier daily and are incorporated into Embase after de-duplication with records already indexed by Elsevier to produce ‘MEDLINE-unique’ records. These MEDLINE-unique records are not re-indexed by Elsevier. However, their indexing is mapped to Emtree terms used in Embase to ensure that Emtree terminology can be used to search all Embase records, including those originally derived from MEDLINE.

Since this coverage expansion—at least in theory and without taking into consideration the different indexing practices of the two databases—a search in Embase alone should cover every record in both Embase and MEDLINE, making Embase a possible “one-stop” search engine for medical research [1].

Emtree is a hierarchically structured, controlled vocabulary for biomedicine and the related life sciences. It includes a whole range of terms for drugs, diseases, medical devices, and essential life science concepts. Emtree is used to index all of the Embase content. This process includes full-text indexing of journal articles, which is done by experts.

The most important index of the technical-scientific literature in Latin America and the Caribbean, LILACS , was created in 1985 to record scientific and technical production in health. It has been maintained and updated by a network of more than 600 institutions of education, government, and health research and coordinated by Latin America and Caribbean Center on Health Sciences Information (BIREME), Pan American Health Organization (PAHO), and World Health Organization (WHO).

LILACS contains scientific and technical literature from over 908 journals from 26 countries in Latin America and the Caribbean, with free access. About 900,000 records from articles with peer review, theses and dissertations, government documents, conference proceedings, and books; more than 480,000 of them are available with the full-text link in open access.

The LILACS Methodology is a set of standards, manuals, guides, and applications in continuous development, intended for the collection, selection, description, indexing of documents, and generation of databases. This centralised methodology enables the cooperation between Latin American and Caribbean countries to create local and national databases, all feeding into the LILACS database.  Currently, the databases LILACS, BBO, BDENF, MEDCARIB, and national databases of the countries of Latin America are part of the LILACS System.

Health Sciences Descriptors (DeCS) is the multilingual and structured vocabulary created by BIREME to serve as a unique language in indexing articles from scientific journals, books, congress proceedings, technical reports, and other types of materials, and also for searching and retrieving subjects from scientific literature from information sources available on the Virtual Health Library (VHL) such as LILACS, MEDLINE, and others. It was developed from the MeSH with the purpose of permitting the use of common terminology for searching in multiple languages, and providing a consistent and unique environment for the retrieval of information. DeCS vocabulary is dynamic and totals 34,118 descriptors and qualifiers, of which 29,716 come from MeSH, and 4,402 are exclusive.

Cochrane CENTRAL

The Cochrane Central Register of Controlled Trials (CENTRAL) is a database of reports of randomized and quasi-randomized controlled trials. Most records are obtained from the bibliographic databases PubMed and Embase, with additional records from the published and unpublished sources of CINAHL, ClinicalTrials.gov, and the WHO’s International Clinical Trials Registry Platform.

Although CENTRAL first began publication in 1996, records are included irrespective of the date of publication, and the language of publication is also not a restriction to being included in the database.  You won’t find the full text to the article on CENTRAL but there is often a summary of the article, in addition to the standard details of author, source, and year.

Within CENTRAL, there are ‘Specialized Registers’ which are collected and maintained by Cochrane Review Groups (plus a few Cochrane Fields), which include reports of controlled trials relevant to their area of interest. Some Cochrane Centres search the general healthcare literature of their countries or regions in order to contribute records to CENTRAL.

ScienceDirect

ScienceDirect i s Elsevier’s most important peer-reviewed academic literature platform. It was launched in 1997 and contains 16 million records from over 2,500 journals, including over 250 Open Access publications, such as Cell Reports and The Lancet Global Health, as well as 39,000 eBooks.

ScienceDirect topics include:

• health sciences;
• life sciences;
• physical sciences;
• engineering;
• social sciences; and
• humanities.

Web of Science

Web of Science (previously Web of Knowledge) is an online scientific citation indexing service created in 1997 by the Institute for Scientific Information (ISI), and currently maintained by Clarivate Analytics.

Web of Science covers several fields of the sciences, social sciences, and arts and humanities. Its main resource is the Web of Science Core Collection which includes over 1 billion cited references dating back to 1900, indexed from 21,100 peer-reviewed journals, including Open Access journals, books and proceedings.

Web of Science also offers regional databases which cover:

• Latin America (SciELO Citation Index);
• China (Chinese Science Citation Database);
• Korea (Korea Citation Index);
• Russia (Russian Science Citation Index).

Boolean operators

To make the search more precise, we can use boolean operators in databases between our keywords.

We use boolean operators to focus on a topic, particularly when this topic contains multiple search terms, and to connect various pieces of information in order to find exactly what we are looking for.

Boolean operators connect the search words to either narrow or broaden the set of results. The three basic boolean operators are: AND, OR, and NOT.

• AND narrows a search by telling the database that all keywords used must be found in the article in order for it to appear in our results.
• OR broadens a search by telling the database that any of the words it connects are acceptable (this is useful when we are searching for synonymous words).
• NOT narrows the search by telling the database to eliminate all terms that follow it from our search results (this is helpful when we are interested in a specific aspect of a topic or when we want to exclude a type of article.

References (pdf)

You may also be interested in the following blogs for further reading:

Conducting a systematic literature search

Reviewing the evidence: what method should I use?

Cochrane Crowd for students: what’s in it for you?

Izabel de Oliveira

Leave a reply cancel reply.

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

Subscribe to our newsletter

You will receive our monthly newsletter and free access to Trip Premium.

Related Articles

Epistemonikos: the world’s largest repository of healthcare systematic reviews

Learn more about the Epistemonikos Foundation and its repository of healthcare systematic reviews. The first in a series of three blogs.

How do you use the Epistemonikos database?

Learn how to use the Epistemonikos database, the world’s largest multilingual repository of healthcare evidence. The second in a series of three blogs.

Epistemonikos: All you need is L·OVE

Discover more about the ‘Living OVerview of Evidence’ platform from Epistemonikos, which maps the best evidence relevant for making health decisions. The final blog in a series of three focusing on the Epistemonikos Foundation.

An official website of the United States government

Here’s how you know

Official websites use .gov A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS A lock ( Lock A locked padlock ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

MEDLINE Overview

MEDLINE is the National Library of Medicine’s (NLM) premier bibliographic database that contains more than 31 million references to journal articles in life sciences with a concentration on biomedicine.

MEDLINE is a primary component of PubMed , a literature database developed and maintained by the NLM National Center for Biotechnology Information (NCBI). MEDLINE is the online counterpart to the MEDical Literature Analysis and Retrieval System (MEDLARS) that originated in 1964 (see MEDLINE history ). A distinctive feature of MEDLINE is that the records are indexed with NLM Medical Subject Headings (MeSH).

Selection process:  The current procedure for adding new journals to MEDLINE is described on the page How to Include a Journal in MEDLINE . Time coverage:  MEDLINE includes literature published from 1966 to present, and selected coverage of literature prior to that period. For details about pre-1966 citations see OLDMEDLINE Data .

Source:  Currently, citations from more than 5,200 worldwide journals in about 40 languages.

Updates:  Citations are added to PubMed 7 days a week.

Subject coverage:  In line with the Collection and Preservation Policy of the NLM , the subject scope of MEDLINE is biomedicine and health, broadly defined to encompass those areas of the life sciences, behavioral sciences, chemical sciences, and bioengineering needed by health professionals and others engaged in basic research and clinical care, public health, health policy development, or related educational activities. MEDLINE also covers life sciences vital to biomedical practitioners, researchers, and educators, including aspects of biology, environmental science, marine biology, plant and animal science as well as biophysics and chemistry.

The majority of the publications in MEDLINE are scholarly journals; however, a small number of newspapers, magazines, and newsletters considered useful to particular segments of the NLM broad user community have historically been included.

Availability: Searching MEDLINE via PubMed results in a list of citations (including authors, title, source, and often an abstract) to journal articles and links to the electronic full-text, if available. Searching PubMed is free of charge and does not require registration.

A growing number of MEDLINE citations in PubMed contain a link to the free full-text of the article archived in PubMed Central . Some MEDLINE citations also have publisher-provided links to the full-text of the article at the journal website; exact details depend on the publisher's access requirements.

PubMed data, including MEDLINE, can be downloaded as described on the Download PubMed Data page.

Last Reviewed: February 5, 2024

How to undertake a literature search: a step-by-step guide

Affiliation.

• 1 Literature Search Specialist, Library and Archive Service, Royal College of Nursing, London.
• PMID: 32279549
• DOI: 10.12968/bjon.2020.29.7.431

Undertaking a literature search can be a daunting prospect. Breaking the exercise down into smaller steps will make the process more manageable. This article suggests 10 steps that will help readers complete this task, from identifying key concepts to choosing databases for the search and saving the results and search strategy. It discusses each of the steps in a little more detail, with examples and suggestions on where to get help. This structured approach will help readers obtain a more focused set of results and, ultimately, save time and effort.

Keywords: Databases; Literature review; Literature search; Reference management software; Research questions; Search strategy.

• Databases, Bibliographic*
• Information Storage and Retrieval / methods*
• Nursing Research
• Review Literature as Topic*

• Open access
• Published: 06 December 2017

Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study

• Wichor M. Bramer 1 ,
• Melissa L. Rethlefsen 2 ,
• Jos Kleijnen 3 , 4 &
• Oscar H. Franco 5  

Systematic Reviews volume  6 , Article number:  245 ( 2017 ) Cite this article

155k Accesses

857 Citations

88 Altmetric

Metrics details

Within systematic reviews, when searching for relevant references, it is advisable to use multiple databases. However, searching databases is laborious and time-consuming, as syntax of search strategies are database specific. We aimed to determine the optimal combination of databases needed to conduct efficient searches in systematic reviews and whether the current practice in published reviews is appropriate. While previous studies determined the coverage of databases, we analyzed the actual retrieval from the original searches for systematic reviews.

Since May 2013, the first author prospectively recorded results from systematic review searches that he performed at his institution. PubMed was used to identify systematic reviews published using our search strategy results. For each published systematic review, we extracted the references of the included studies. Using the prospectively recorded results and the studies included in the publications, we calculated recall, precision, and number needed to read for single databases and databases in combination. We assessed the frequency at which databases and combinations would achieve varying levels of recall (i.e., 95%). For a sample of 200 recently published systematic reviews, we calculated how many had used enough databases to ensure 95% recall.

A total of 58 published systematic reviews were included, totaling 1746 relevant references identified by our database searches, while 84 included references had been retrieved by other search methods. Sixteen percent of the included references (291 articles) were only found in a single database; Embase produced the most unique references ( n  = 132). The combination of Embase, MEDLINE, Web of Science Core Collection, and Google Scholar performed best, achieving an overall recall of 98.3 and 100% recall in 72% of systematic reviews. We estimate that 60% of published systematic reviews do not retrieve 95% of all available relevant references as many fail to search important databases. Other specialized databases, such as CINAHL or PsycINFO, add unique references to some reviews where the topic of the review is related to the focus of the database.

Conclusions

Optimal searches in systematic reviews should search at least Embase, MEDLINE, Web of Science, and Google Scholar as a minimum requirement to guarantee adequate and efficient coverage.

Peer Review reports

Investigators and information specialists searching for relevant references for a systematic review (SR) are generally advised to search multiple databases and to use additional methods to be able to adequately identify all literature related to the topic of interest [ 1 , 2 , 3 , 4 , 5 , 6 ]. The Cochrane Handbook, for example, recommends the use of at least MEDLINE and Cochrane Central and, when available, Embase for identifying reports of randomized controlled trials [ 7 ]. There are disadvantages to using multiple databases. It is laborious for searchers to translate a search strategy into multiple interfaces and search syntaxes, as field codes and proximity operators differ between interfaces. Differences in thesaurus terms between databases add another significant burden for translation. Furthermore, it is time-consuming for reviewers who have to screen more, and likely irrelevant, titles and abstracts. Lastly, access to databases is often limited and only available on subscription basis.

Previous studies have investigated the added value of different databases on different topics [ 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 ]. Some concluded that searching only one database can be sufficient as searching other databases has no effect on the outcome [ 16 , 17 ]. Nevertheless others have concluded that a single database is not sufficient to retrieve all references for systematic reviews [ 18 , 19 ]. Most articles on this topic draw their conclusions based on the coverage of databases [ 14 ]. A recent paper tried to find an acceptable number needed to read for adding an additional database; sadly, however, no true conclusion could be drawn [ 20 ]. However, whether an article is present in a database may not translate to being found by a search in that database. Because of this major limitation, the question of which databases are necessary to retrieve all relevant references for a systematic review remains unanswered. Therefore, we research the probability that single or various combinations of databases retrieve the most relevant references in a systematic review by studying actual retrieval in various databases.

The aim of our research is to determine the combination of databases needed for systematic review searches to provide efficient results (i.e., to minimize the burden for the investigators without reducing the validity of the research by missing relevant references). A secondary aim is to investigate the current practice of databases searched for published reviews. Are included references being missed because the review authors failed to search a certain database?

Development of search strategies

At Erasmus MC, search strategies for systematic reviews are often designed via a librarian-mediated search service. The information specialists of Erasmus MC developed an efficient method that helps them perform searches in many databases in a much shorter time than other methods. This method of literature searching and a pragmatic evaluation thereof are published in separate journal articles [ 21 , 22 ]. In short, the method consists of an efficient way to combine thesaurus terms and title/abstract terms into a single line search strategy. This search is then optimized. Articles that are indexed with a set of identified thesaurus terms, but do not contain the current search terms in title or abstract, are screened to discover potential new terms. New candidate terms are added to the basic search and evaluated. Once optimal recall is achieved, macros are used to translate the search syntaxes between databases, though manual adaptation of the thesaurus terms is still necessary.

Review projects at Erasmus MC cover a wide range of medical topics, from therapeutic effectiveness and diagnostic accuracy to ethics and public health. In general, searches are developed in MEDLINE in Ovid (Ovid MEDLINE® In-Process & Other Non-Indexed Citations, Ovid MEDLINE® Daily and Ovid MEDLINE®, from 1946); Embase.com (searching both Embase and MEDLINE records, with full coverage including Embase Classic); the Cochrane Central Register of Controlled Trials (CENTRAL) via the Wiley Interface; Web of Science Core Collection (hereafter called Web of Science); PubMed restricting to records in the subset “as supplied by publisher” to find references that not yet indexed in MEDLINE (using the syntax publisher [sb]); and Google Scholar. In general, we use the first 200 references as sorted in the relevance ranking of Google Scholar. When the number of references from other databases was low, we expected the total number of potential relevant references to be low. In this case, the number of hits from Google Scholar was limited to 100. When the overall number of hits was low, we additionally searched Scopus, and when appropriate for the topic, we included CINAHL (EBSCOhost), PsycINFO (Ovid), and SportDiscus (EBSCOhost) in our search.

Beginning in May 2013, the number of records retrieved from each search for each database was recorded at the moment of searching. The complete results from all databases used for each of the systematic reviews were imported into a unique EndNote library upon search completion and saved without deduplication for this research. The researchers that requested the search received a deduplicated EndNote file from which they selected the references relevant for inclusion in their systematic review. All searches in this study were developed and executed by W.M.B.

Determining relevant references of published reviews

We searched PubMed in July 2016 for all reviews published since 2014 where first authors were affiliated to Erasmus MC, Rotterdam, the Netherlands, and matched those with search registrations performed by the medical library of Erasmus MC. This search was used in earlier research [ 21 ]. Published reviews were included if the search strategies and results had been documented at the time of the last update and if, at minimum, the databases Embase, MEDLINE, Cochrane CENTRAL, Web of Science, and Google Scholar had been used in the review. From the published journal article, we extracted the list of final included references. We documented the department of the first author. To categorize the types of patient/population and intervention, we identified broad MeSH terms relating to the most important disease and intervention discussed in the article. We copied from the MeSH tree the top MeSH term directly below the disease category or, in to case of the intervention, directly below the therapeutics MeSH term. We selected the domain from a pre-defined set of broad domains, including therapy, etiology, epidemiology, diagnosis, management, and prognosis. Lastly, we checked whether the reviews described limiting their included references to a particular study design.

To identify whether our searches had found the included references, and if so, from which database(s) that citation was retrieved, each included reference was located in the original corresponding EndNote library using the first author name combined with the publication year as a search term for each specific relevant publication. If this resulted in extraneous results, the search was subsequently limited using a distinct part of the title or a second author name. Based on the record numbers of the search results in EndNote, we determined from which database these references came. If an included reference was not found in the EndNote file, we presumed the authors used an alternative method of identifying the reference (e.g., examining cited references, contacting prominent authors, or searching gray literature), and we did not include it in our analysis.

Data analysis

We determined the databases that contributed most to the reviews by the number of unique references retrieved by each database used in the reviews. Unique references were included articles that had been found by only one database search. Those databases that contributed the most unique included references were then considered candidate databases to determine the most optimal combination of databases in the further analyses.

In Excel, we calculated the performance of each individual database and various combinations. Performance was measured using recall, precision, and number needed to read. See Table  1 for definitions of these measures. These values were calculated both for all reviews combined and per individual review.

Performance of a search can be expressed in different ways. Depending on the goal of the search, different measures may be optimized. In the case of a clinical question, precision is most important, as a practicing clinician does not have a lot of time to read through many articles in a clinical setting. When searching for a systematic review, recall is the most important aspect, as the researcher does not want to miss any relevant references. As our research is performed on systematic reviews, the main performance measure is recall.

We identified all included references that were uniquely identified by a single database. For the databases that retrieved the most unique included references, we calculated the number of references retrieved (after deduplication) and the number of included references that had been retrieved by all possible combinations of these databases, in total and per review. For all individual reviews, we determined the median recall, the minimum recall, and the percentage of reviews for which each single database or combination retrieved 100% recall.

For each review that we investigated, we determined what the recall was for all possible different database combinations of the most important databases. Based on these, we determined the percentage of reviews where that database combination had achieved 100% recall, more than 95%, more than 90%, and more than 80%. Based on the number of results per database both before and after deduplication as recorded at the time of searching, we calculated the ratio between the total number of results and the number of results for each database and combination.

Improvement of precision was calculated as the ratio between the original precision from the searches in all databases and the precision for each database and combination.

To compare our practice of database usage in systematic reviews against current practice as evidenced in the literature, we analyzed a set of 200 recent systematic reviews from PubMed. On 5 January 2017, we searched PubMed for articles with the phrase “systematic review” in the title. Starting with the most recent articles, we determined the databases searched either from the abstract or from the full text until we had data for 200 reviews. For the individual databases and combinations that were used in those reviews, we multiplied the frequency of occurrence in that set of 200 with the probability that the database or combination would lead to an acceptable recall (which we defined at 95%) that we had measured in our own data.

Our earlier research had resulted in 206 systematic reviews published between 2014 and July 2016, in which the first author was affiliated with Erasmus MC [ 21 ]. In 73 of these, the searches and results had been documented by the first author of this article at the time of the last search. Of those, 15 could not be included in this research, since they had not searched all databases we investigated here. Therefore, for this research, a total of 58 systematic reviews were analyzed. The references to these reviews can be found in Additional file 1 . An overview of the broad topical categories covered in these reviews is given in Table  2 . Many of the reviews were initiated by members of the departments of surgery and epidemiology. The reviews covered a wide variety of disease, none of which was present in more than 12% of the reviews. The interventions were mostly from the chemicals and drugs category, or surgical procedures. Over a third of the reviews were therapeutic, while slightly under a quarter answered an etiological question. Most reviews did not limit to certain study designs, 9% limited to RCTs only, and another 9% limited to other study types.

Together, these reviews included a total of 1830 references. Of these, 84 references (4.6%) had not been retrieved by our database searches and were not included in our analysis, leaving in total 1746 references. In our analyses, we combined the results from MEDLINE in Ovid and PubMed (the subset as supplied by publisher) into one database labeled MEDLINE.

Unique references per database

A total of 292 (17%) references were found by only one database. Table  3 displays the number of unique results retrieved for each single database. Embase retrieved the most unique included references, followed by MEDLINE, Web of Science, and Google Scholar. Cochrane CENTRAL is absent from the table, as for the five reviews limited to randomized trials, it did not add any unique included references. Subject-specific databases such as CINAHL, PsycINFO, and SportDiscus only retrieved additional included references when the topic of the review was directly related to their special content, respectively nursing, psychiatry, and sports medicine.

Overall performance

The four databases that had retrieved the most unique references (Embase, MEDLINE, Web of Science, and Google Scholar) were investigated individually and in all possible combinations (see Table  4 ). Of the individual databases, Embase had the highest overall recall (85.9%). Of the combinations of two databases, Embase and MEDLINE had the best results (92.8%). Embase and MEDLINE combined with either Google Scholar or Web of Science scored similarly well on overall recall (95.9%). However, the combination with Google Scholar had a higher precision and higher median recall, a higher minimum recall, and a higher proportion of reviews that retrieved all included references. Using both Web of Science and Google Scholar in addition to MEDLINE and Embase increased the overall recall to 98.3%. The higher recall from adding extra databases came at a cost in number needed to read (NNR). Searching only Embase produced an NNR of 57 on average, whereas, for the optimal combination of four databases, the NNR was 73.

Probability of appropriate recall

We calculated the recall for individual databases and databases in all possible combination for all reviews included in the research. Figure  1 shows the percentages of reviews where a certain database combination led to a certain recall. For example, in 48% of all systematic reviews, the combination of Embase and MEDLINE (with or without Cochrane CENTRAL; Cochrane CENTRAL did not add unique relevant references) reaches a recall of at least 95%. In 72% of studied systematic reviews, the combination of Embase, MEDLINE, Web of Science, and Google Scholar retrieved all included references. In the top bar, we present the results of the complete database searches relative to the total number of included references. This shows that many database searches missed relevant references.

Percentage of systematic reviews for which a certain database combination reached a certain recall. The X -axis represents the percentage of reviews for which a specific combination of databases, as shown on the y -axis, reached a certain recall (represented with bar colors). Abbreviations: EM Embase, ML MEDLINE, WoS Web of Science, GS Google Scholar. Asterisk indicates that the recall of all databases has been calculated over all included references. The recall of the database combinations was calculated over all included references retrieved by any database

Differences between domains of reviews

We analyzed whether the added value of Web of Science and Google Scholar was dependent of the domain of the review. For 55 reviews, we determined the domain. See Fig.  2 for the comparison of the recall of Embase, MEDLINE, and Cochrane CENTRAL per review for all identified domains. For all but one domain, the traditional combination of Embase, MEDLINE, and Cochrane CENTRAL did not retrieve enough included references. For four out of five systematic reviews that limited to randomized controlled trials (RCTs) only, the traditional combination retrieved 100% of all included references. However, for one review of this domain, the recall was 82%. Of the 11 references included in this review, one was found only in Google Scholar and one only in Web of Science.

Percentage of systematic reviews of a certain domain for which the combination Embase, MEDLINE and Cochrane CENTRAL reached a certain recall

Reduction in number of results

We calculated the ratio between the number of results found when searching all databases, including databases not included in our analyses, such as Scopus, PsycINFO, and CINAHL, and the number of results found searching a selection of databases. See Fig.  3 for the legend of the plots in Figs.  4 and 5 . Figure  4 shows the distribution of this value for individual reviews. The database combinations with the highest recall did not reduce the total number of results by large margins. Moreover, in combinations where the number of results was greatly reduced, the recall of included references was lower.

Legend of Figs. 3 and 4

The ratio between number of results per database combination and the total number of results for all databases

The ratio between precision per database combination and the total precision for all databases

Improvement of precision

To determine how searching multiple databases affected precision, we calculated for each combination the ratio between the original precision, observed when all databases were searched, and the precision calculated for different database combinations. Figure  5 shows the improvement of precision for 15 databases and database combinations. Because precision is defined as the number of relevant references divided by the number of total results, we see a strong correlation with the total number of results.

Status of current practice of database selection

From a set of 200 recent SRs identified via PubMed, we analyzed the databases that had been searched. Almost all reviews (97%) reported a search in MEDLINE. Other databases that we identified as essential for good recall were searched much less frequently; Embase was searched in 61% and Web of Science in 35%, and Google Scholar was only used in 10% of all reviews. For all individual databases or combinations of the four important databases from our research (MEDLINE, Embase, Web of Science, and Google Scholar), we multiplied the frequency of occurrence of that combination in the random set, with the probability we found in our research that this combination would lead to an acceptable recall of 95%. The calculation is shown in Table  5 . For example, around a third of the reviews (37%) relied on the combination of MEDLINE and Embase. Based on our findings, this combination achieves acceptable recall about half the time (47%). This implies that 17% of the reviews in the PubMed sample would have achieved an acceptable recall of 95%. The sum of all these values is the total probability of acceptable recall in the random sample. Based on these calculations, we estimate that the probability that this random set of reviews retrieved more than 95% of all possible included references was 40%. Using similar calculations, also shown in Table  5 , we estimated the probability that 100% of relevant references were retrieved is 23%.

Our study shows that, to reach maximum recall, searches in systematic reviews ought to include a combination of databases. To ensure adequate performance in searches (i.e., recall, precision, and number needed to read), we find that literature searches for a systematic review should, at minimum, be performed in the combination of the following four databases: Embase, MEDLINE (including Epub ahead of print), Web of Science Core Collection, and Google Scholar. Using that combination, 93% of the systematic reviews in our study obtained levels of recall that could be considered acceptable (> 95%). Unique results from specialized databases that closely match systematic review topics, such as PsycINFO for reviews in the fields of behavioral sciences and mental health or CINAHL for reviews on the topics of nursing or allied health, indicate that specialized databases should be used additionally when appropriate.

We find that Embase is critical for acceptable recall in a review and should always be searched for medically oriented systematic reviews. However, Embase is only accessible via a paid subscription, which generally makes it challenging for review teams not affiliated with academic medical centers to access. The highest scoring database combination without Embase is a combination of MEDLINE, Web of Science, and Google Scholar, but that reaches satisfactory recall for only 39% of all investigated systematic reviews, while still requiring a paid subscription to Web of Science. Of the five reviews that included only RCTs, four reached 100% recall if MEDLINE, Web of Science, and Google Scholar combined were complemented with Cochrane CENTRAL.

The Cochrane Handbook recommends searching MEDLINE, Cochrane CENTRAL, and Embase for systematic reviews of RCTs. For reviews in our study that included RCTs only, indeed, this recommendation was sufficient for four (80%) of the reviews. The one review where it was insufficient was about alternative medicine, specifically meditation and relaxation therapy, where one of the missed studies was published in the Indian Journal of Positive Psychology . The other study from the Journal of Advanced Nursing is indexed in MEDLINE and Embase but was only retrieved because of the addition of KeyWords Plus in Web of Science. We estimate more than 50% of reviews that include more study types than RCTs would miss more than 5% of included references if only traditional combination of MEDLINE, Embase, and Cochrane CENTAL is searched.

We are aware that the Cochrane Handbook [ 7 ] recommends more than only these databases, but further recommendations focus on regional and specialized databases. Though we occasionally used the regional databases LILACS and SciELO in our reviews, they did not provide unique references in our study. Subject-specific databases like PsycINFO only added unique references to a small percentage of systematic reviews when they had been used for the search. The third key database we identified in this research, Web of Science, is only mentioned as a citation index in the Cochrane Handbook, not as a bibliographic database. To our surprise, Cochrane CENTRAL did not identify any unique included studies that had not been retrieved by the other databases, not even for the five reviews focusing entirely on RCTs. If Erasmus MC authors had conducted more reviews that included only RCTs, Cochrane CENTRAL might have added more unique references.

MEDLINE did find unique references that had not been found in Embase, although our searches in Embase included all MEDLINE records. It is likely caused by difference in thesaurus terms that were added, but further analysis would be required to determine reasons for not finding the MEDLINE records in Embase. Although Embase covers MEDLINE, it apparently does not index every article from MEDLINE. Thirty-seven references were found in MEDLINE (Ovid) but were not available in Embase.com . These are mostly unique PubMed references, which are not assigned MeSH terms, and are often freely available via PubMed Central.

Google Scholar adds relevant articles not found in the other databases, possibly because it indexes the full text of all articles. It therefore finds articles in which the topic of research is not mentioned in title, abstract, or thesaurus terms, but where the concepts are only discussed in the full text. Searching Google Scholar is challenging as it lacks basic functionality of traditional bibliographic databases, such as truncation (word stemming), proximity operators, the use of parentheses, and a search history. Additionally, search strategies are limited to a maximum of 256 characters, which means that creating a thorough search strategy can be laborious.

Whether Embase and Web of Science can be replaced by Scopus remains uncertain. We have not yet gathered enough data to be able to make a full comparison between Embase and Scopus. In 23 reviews included in this research, Scopus was searched. In 12 reviews (52%), Scopus retrieved 100% of all included references retrieved by Embase or Web of Science. In the other 48%, the recall by Scopus was suboptimal, in one occasion as low as 38%.

Of all reviews in which we searched CINAHL and PsycINFO, respectively, for 6 and 9% of the reviews, unique references were found. For CINAHL and PsycINFO, in one case each, unique relevant references were found. In both these reviews, the topic was highly related to the topic of the database. Although we did not use these special topic databases in all of our reviews, given the low number of reviews where these databases added relevant references, and observing the special topics of those reviews, we suggest that these subject databases will only add value if the topic is related to the topic of the database.

Many articles written on this topic have calculated overall recall of several reviews, instead of the effects on all individual reviews. Researchers planning a systematic review generally perform one review, and they need to estimate the probability that they may miss relevant articles in their search. When looking at the overall recall, the combination of Embase and MEDLINE and either Google Scholar or Web of Science could be regarded sufficient with 96% recall. This number however is not an answer to the question of a researcher performing a systematic review, regarding which databases should be searched. A researcher wants to be able to estimate the chances that his or her current project will miss a relevant reference. However, when looking at individual reviews, the probability of missing more than 5% of included references found through database searching is 33% when Google Scholar is used together with Embase and MEDLINE and 30% for the Web of Science, Embase, and MEDLINE combination. What is considered acceptable recall for systematic review searches is open for debate and can differ between individuals and groups. Some reviewers might accept a potential loss of 5% of relevant references; others would want to pursue 100% recall, no matter what cost. Using the results in this research, review teams can decide, based on their idea of acceptable recall and the desired probability which databases to include in their searches.

Strengths and limitations

We did not investigate whether the loss of certain references had resulted in changes to the conclusion of the reviews. Of course, the loss of a minor non-randomized included study that follows the systematic review’s conclusions would not be as problematic as losing a major included randomized controlled trial with contradictory results. However, the wide range of scope, topic, and criteria between systematic reviews and their related review types make it very hard to answer this question.

We found that two databases previously not recommended as essential for systematic review searching, Web of Science and Google Scholar, were key to improving recall in the reviews we investigated. Because this is a novel finding, we cannot conclude whether it is due to our dataset or to a generalizable principle. It is likely that topical differences in systematic reviews may impact whether databases such as Web of Science and Google Scholar add value to the review. One explanation for our finding may be that if the research question is very specific, the topic of research might not always be mentioned in the title and/or abstract. In that case, Google Scholar might add value by searching the full text of articles. If the research question is more interdisciplinary, a broader science database such as Web of Science is likely to add value. The topics of the reviews studied here may simply have fallen into those categories, though the diversity of the included reviews may point to a more universal applicability.

Although we searched PubMed as supplied by publisher separately from MEDLINE in Ovid, we combined the included references of these databases into one measurement in our analysis. Until 2016, the most complete MEDLINE selection in Ovid still lacked the electronic publications that were already available in PubMed. These could be retrieved by searching PubMed with the subset as supplied by publisher. Since the introduction of the more complete MEDLINE collection Epub Ahead of Print , In-Process & Other Non-Indexed Citations , and Ovid MEDLINE® , the need to separately search PubMed as supplied by publisher has disappeared. According to our data, PubMed’s “as supplied by publisher” subset retrieved 12 unique included references, and it was the most important addition in terms of relevant references to the four major databases. It is therefore important to search MEDLINE including the “Epub Ahead of Print, In-Process, and Other Non-Indexed Citations” references.

These results may not be generalizable to other studies for other reasons. The skills and experience of the searcher are one of the most important aspects in the effectiveness of systematic review search strategies [ 23 , 24 , 25 ]. The searcher in the case of all 58 systematic reviews is an experienced biomedical information specialist. Though we suspect that searchers who are not information specialists or librarians would have a higher possibility of less well-constructed searches and searches with lower recall, even highly trained searchers differ in their approaches to searching. For this study, we searched to achieve as high a recall as possible, though our search strategies, like any other search strategy, still missed some relevant references because relevant terms had not been used in the search. We are not implying that a combined search of the four recommended databases will never result in relevant references being missed, rather that failure to search any one of these four databases will likely lead to relevant references being missed. Our experience in this study shows that additional efforts, such as hand searching, reference checking, and contacting key players, should be made to retrieve extra possible includes.

Based on our calculations made by looking at random systematic reviews in PubMed, we estimate that 60% of these reviews are likely to have missed more than 5% of relevant references only because of the combinations of databases that were used. That is with the generous assumption that the searches in those databases had been designed sensitively enough. Even when taking into account that many searchers consider the use of Scopus as a replacement of Embase, plus taking into account the large overlap of Scopus and Web of Science, this estimate remains similar. Also, while the Scopus and Web of Science assumptions we made might be true for coverage, they are likely very different when looking at recall, as Scopus does not allow the use of the full features of a thesaurus. We see that reviewers rarely use Web of Science and especially Google Scholar in their searches, though they retrieve a great deal of unique references in our reviews. Systematic review searchers should consider using these databases if they are available to them, and if their institution lacks availability, they should ask other institutes to cooperate on their systematic review searches.

The major strength of our paper is that it is the first large-scale study we know of to assess database performance for systematic reviews using prospectively collected data. Prior research on database importance for systematic reviews has looked primarily at whether included references could have theoretically been found in a certain database, but most have been unable to ascertain whether the researchers actually found the articles in those databases [ 10 , 12 , 16 , 17 , 26 ]. Whether a reference is available in a database is important, but whether the article can be found in a precise search with reasonable recall is not only impacted by the database’s coverage. Our experience has shown us that it is also impacted by the ability of the searcher, the accuracy of indexing of the database, and the complexity of terminology in a particular field. Because these studies based on retrospective analysis of database coverage do not account for the searchers’ abilities, the actual findings from the searches performed, and the indexing for particular articles, their conclusions lack immediate translatability into practice. This research goes beyond retrospectively assessed coverage to investigate real search performance in databases. Many of the articles reporting on previous research concluded that one database was able to retrieve most included references. Halladay et al. [ 10 ] and van Enst et al. [ 16 ] concluded that databases other than MEDLINE/PubMed did not change the outcomes of the review, while Rice et al. [ 17 ] found the added value of other databases only for newer, non-indexed references. In addition, Michaleff et al. [ 26 ] found that Cochrane CENTRAL included 95% of all RCTs included in the reviews investigated. Our conclusion that Web of Science and Google Scholar are needed for completeness has not been shared by previous research. Most of the previous studies did not include these two databases in their research.

We recommend that, regardless of their topic, searches for biomedical systematic reviews should combine Embase, MEDLINE (including electronic publications ahead of print), Web of Science (Core Collection), and Google Scholar (the 200 first relevant references) at minimum. Special topics databases such as CINAHL and PsycINFO should be added if the topic of the review directly touches the primary focus of a specialized subject database, like CINAHL for focus on nursing and allied health or PsycINFO for behavioral sciences and mental health. For reviews where RCTs are the desired study design, Cochrane CENTRAL may be similarly useful. Ignoring one or more of the databases that we identified as the four key databases will result in more precise searches with a lower number of results, but the researchers should decide whether that is worth the >increased probability of losing relevant references. This study also highlights once more that searching databases alone is, nevertheless, not enough to retrieve all relevant references.

Future research should continue to investigate recall of actual searches beyond coverage of databases and should consider focusing on the most optimal database combinations, not on single databases.

Levay P, Raynor M, Tuvey D. The contributions of MEDLINE, other bibliographic databases and various search techniques to NICE public health guidance. Evid Based Libr Inf Pract. 2015;10:50–68.

Article   Google Scholar  

Stevinson C, Lawlor DA. Searching multiple databases for systematic reviews: added value or diminishing returns? Complement Ther Med. 2004;12:228–32.

Article   CAS   PubMed   Google Scholar  

Lawrence DW. What is lost when searching only one literature database for articles relevant to injury prevention and safety promotion? Inj Prev. 2008;14:401–4.

Lemeshow AR, Blum RE, Berlin JA, Stoto MA, Colditz GA. Searching one or two databases was insufficient for meta-analysis of observational studies. J Clin Epidemiol. 2005;58:867–73.

Article   PubMed   Google Scholar  

Zheng MH, Zhang X, Ye Q, Chen YP. Searching additional databases except PubMed are necessary for a systematic review. Stroke. 2008;39:e139. author reply e140

Beyer FR, Wright K. Can we prioritise which databases to search? A case study using a systematic review of frozen shoulder management. Health Inf Libr J. 2013;30:49–58.

Higgins JPT, Green S. Cochrane handbook for systematic reviews of interventions: The Cochrane Collaboration, London, United Kingdom. 2011.

Wright K, Golder S, Lewis-Light K. What value is the CINAHL database when searching for systematic reviews of qualitative studies? Syst Rev. 2015;4:104.

Article   PubMed   PubMed Central   Google Scholar  

Wilkins T, Gillies RA, Davies K. EMBASE versus MEDLINE for family medicine searches: can MEDLINE searches find the forest or a tree? Can Fam Physician. 2005;51:848–9.

PubMed   Google Scholar  

Halladay CW, Trikalinos TA, Schmid IT, Schmid CH, Dahabreh IJ. Using data sources beyond PubMed has a modest impact on the results of systematic reviews of therapeutic interventions. J Clin Epidemiol. 2015;68:1076–84.

Ahmadi M, Ershad-Sarabi R, Jamshidiorak R, Bahaodini K. Comparison of bibliographic databases in retrieving information on telemedicine. J Kerman Univ Med Sci. 2014;21:343–54.

Google Scholar  

Lorenzetti DL, Topfer L-A, Dennett L, Clement F. Value of databases other than MEDLINE for rapid health technology assessments. Int J Technol Assess Health Care. 2014;30:173–8.

Beckles Z, Glover S, Ashe J, Stockton S, Boynton J, Lai R, Alderson P. Searching CINAHL did not add value to clinical questions posed in NICE guidelines. J Clin Epidemiol. 2013;66:1051–7.

Hartling L, Featherstone R, Nuspl M, Shave K, Dryden DM, Vandermeer B. The contribution of databases to the results of systematic reviews: a cross-sectional study. BMC Med Res Methodol. 2016;16:1–13.

Aagaard T, Lund H, Juhl C. Optimizing literature search in systematic reviews—are MEDLINE, EMBASE and CENTRAL enough for identifying effect studies within the area of musculoskeletal disorders? BMC Med Res Methodol. 2016;16:161.

van Enst WA, Scholten RJ, Whiting P, Zwinderman AH, Hooft L. Meta-epidemiologic analysis indicates that MEDLINE searches are sufficient for diagnostic test accuracy systematic reviews. J Clin Epidemiol. 2014;67:1192–9.

Rice DB, Kloda LA, Levis B, Qi B, Kingsland E, Thombs BD. Are MEDLINE searches sufficient for systematic reviews and meta-analyses of the diagnostic accuracy of depression screening tools? A review of meta-analyses. J Psychosom Res. 2016;87:7–13.

Bramer WM, Giustini D, Kramer BM, Anderson PF. The comparative recall of Google Scholar versus PubMed in identical searches for biomedical systematic reviews: a review of searches used in systematic reviews. Syst Rev. 2013;2:115.

Bramer WM, Giustini D, Kramer BMR. Comparing the coverage, recall, and precision of searches for 120 systematic reviews in Embase, MEDLINE, and Google Scholar: a prospective study. Syst Rev. 2016;5:39.

Ross-White A, Godfrey C. Is there an optimum number needed to retrieve to justify inclusion of a database in a systematic review search? Health Inf Libr J. 2017;33:217–24.

Bramer WM, Rethlefsen ML, Mast F, Kleijnen J. A pragmatic evaluation of a new method for librarian-mediated literature searches for systematic reviews. Res Synth Methods. 2017. doi: 10.1002/jrsm.1279 .

Bramer WM, de Jonge GB, Rethlefsen ML, Mast F, Kleijnen J. A systematic approach to searching: how to perform high quality literature searches more efficiently. J Med Libr Assoc. 2018.

Rethlefsen ML, Farrell AM, Osterhaus Trzasko LC, Brigham TJ. Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews. J Clin Epidemiol. 2015;68:617–26.

McGowan J, Sampson M. Systematic reviews need systematic searchers. J Med Libr Assoc. 2005;93:74–80.

PubMed   PubMed Central   Google Scholar  

McKibbon KA, Haynes RB, Dilks CJW, Ramsden MF, Ryan NC, Baker L, Flemming T, Fitzgerald D. How good are clinical MEDLINE searches? A comparative study of clinical end-user and librarian searches. Comput Biomed Res. 1990;23:583–93.

Michaleff ZA, Costa LO, Moseley AM, Maher CG, Elkins MR, Herbert RD, Sherrington C. CENTRAL, PEDro, PubMed, and EMBASE are the most comprehensive databases indexing randomized controlled trials of physical therapy interventions. Phys Ther. 2011;91:190–7.

Download references

Acknowledgements

Not applicable

Melissa Rethlefsen receives funding in part from the National Center for Advancing Translational Sciences of the National Institutes of Health under Award Number UL1TR001067. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Availability of data and materials

The datasets generated and/or analyzed during the current study are available from the corresponding author on a reasonable request.

Author information

Authors and affiliations.

Medical Library, Erasmus MC, Erasmus University Medical Centre Rotterdam, 3000 CS, Rotterdam, the Netherlands

Wichor M. Bramer

Spencer S. Eccles Health Sciences Library, University of Utah, Salt Lake City, Utah, USA

Melissa L. Rethlefsen

Kleijnen Systematic Reviews Ltd., York, UK

Jos Kleijnen

School for Public Health and Primary Care (CAPHRI), Maastricht University, Maastricht, the Netherlands

Department of Epidemiology, Erasmus MC, Erasmus University Medical Centre Rotterdam, Rotterdam, the Netherlands

Oscar H. Franco

You can also search for this author in PubMed   Google Scholar

Contributions

WB, JK, and OF designed the study. WB designed the searches used in this study and gathered the data. WB and ML analyzed the data. WB drafted the first manuscript, which was revised critically by the other authors. All authors have approved the final manuscript.

Corresponding author

Correspondence to Wichor M. Bramer .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

WB has received travel allowance from Embase for giving a presentation at a conference. The other authors declare no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:.

Reviews included in the research . References to the systematic reviews published by Erasmus MC authors that were included in the research. (DOCX 19 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Bramer, W.M., Rethlefsen, M.L., Kleijnen, J. et al. Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Syst Rev 6 , 245 (2017). https://doi.org/10.1186/s13643-017-0644-y

Download citation

Received : 21 August 2017

Accepted : 24 November 2017

Published : 06 December 2017

DOI : https://doi.org/10.1186/s13643-017-0644-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Databases, bibliographic
• Review literature as topic
• Sensitivity and specificity
• Information storage and retrieval

Systematic Reviews

ISSN: 2046-4053

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Reference management. Clean and simple.

The top list of research databases for medicine and healthcare

3. Cochrane Library

4. pubmed central (pmc), 5. uptodate, frequently asked questions about research databases for medicine and healthcare, related articles.

Web of Science and Scopus are interdisciplinary research databases and have a broad scope. For biomedical research, medicine, and healthcare there are a couple of outstanding academic databases that provide true value in your daily research.

Scholarly databases can help you find scientific articles, research papers , conference proceedings, reviews and much more. We have compiled a list of the top 5 research databases with a special focus on healthcare and medicine.

PubMed is the number one source for medical and healthcare research. It is hosted by the National Institutes of Health (NIH) and provides bibliographic information including abstracts and links to the full text publisher websites for more than 28 million articles.

• Coverage: around 35 million items
• Abstracts: ✔
• Related articles: ✔
• References: ✘
• Cited by: ✘
• Links to full text: ✔
• Export formats: XML, NBIB

Pro tip: Use a reference manager like Paperpile to keep track of all your sources. Paperpile integrates with PubMed and many popular databases. You can save references and PDFs directly to your library using the Paperpile buttons and later cite them in thousands of citation styles:

EMBASE (Excerpta Medica Database) is a proprietary research database that also includes PubMed. It can also be accessed by other database providers such as Ovid .

• Coverage: 38 million articles
• References: ✔
• Cited by: ✔
• Full text: ✔ (requires institutional subscription to EMBASE and individual publishers)
• Export formats: RIS

The Cochrane Library is best know for its systematic reviews. There are 53 review groups around the world that ensure that the published reviews are of high-quality and evidence based. Articles are updated over time to reflect new research.

• Coverage: several thousand high quality reviews
• Full text: ✔
• Export formats: RIS, BibTeX

PubMed Central is the free, open access branch of PubMed. It includes full-text versions for all indexed papers. You might also want to check out its sister site Europe PMC .

• Coverage: more than 8 million articles
• Export formats: APA, MLA, AMA, RIS, NBIB

Like the Cochrane Library, UpToDate provides detailed reviews for clinical topics. Reviews are constantly updated to provide an up-to-date view.

• Coverage: several thousand articles from over 420 peer-reviewed journals
• Related articles: ✘
• Full text: ✔ (requires institutional subscription)
• Export formats: ✘

PubMed is the number one source for medical and healthcare research. It is hosted at the National Institutes of Health (NIH) and provides bibliographic information including abstracts and links to the full text publisher websites for more than 35 million items.

EMBASE (Excerpta Medica Database) is a proprietary research database that also includes in its corpus PubMed. It can also be accessed by other database providers such as Ovid.

An official website of the United States government

Here's how you know

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Literature Search: Databases and Gray Literature

The literature search.

• A systematic review search includes a search of databases, gray literature, personal communications, and a handsearch of high impact journals in the related field.  See our list of recommended databases and gray literature sources on this page.
• a comprehensive literature search can not be dependent on a single database, nor on bibliographic databases only.
• inclusion of multiple databases helps avoid publication bias (georaphic bias or bias against publication of negative results).
• The Cochrane Collaboration recommends PubMed, Embase and the Cochrane Central Register of Controlled Trials (CENTRAL) at a minimum.     
• NOTE:  The Cochrane Collaboration and the IOM recommend that the literature search be conducted by librarians or persons with extensive literature search experience. Please contact the NIH Librarians for assistance with the literature search component of your systematic review. 

Cochrane Library

A collection of six databases that contain different types of high-quality, independent evidence to inform healthcare decision-making. Search the Cochrane Central Register of Controlled Trials here.

European database of biomedical and pharmacologic literature.

PubMed comprises more than 21 million citations for biomedical literature from MEDLINE, life science journals, and online books.

Largest abstract and citation database of peer-reviewed literature and quality web sources. Contains conference papers.

Web of Science

World's leading citation databases. Covers over 12,000 of the highest impact journals worldwide, including Open Access journals and over 150,000 conference proceedings. Coverage in the sciences, social sciences, arts, and humanities, with coverage to 1900.

Subject Specific Databases

APA PsycINFO

Over 4.5 million abstracts of peer-reviewed literature in the behavioral and social sciences. Includes conference papers, book chapters, psychological tests, scales and measurement tools.

CINAHL Plus

Comprehensive journal index to nursing and allied health literature, includes books, nursing dissertations, conference proceedings, practice standards and book chapters.

Latin American and Caribbean health sciences literature database

Gray Literature

• Gray Literature  is the term for information that falls outside the mainstream of published journal and mongraph literature, not controlled by commercial publishers
• hard to find studies, reports, or dissertations
• conference abstracts or papers
• governmental or private sector research
• clinical trials - ongoing or unpublished
• experts and researchers in the field     
• Library catalogs
• Professional association websites
• Google Scholar  - Search scholarly literature across many disciplines and sources, including theses, books, abstracts and articles.
• Dissertation Abstracts - dissertation and theses database - NIH Library biomedical librarians can access and search for you.
• NTIS  - central resource for government-funded scientific, technical, engineering, and business related information.
• AHRQ  - agency for healthcare research and quality
• Open Grey  - system for information on grey literature in Europe. Open access to 700,000 references to the grey literature.
• World Health Organization  - providing leadership on global health matters, shaping the health research agenda, setting norms and standards, articulating evidence-based policy options, providing technical support to countries and monitoring and assessing health trends.
• New York Academy of Medicine Grey Literature Report  - a bimonthly publication of The New York Academy of Medicine (NYAM) alerting readers to new gray literature publications in health services research and selected public health topics. NOTE: Discontinued as of Jan 2017, but resources are still accessible.
• Gray Source Index
• OpenDOAR - directory of academic repositories
• International Clinical Trials Registery Platform  - from the World Health Organization
• Australian New Zealand Clinical Trials Registry
• Brazilian Clinical Trials Registry
• Chinese Clinical Trial Registry - 
• ClinicalTrials.gov   - U.S.  and international federally and privately supported clinical trials registry and results database
• Clinical Trials Registry  - India
• EU clinical Trials Register
• Japan Primary Registries Network  
• Pan African Clinical Trials Registry

• Northeastern University Library
• Research Subject Guides
• Guides for Library Services
• Systematic Reviews and Evidence Syntheses
• Evidence Synthesis Service
• Types of Systematic Reviews in the Health Sciences
• Beginning Your Project
• Standards & Guidance
• Critical Appraisal
• Evidence-Based Assignments
• Tips for a Successful Review Team
• Training and Tutorials

Systematic Reviews and Evidence Syntheses : Databases

You will want to search at least three databases for your systematic review. Three databases alone does not complete the search standards for systematic review requirements. You will also have to complete a search of the grey literature and complete additional hand searches. Which databases you should search is highly dependent on your systematic review topic, so it is recommended you  meet with a librarian . 

Commonly Used Health Sciences Databases

Commonly used social sciences databases, commonly used education databases.

• Resources for Finding Systematic Reviews

You will want to search at least three databases for your systematic review. Three databases alone does not complete the search standards for systematic review requirements as you will also have additional searches of the grey literature and hand searches to complete.  Which databases you search is highly dependent on your systematic review topic, so it is recommended you  meet with a librarian . 

Cochrane, which is considered the gold standard for clinical systematic reviews, recommends searching the following three databases, at a minimum: PubMed, Embase, and Cochrane Central Register of Controlled Trials (CENTRAL).

• ERIC (Education Resources Institute) This link opens in a new window Citations to education information, including scholarly articles, professional literature, education dissertations, and books, plus grey literature such as curriculum guides, conference proceedings, government publications, and white papers. Covers 1966 to the present. more... less... Sponsored by the U.S. Department of Education.

Looking to Find Systematic Reviews?

There are a number of places to look for systematic reviews, including within the commonly used databases listed on this page. Some other resources to consider are:

• Systematic Review Repository - International Initiative for Impact Evaluation The systematic review repository from International Initiative for Impact Evaluation is an essential resource for policymakers and researchers who are looking for synthesized evidence on the effects of social and economic interventions in low- and middle- income countries.
• Epistemonikos Epistemonikos is a collaborative, multilingual database of health evidence. It is the largest source of systematic reviews relevant for health-decision making, and a large source of other types of scientific evidence. PLEASE NOTE: Epistemonikos is a systematic reviews focused database. It pulls in systematic reviews from a number of different international sources and pulls in the studies those reviews. While you will find randomized controlled trials and other primary studies in this database, they are only added in because of their association with a systematic review. Therefore, searching here for randomized controlled trials or other primary studies would NOT be considered a comprehensive search.
• << Previous: Types of Systematic Reviews in the Health Sciences
• Next: Resources for Completing Evidence Syntheses >>
• Ask a Librarian
• Last Updated: May 16, 2024 7:56 AM
• URL: https://subjectguides.lib.neu.edu/systematicreview

Article Definition

Written content on a narrow subject and published in a periodical or website. In some contexts, academics may use article as a shortened form of journal article.

• Green Paper
• Grey Literature

Bibliography Definition

A detailed list of resources cited in an article, book, or other publication. Also called a List of References.

Call Number Definition

A label of letters and/or numbers that tell you where the resource can be found in the library. Call numbers are displayed on print books and physical resources and correspond with a topic or subject area.

Systematic Reviews and Meta Analysis

• Getting Started
• Guides and Standards
• Review Protocols
• Databases and Sources
• Randomized Controlled Trials
• Controlled Clinical Trials
• Observational Designs
• Tests of Diagnostic Accuracy
• Software and Tools
• Where do I get all those articles?
• Collaborations
• EPI 233/528
• Countway Mediated Search
• Risk of Bias (RoB)

Data Sources

Databases you will probably search.

No one database can cover the literature for any topic. For medical topics, a combination of PubMed (or other search of PubMed data) plus Embase, Web of Science, and Google Scholar has been shown to provide adequate recall ( Syst Rev. 2017;6(1):245 ). For topics that reach beyond the biomedicine, other databases need to be considered.

• PubMed PubMed is both the search platform provided by the National Center for Biotechnology information and the database. PubMed includes MEDLINE (records indexed with MeSH terms) but also material in process, older records from before the inception of MEDLINE, and material from journals not included in MEDLINE. The PubMed database is available on independent platforms including Ovid SP, Web of Science, and several others.
• Embase Note: Embase requires users to either create an individual account (free) or log in with an institutional email address to enable the export of records. Before you start a session, 'log in" at the upper right. You can either create an account or use your Harvard email (recommended).   Embase includes materials second tier European and Asian journals not included in MEDLINE as well as conference abstracts. The Emtree controlled vocabulary is well developed. Embase records include more Emtree terms than MEDLINE records do MeSH term. Hence, results sets can often be significantly large in Embase, especially for drug-related searches.
• Cochrane Central Register of Controlled Trials Cochrane Central contains trials from both MEDLINE and Embase plus many trials from other, non-indexed sources; limited to randomized and non-randomized controlled trials.  MeSH for MEDLINE records, but no other controlled vocabulary. To limit to results in Central, click the "Trials" limit to the left of your results.
• Web of Science Core Collection (includes the Science Citation Index) Broad coverage of all sciences.  Will cover some journals at the edge of the biomedical sciences missed by PubMed and Embase. Some meeting information. No controlled vocabulary. Alternatively, the Elsevier database Scopus can be used. Harvard does not license access to Scopus.
• GoogleScholar Consider as a supplement to the literature databases. It can improve sensitivity because it searches the full-text of articles. Screening the first 200-400 records in a search is recommended.
• ClinicalTrials.gov Registers trials that are recruiting, completed, or terminated. Some records includes results.  Searching here helps identify unpublished trials. See below for other registries.

These database can be an effective complement to your search.  They can be essential in their specialized topic areas.

• BIOSIS Previews Although it is primarly useful for biologists, it contains a lot of meetings and some medical journals.  Controlled vocabulary is not suitable for medical searching.
• CINAHL Nursing and other health related information; excellent source for issues in patient care.  Well developed controlled vocabulary.
• PsycINFO Cognitive and behavioral therapies are well covered.  Controlled vocabulary.
• Google Scholar Add as an additional source. Here are some search tips.
• WHO Global Index Medicus Search all WHO regional indexes, including the South-East Asia and Western Pacific Pacific regional databases.
• Sociological Abstracts The primary index for sociological literature.  May be useful for community-related studies or interpersonal issues. Controlled vocabulary.
• 3ie Impact Evaluation Repository Investigating an ecomomic or social intervention? The 3ie Impact Evaluation Repository is a currated database for evidence of what works in international development in low- and middle-income countries.
• EconLit Economics. Almost any social intervetion and many medical ones get studied by economists.
• RePEc IDEAS A repository of economics literature. It includes bibliographic metadata from many archives.

Resources for Meetings and Other Grey Literature

Truely unbiased searches look for unpublished literature in a number of places, included meeting abstracts, white papers, clinical trial registries, and searching by hand.

• GreyNet GreyNet is an organization dedicated to promoting and facilitating the use of grey literature. Includes of listing of grey literature resources, GreySource .  OpenGrey, a former multidisciplinary database of technical reports, meetings, dissertations, and official publications is now archived in GreyNet. 
• Grey Literature Report A bi-monthly publication of the New York Academy of Medicine, the GLR includes listings of recently published reports in health science and public health. The archives are tagged with MeSH terms and are searchable.
• BIOSIS Previews Meetings! BIOSIS Previews includes proceedings of many meetings that may not be electronically available elsewhere.
• ProQuest Dissertations & Theses Global A central authoritative source for locating doctoral dissertations and master's theses. Provides full text for most indexed dissertations from 1990-present. Includes theses and dissertations from the Harvard T.H. Chan School of Public Health, Harvard Medical School, and Harvard School of Dental Medicine.
• greylitsearcher A web-based tool for performing systematic and transparent searches of organizational websites

Identifying sources for grey literature and being sure you've done enough is a challenge. The Canadian Agency for Drugs and Technologies in Health (CADTH) feels your pain and has produced a checklist that might help guide your grey research. The Grey Matters checklist provides an organized source of health technology assessment sites, regulatory agencies, trial registries, and other databases in a form that can help ensure the completeness of you search.

Clinical Trial Registries

• ClinicalTrials.gov
• European Union Clinical Trials Registry
• ISRCTN registry
• International Clinical Trials Registry Platform  (ICTRP)

When you search Cochrane Library/Trials , you will see results from both ClinicalTrials.gov and ICTRP. 

More information about trial registries and solving the problems associated with searching them is available through this site: Medical and health-related trials registers and research registers which is maintained by Julie Glanville and Carol Lefebvre and hosted by the York Health Economics Health Consortium.

• << Previous: Review Protocols
• Next: Methodology Filters >>
• Last Updated: Feb 26, 2024 3:17 PM
• URL: https://guides.library.harvard.edu/meta-analysis

• Reserve a study room
• Library Account
• Undergraduate Students
• Graduate Students
• Faculty & Staff

How to Conduct a Literature Review (Health Sciences and Beyond)

• What is a Literature Review?
• Developing a Research Question
• Selection Criteria

Popular Databases

Finding additional databases.

• Database Search
• Documenting Your Search
• Organize Key Findings
• Reference Management

Below is a list of the most commonly used databases. Select one or more that align with the scope of your research discipline.

• ERIC (Education Resources Information Center)

We also recommend that you consult your discipline's  research guide  for additional database suggestions. Below are a few research guides you may find useful, or you can browse our full list of research guides .

• Dentistry by Erica Brody Last Updated Aug 14, 2023 55 views this year
• Nursing by Roy Brown Last Updated May 29, 2024 2683 views this year
• Medicine by John Cyrus Last Updated Nov 9, 2023 528 views this year
• Pharmacy by Erica Brody Last Updated Jun 4, 2024 1025 views this year
• << Previous: Selection Criteria
• Next: Database Search >>
• Last Updated: Mar 15, 2024 12:22 PM
• URL: https://guides.library.vcu.edu/health-sciences-lit-review

• - Google Chrome

Intended for healthcare professionals

• Access provided by Google Indexer
• My email alerts
• BMA member login
• Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Search form

• Advanced search
• Search responses
• Search blogs
• Medical bibliographic...

Medical bibliographic databases

• Related content
• Peer review
• Lukas A Holzer , medical student 1 ,
• John E Eyers , health information consultant 2
• 1 Medical University of Vienna, Vienna, Austria
• 2 Railway Cottage, Kinnersley

Lukas Holzer and John Eyers describe how to use databases to search the medical literature

• Download figure
• Open in new tab
• Download powerpoint

There are many types of medical databases. They cover medical and scientific literature, morbidity and mortality statistics, therapeutic regimens, patient records, x ray films, and reviews of evidence based medicine. Most undergraduate medical courses include an introduction to bibliographic databases, but this is often given at an inappropriate time such as the beginning of the first year and is usually too brief to be of much use. In medicine such databases are essential search tools for research and clinical practice, providing the latest scientific insights and evidence based medicine.

Most databases index the journal literature, but some interdisciplinary databases also include books, conference reports, newsletters, and other forms of publication. As tens of thousands of biomedical journals are published worldwide, most bibliographic databases will not index the total literature. Selection criteria are usually based on a number of factors, including whether the journal is peer reviewed (the process in which an article is refereed impartially by other experts in the field to ensure that it meets the journal's standard); the number of times it is cited (the number of times an article is referred to by authors) in the literature; the impact factor (the number of citations the articles in a journal receive in a given year or years divided by the number of articles published); how long the journal has been established; and the language of publication. Most databases contain the article citations and selected abstracts, with occasional full text or links to the full text sources.

General medical databases

Medline/Pubmed ( www.ncbi.nlm.nih.gov/sites/entrez?db=pubmed ) is the most widely used literature database of life sciences and biomedicine, with an emphasis on English language journals. It contains about 5000 selected journals with records of more than 15 million articles from 1950 to the present. It is excellent for clinical medicine with most specialties represented, but it also covers non-clinical journals such as healthcare research, health economics and psychology. It is free to all. This is one of a large number of other health related databases from the US National Library of Medicine (see www.nlm.nih.gov/databases for others).

EMBASE ( www.embase.com/ ) is a biomedical and pharmacological database produced by Elsevier which contains over 11 million records of articles starting from 1974. Each record is fully indexed and covers over 5000 biomedical journals from 70 countries. It includes more non-English biomedical journals than Medline/Pubmed. It is a subscription only database available mostly through university or medical libraries.

Scopus ( www.scopus.com/scopus/home.url ) is a new database and web based research tool provided by Elsevier. It is the largest collection of medical, natural, engineering, and social sciences articles worldwide, containing 25 million abstracts from 14 000 journals. Searches offer many features including the number of times an article has been referred to and by which authors. This is a subscription only database and may be available through medical school libraries.

Specialist databases

Popline ( http://db.jhuccp.org/popinform/basic.html ) is a database of reproductive and sexual health, fertility, family planning, and population issues. This database is produced from the combined resources of a number of population centres in the United States and contains journal articles, reports, unpublished material, books, and conference papers. Where available, it provides links to full text.

The Cochrane Library ( http://www3.interscience.wiley.com/cgi-bin/mrwhome/106568753/HOME ) was developed by the worldwide Cochrane Collaboration ( www.cochrane.org/ ) and is the most important resource for evidence based medicine, containing databases of systematic reviews, clinical trials, economic evaluations, and methodologies. The most important is the database of systematic reviews which summarises and interprets the results of medical research, in particular randomised controlled trials; the reviews are available with full text. This resource is subscription based, but many countries now offer free access (see http://www3.interscience.wiley.com/cgi-bin/mrwhome/106568753/AccessCochraneLibrary.html . Access is also available for low income countries through HINARI (the Health Internetwork Access to Research Initiative; www.who.int/hinari/en/ ).

Trip Database ( www.tripdatabase.com/index.html ) is a useful free alternative to the Cochrane Library, providing information on evidence based medicine from a large number of resources: major journals, textbooks, evidence based medicine websites, as well as short abstracts from the Cochrane Library. It is a good quick starting point for material on evidence based medicine.

Clinical Evidence ( http://clinicalevidence.bmj.com/ceweb/index.jsp ), published by the BMJ Group, provides summaries to improve clinical decisions and patient care based on systematic reviews of evidence. Regularly updated when new evidence becomes available, it is a subscription service but is available to low income countries free or at a reduced rate through HINARI.

ISI Web of Knowledge ( http://isiwebofknowledge.com/currentuser_wokhome/cu_aboutwok ) provides access to a number of multidisciplinary databases. It includes Web of Science (covering about 8700 leading journals in science, technology, social sciences, arts, and humanities) and Journal Citation Reports. A special feature of Web of Science is the ability to run “citation” searches which identify recent papers that have cited a known reference. It is a subscription database to which many university libraries provide access.

Global Health ( www.cabi.org/datapage.asp?iDocID=169 ), produced by CAB International, is dedicated to international public health research and practice from 1973 and covers 3500 journals as well as books, conference proceedings, and reports. Excellent for its coverage of health in the developing world, its sister database Global Health Archive covers the history of public health and research from the end of the 19th century to 1972.

Virtual Health Library ( www.virtualhealthlibrary.org/php/index.php?lang=en ) contains many health databases intended primarily for a South American audience. These include LILACS, a database of Latin American medical journals many of which are not covered by Pubmed or EMBASE, and databases of bioethics, adolescent health, and environmental disasters, with access to full text journals not provided for elsewhere (Scielo; www.scielo.org/php/index.php?lang=en ).

WHO Global Health Library ( www.globalhealthlibrary.net/php/index.php ). Some of the regional offices of the World Health Organization have produced their own Index Medicus which includes regional medical journals not indexed in the Western bibliographic databases. This forms a unified database from Africa, the eastern Mediterranean, the Pan American region, and the western Pacific. Other databases from the European and the South East Asian regions are also available from this site.

Search engines

Search engines such as Google ( www.google.com ) are used universally to access internet information. However, since there are few controls on the reliability of information on the web, they should be used with extreme caution to find medical information. Google Scholar ( http://scholar.google.co.uk ) is a freely accessible web search engine that attempts to provide access to free online research publications from the world's largest scientific publishers. It also covers websites and journal sources in various languages. It is useful for checking whether a reference is available free in full text online and for identifying articles that have cited a particular paper.

MedHunt ( www.hon.ch/MedHunt ), produced by the Health On the Net Foundation, provides access to evaluated sites to improve the quality of internet searching. This is the most reliable search engine.

How to search databases: basic principles

Most databases are searched by typing in keywords that are identified in the reference citation, usually in the title or abstract of the paper. This is called natural language or free text searching. This method immediately presents a problem for searching effectively, as it requires the searcher to take account of synonyms (words meaning the same but spelt differently), word endings (singulars and plurals and so on), and US or English spellings, which different authors will often use in an interchangeable way to describe the same topic.

To take a simple example, a search of the literature on the treatment of anaemia in children would require the following terms to be searched: treatment, anaemia, and children. Combining just these three terms would not be an effective search as it would miss many papers. To increase the number of papers retrieved, the following need to be searched: treat, treated, treatment, and treating as well as synonyms such as therapy, therapeutic, and the names of treatments themselves such as the terms iron or folic. For anaemia we would also need to account for the US spelling anemia as well as word endings such as anaemic or anemic. Finally, the term children would perhaps need to encompass infant(s) as well as child and infantile.

Most databases allow use of a truncation or stemming device (usually a * or $) to retrieve variable word endings, which makes searching easier. Thus, treat* would retrieve treat, treats, treated, treatment, treating, and so on. Taking account of these variants can make a huge difference to the number of papers retrieved; this can be crucial where few papers are published on the topic, as it would be important to find all the papers available.

Once we have decided on the terms to be searched with all the variants, word endings, and synonyms listed, the next stage is to decide on the relation between the terms. In the example above, each of the related terms would need to be added together using the Boolean operator OR: to retrieve all possible word endings we would search for the term “anaem* OR anem*”. Once all the other terms in this example have been searched in a similar way using OR, we should have four separate search statements. The final part of the search would be to combine these (using the search statement numbers) to narrow the focus to the result we want. To do this we would use the Boolean operator AND. So using the search statement number we would get the following final combination: 1 (treatment search) AND 2 (anaemia search) AND 3 (children or infants search). Once the results have been looked at most databases allow further terms to be added to narrow the search further or fewer terms used to broaden the search. Limits can also be made on age group, language of the article, type of article, and so on.

Combining search terms

Some databases use controlled language or thesaurus searching. Here searches make use of the index terms that describe its contents, which the database producer assigns to each article at the input stage. Medline/Pubmed has a powerful system called MeSH (medical subject headings; see www.nlm.nih.gov/pubs/factsheets/mesh.html ) which other databases, for example, the Cochrane Library, are beginning to adopt to improve the quality of searches. The advantage of this system is that there is no need to think of synonyms or word endings or US or English spelling as the unique thesaurus terms are usually sufficient to retrieve all variants. Learning to use this system can be difficult but the help screens of Pubmed or EMBASE give guidance on how to use it, and many medical libraries have published guides which are easily found using Google.

Access to full text information

Once a set of relevant references has been found from a database search the problem arises of where to find the full text. Some databases provide a link to the full text, but this is not always free and can be expensive to download from a publisher's website. You are fortunate if you have access to a large medical library with many medical journals (especially if they are available online), but it is unlikely that you would find all the papers on your list available in your library. A number of options are available. The first is to see if the journal is one of the open access journals (box). You could email the author of the paper as the email address of one author is usually given in a reference citation; do not be tempted to email all authors for copies of their papers as authors soon tire of such requests. Finally, you could see if the paper is available somewhere on the internet by using a search engine such as Google. Typing in the title of the paper with quotation marks at the beginning and end of the title might produce a full text copy of the article free—for example, “Treatment of anaemic children in a paediatric hospital”.

For people in poor countries another option is access to HINARI ( www.who.int/hinari/en ), which makes available full text articles from thousands of biomedical journals free or at low cost (depending on the country you are from) as well as databases and other resources.

Some open access journals

Bioline ( www.bioline.org.br )

BioMed Central ( www.biomedcentral.com )

BMJ ( www.bmj.com )

Directory of Open Access Journals ( www.doaj.org )

Free Medical Journals ( www.freemedicaljournals.com )

HighWire Press ( http://highwire.stanford.edu/lists/freeart.dtl )

Institute of Tropical Medicine Library Antwerp ( http://lib.itg.be/journals.htm )

Open Journals Publishing (Africa) ( www.openjournals.net )

PubMed Central ( www.pubmedcentral.nih.gov )

SciELO ( www.scielo.org/php/index.php?lang=en )

Further reading

Hinari training materials ( www.who.int/hinari/training/en ). A useful series of information skills modules, including a comprehensive guide to searching PubMed

Eyers JE. Searching bibliographic databases effectively. Health Policy and Planning 1998;13:339-42. ( http://heapol.oxfordjournals.org/cgi/reprint/13/3/339 )

Guide to searching the Cochrane Library—in various languages. ( www3.interscience.wiley.com/cgi-bin/mrwhome/106568753/HELP_Cochrane.html?CRETRY=1&SRETRY=0 )

Al-Ubaydli M. Using search engines to find online medical information. PLoS Med 2005;2(9):e228, doi: 10.1371/journal.pmed.0020228

Originally published as: Student BMJ 2008;16:366

Literature Reviews & Search Strategies

• Defining the Literature Review
• Types of Literature Reviews

Use Multiple Databases

• Search Strategies
• Organizing Your Literature
• Books: Research Design & Scholarly Writing
• Recommended Tutorials

While not every literature search you undertake will be for a systematic review, the Cochrane Handbook's statement that "a search of MEDLINE alone is not considered adequate" holds true for almost all literature reviews. You need to go beyond one database to get a more comprehensive picture of your topic and to minimize selection bias. 

There are A LOT of databases that you could potential search for academic/scholarly articles to use in your literature review. We recommend focusing on resources that specializes in academic sources (ie databases), rather than a general search tool like Google because a lot of scholarly literature is still not discoverable on the open web and when it is you'll often hit a paywall and have to head to a subscription database available through the library to read the full article any way.

All our databases are listed on the A-Z Databases List , these are a few, often recommended, examples:

• << Previous: Types of Literature Reviews
• Next: Search Strategies >>
• Last Updated: Jun 14, 2023 11:18 AM
• URL: https://mcphs.libguides.com/litreviews

Health (Nursing, Medicine, Allied Health)

• Find Articles/Databases
• Reference Resources
• Evidence Summaries & Clinical Guidelines
• Drug Information
• Health Data & Statistics
• Patient/Consumer Facing Materials
• Images and Streaming Video
• Grey Literature
• Mobile Apps & "Point of Care" Tools
• Tests & Measures This link opens in a new window
• Citing Sources
• Selecting Databases
• Framing Research Questions
• Crafting a Search
• Narrowing / Filtering a Search
• Expanding a Search
• Cited Reference Searching
• Saving Searches
• Term Glossary
• Critical Appraisal Resources
• What are Literature Reviews?
• Conducting & Reporting Systematic Reviews
• Finding Systematic Reviews
• Tutorials & Tools for Literature Reviews
• Finding Full Text

What are Systematic Reviews? (3 minutes, 24 second YouTube Video)

Systematic Literature Reviews: Steps & Resources

These steps for conducting a systematic literature review are listed below . 

Also see subpages for more information about:

• The different types of literature reviews, including systematic reviews and other evidence synthesis methods
• Tools & Tutorials

Literature Review & Systematic Review Steps

• Develop a Focused Question
• Scope the Literature  (Initial Search)
• Refine & Expand the Search
• Limit the Results
• Download Citations
• Abstract & Analyze
• Create Flow Diagram
• Synthesize & Report Results

1. Develop a Focused   Question 

Consider the PICO Format: Population/Problem, Intervention, Comparison, Outcome

Focus on defining the Population or Problem and Intervention (don't narrow by Comparison or Outcome just yet!)

"What are the effects of the Pilates method for patients with low back pain?"

Tools & Additional Resources:

• PICO Question Help
• Stillwell, Susan B., DNP, RN, CNE; Fineout-Overholt, Ellen, PhD, RN, FNAP, FAAN; Melnyk, Bernadette Mazurek, PhD, RN, CPNP/PMHNP, FNAP, FAAN; Williamson, Kathleen M., PhD, RN Evidence-Based Practice, Step by Step: Asking the Clinical Question, AJN The American Journal of Nursing : March 2010 - Volume 110 - Issue 3 - p 58-61 doi: 10.1097/01.NAJ.0000368959.11129.79

2. Scope the Literature

A "scoping search" investigates the breadth and/or depth of the initial question or may identify a gap in the literature. 

Eligible studies may be located by searching in:

• Background sources (books, point-of-care tools)
• Article databases
• Trial registries
• Grey literature
• Cited references
• Reference lists

When searching, if possible, translate terms to controlled vocabulary of the database. Use text word searching when necessary.

Use Boolean operators to connect search terms:

• Combine separate concepts with AND  (resulting in a narrower search)
• Connecting synonyms with OR  (resulting in an expanded search)

Search:  pilates AND ("low back pain"  OR  backache )

Video Tutorials - Translating PICO Questions into Search Queries

• Translate Your PICO Into a Search in PubMed (YouTube, Carrie Price, 5:11) 
• Translate Your PICO Into a Search in CINAHL (YouTube, Carrie Price, 4:56)

3. Refine & Expand Your Search

Expand your search strategy with synonymous search terms harvested from:

• database thesauri
• reference lists
• relevant studies

Example: 

(pilates OR exercise movement techniques) AND ("low back pain" OR backache* OR sciatica OR lumbago OR spondylosis)

As you develop a final, reproducible strategy for each database, save your strategies in a:

• a personal database account (e.g., MyNCBI for PubMed)
• Log in with your NYU credentials
• Open and "Make a Copy" to create your own tracker for your literature search strategies

4. Limit Your Results

Use database filters to limit your results based on your defined inclusion/exclusion criteria.  In addition to relying on the databases' categorical filters, you may also need to manually screen results.  

• Limit to Article type, e.g.,:  "randomized controlled trial" OR multicenter study
• Limit by publication years, age groups, language, etc.

NOTE: Many databases allow you to filter to "Full Text Only".  This filter is  not recommended . It excludes articles if their full text is not available in that particular database (CINAHL, PubMed, etc), but if the article is relevant, it is important that you are able to read its title and abstract, regardless of 'full text' status. The full text is likely to be accessible through another source (a different database, or Interlibrary Loan).  

• Filters in PubMed
• CINAHL Advanced Searching Tutorial

5. Download Citations

Selected citations and/or entire sets of search results can be downloaded from the database into a citation management tool. If you are conducting a systematic review that will require reporting according to PRISMA standards, a citation manager can help you keep track of the number of articles that came from each database, as well as the number of duplicate records.

In Zotero, you can create a Collection for the combined results set, and sub-collections for the results from each database you search.  You can then use Zotero's 'Duplicate Items" function to find and merge duplicate records.

• Citation Managers - General Guide

6. Abstract and Analyze

• Migrate citations to data collection/extraction tool
• Screen Title/Abstracts for inclusion/exclusion
• Screen and appraise full text for relevance, methods, 
• Resolve disagreements by consensus

Covidence is a web-based tool that enables you to work with a team to screen titles/abstracts and full text for inclusion in your review, as well as extract data from the included studies.

• Covidence Support
• Critical Appraisal Tools
• Data Extraction Tools

7. Create Flow Diagram

The PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) flow diagram is a visual representation of the flow of records through different phases of a systematic review.  It depicts the number of records identified, included and excluded.  It is best used in conjunction with the PRISMA checklist .

Example from: Stotz, S. A., McNealy, K., Begay, R. L., DeSanto, K., Manson, S. M., & Moore, K. R. (2021). Multi-level diabetes prevention and treatment interventions for Native people in the USA and Canada: A scoping review. Current Diabetes Reports, 2 (11), 46. https://doi.org/10.1007/s11892-021-01414-3

• PRISMA Flow Diagram Generator (ShinyApp.io, Haddaway et al. )
• PRISMA Diagram Templates  (Word and PDF)
• Make a copy of the file to fill out the template
• Image can be downloaded as PDF, PNG, JPG, or SVG
• Covidence generates a PRISMA diagram that is automatically updated as records move through the review phases

8. Synthesize & Report Results

There are a number of reporting guideline available to guide the synthesis and reporting of results in systematic literature reviews.

It is common to organize findings in a matrix, also known as a Table of Evidence (ToE).

• Reporting Guidelines for Systematic Reviews
• Download a sample template of a health sciences review matrix  (GoogleSheets)

Steps modified from: 

Cook, D. A., & West, C. P. (2012). Conducting systematic reviews in medical education: a stepwise approach.   Medical Education , 46 (10), 943–952.

• << Previous: Critical Appraisal Resources
• Next: What are Literature Reviews? >>
• Last Updated: May 31, 2024 10:32 AM
• URL: https://guides.nyu.edu/health

News alert: UC Berkeley has announced its next university librarian

Secondary menu

• Log in to your Library account
• Hours and Maps
• Connect from Off Campus
• UC Berkeley Home

Search form

Searching the public health & medical literature more effectively: more sources: databases, systematic reviews, grey literature.

• Getting Started
• Articles: Searching PubMed This link opens in a new window
• More Sources: Databases, Systematic Reviews, Grey Literature
• Organize Citations & Search Strategies
• Literature Review Help
• Need More Help?

Additional Databases: Beyond PubMed

• Embase Search Exercise
• PsycInfo Exercise

• Web of Science Citation Analysis Exercise
• the ability to use iteration in developing search strategies,
• any indexing,
• bulk downloading or saving of records,
• a complete description of its contents.

Looking for older articles? Here are some tips from the US National Library of Medicine on where to search for old (generally pre-1960s) medical/health articles .

Library Catalog

Use the library catalog to find books, reports, videos, etc. on your topic. Books, while not often where original research is published, can often provide an overview of a topic and get you started with some key concepts.

• UC Library Search Search for and borrow items from UCB, all 10 UC libraries, and beyond.

Online Newspapers & News : Guide to news sources worldwide. Includes major dailies, local, alternative, and ethnic news sources, and science & health specialty news sources.

Low & Middle Income (LMIC) Search Hedge

If you are searching for information on, or studies that were done in, LMICs, you are welcome to copy and paste this list of terms into the search box of whatever database(s) you are using. Please be aware the list is almost 600 words long.

USA Search Hedge

If you are searching for information on, or studies that were done in, the USA, you are welcome to copy and paste this list of terms into the search box of whatever database(s) you are using. Please be aware the list is about 175 words long.

Qualitative or Quantitative Studies Searches

The following include links with strings of search terms -- both thesaurus terms, such as MeSH, and keywords -- useful to find qualitative research studies in several databases. These include PubMed, Embase, CINAHL, PsycInfo, and more.

Finding Qualitative Research Articles (University of Washington Health Sciences Library).

Qualitative Research: Filters (InterTASC Information Specialists' Sub-Group, University of York).

To find quantitative studies, use filters such as Clinical Trials, and/or use thesaurus terms or keywords that describe the technique(s) you are interested in: Survival Analysis, Linear Models, Analysis of variance, etc.

Researching Chemical Substances?

Use this Chemical Search Guide (docx) to find chemical and substance information for your chemical search. The guide includes links to resource suggestions for locating various aspects of your chemical or substance.

Grey Literature

Grey Literature generally refers to publications not produced by commercial publishers, including reports (pre-prints, preliminary progress and advanced reports, technical reports, market research reports, etc.), theses, conference proceedings, and other documents. They are often produced by government entities, research institutions, or NGOs/IGOs.

The Library's  Public Health Subject Guides lists guides by topic. Each guide consists of annotated lists of organizations, agencies, databases, statistical/data sources, and publications. Topics include: 

• Environmental Health
• Food/Nutrition
• Maternal and Child Health
• Statistical/Data Resources

and many more.

Google and other search engines can be useful for finding grey literature. Improve your search using:

• Quotes for phrase searching: "social marketing"
• Site: to specify a particular site or domain: "social marketing" site:.org (for a domain search); "social marketing" site:npin.cdc.gov (for a specific site search)
• Boolean search statements (eg, OR): ("social marketing" OR "audience segmentation")
• Think Tank Search (Harvard) Think Tank Search will search the websites of over 1200 think tanks, research centers, and NGOs.
• Policy Commons Requires free registration. Search engine for "fact-based research from the world’s leading policy experts, think tanks, IGOs and NGOs." Advanced search includes geography, language, and document type options.
• MetaLib Use MetaLib to find US Government produced grey literature on all topics.

• Grey Matters: a practical tool for searching health-related grey literature Ensure the retrieval of all relevant health technology assessment (HTA), government, and evidence-based agency reports that may not be indexed in bibliographic databases; document the grey literature search process, thereby increasing transparency and the potential for reproducibility; and ensure that grey literature searching is done in a comprehensive way. From Canadian Agency for Drugs and Technologies in Health.

Doing Systematic (& Similar) Reviews

Systematic Reviews should address a clearly formulated, relatively narrowly focused question and use systematic and explicit methods to identify, select, and assess relevant research.

Before you embark on a systematic review, please understand that this could easily be a one year or more project. Here is a decision tree ( source ) to help you decide is a systematic, or other type or review, is appropriate. The SPARK Tool to prioritise questions for systematic reviews in health policy and systems research can help you decide if a systematic review is appropriate and needed. If you do decide to conduct a systematic review, please register your protocol .

PRISMA for searching : 16 item checklist to ensure your systematic review database searching complies with standards.

You may wish to consider conducting another type of literature review; see this table for information on several types of reviews (eg, scoping review, mapping review, rapid review, etc.). (Table reproduced from A typology of reviews: an analysis of 14 review types and associated methodologies ).

These articles may also be helpful:

How to conduct a systematic review from beginning to end (from Covidence; easy to read summary of the 7 steps).

Five steps to conducting a systematic review . Khan KS, Kunz R, Kleijnen J, Antes G. Journal of the Royal Society of Medicine. 2003 Mar;96(3):118-21.

A Guide to Conducting a Standalone Systematic Literature Review . Okoli C. Communications of the Association for Information Systems 2015; 37(1): 879-910.

Incorporating Judgments about Study Quality into Research Syntheses . (pdf). Valentine JC. Chapter 7 of Cooper, Harris, Hedges, Larry V., Valentine, Jeffrey C. The Handbook of Research Synthesis and Meta-Analysis . New York : Russell Sage Foundation, 2019. Excellent review of what to consider, and what not to consider, when judging study quality.

Performing Rapid Reviews . King VJ et al. Systematic Reviews 2022; 11:151. Highly recommended if you are considering performing a rapid review.

Rapid Review Guidebook . Rapid reviews differ from systematic reviews in that the process is tailored for a shorter timeline, but it is still important to use rigorous methodology to ensure that the best available research evidence is used in decision making. The National Collaborating Centre of Methods and Tools (Canada) has developed a Rapid Review Guidebook that details each step in the process. It also includes information on how to structure the written report, and what to include in each section.

The difference between a systematic review and a scoping review (from Covidence).

Systematic vs Scoping Review: What's the Difference? (5 minute video, Carrie Price, Health Professions Librarian, Towson University).

Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach . Munn Z et al. BMC Medical Research Methodology volume 18, Article number: 143 (2018).

PRISMA for Scoping Reviews . Includes a checklist with 20 essential reporting items and 2 optional items to include when completing a scoping review, as well as one-page tip sheets on each item.

Recommendations for the extraction, analysis, and presentation of results in scoping reviews . Pollock D et al. JBI Evidence Synthesis. 2022 Sep 8. Recommended reading if you are considering conducting a scoping review.

Guidance to best tools and practices for systematic reviews . "Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses." British Journal of Pharmacology , 1–31, 2023.

If you are faced with a large number of abstracts to screen, see Best practice guidelines for abstract screening large-evidence systematic reviews and meta-analyses ; Res Syn Meth. 2019; 10: 330–342. https://doi.org/10.1002/jrsm.1354.

An article on the importance of looking at the science behind the articles you review when assessing quality:  Challenges and recommendations on the conduct of systematic reviews of observational epidemiologic studies in environmental and occupational health Arroyave WD, et al. Journal of exposure science & environmental epidemiology 2021; 31(1):21-30.

Consult the Cochrane Handbook for Systematic Reviews of Interventions (2nd edition) for a very thorough discussion of the systematic review process.

The UC Berkeley Library licenses Covidence , a tool to help you with your systematic reviews. In Covidence, you can:

• import citations;
• screen titles and abstracts;
• upload article PDFs;
• screen full text;
• create forms for critical appraisal;
• perform risk of bias tables;
• complete data extraction, and;
• export a PRISMA flowchart summarizing your review process.

As an institutional member, our users have priority access to Covidence support. Our license allows unlimited simultaneous reviews, and you can add people who are not affiliated with UCB. To access Covidence using the UC Berkeley institutional account , start at this page and follow the instructions. Many tutorials, help pages, webinar recordings, and more may be found in the Covidence Knowledge Base ; detailed Step-by-Step webinars are available on YouTube.

Other systematic review tools besides Covidence include PICO Portal , Rayyan , Sysrev , and more.

How long will it take to complete a systematic review? Use the PredicTER tool to find out!

And, some thoughts on assigning systematic reviews as course assignments : Is there better teaching opportunity than assigning systematic reviews in a semester?

Systematic Review and Meta-Analysis: Free Online Courses

• Introduction to conducting systematic reviews (Cochrane)
• Systematic Reviews and Meta-Analysis: A Campbell Collaboration Online Course
• CERTaIN: Knowledge Synthesis: Systematic Reviews and Clinical Decision Making
• The Art and Science of Searching in Systematic Reviews (NUS)
• Introduction to Systematic Reviews (Stanford)
• Introduction to Systematic Review and Meta-Analysis (Johns Hopkins)

(Courtesy of Asad Naveed , University of Toronto)

Finding Systematic Reviews

Systematic reviews seek to collate all evidence that fits pre-specified eligibility criteria in order to address a specific research question. They aim to minimize bias by using explicit, systematic methods. (from Cochrane Handbook for Systematic Reviews of Interventions )

A systematic review is a review that reports or includes the following: i) research question ii) sources that were searched, with a reproducible search strategy (naming of databases, naming of search platforms/engines, search date and complete search strategy) iii) inclusion and exclusion criteria iv) selection (screening) methods v) critically appraises and reports the quality/risk of bias of the included studies vi) information about data analysis and synthesis that allows the reproducibility of the results (from Krnic Martinic, M., Pieper, D., Glatt, A. et al. Definition of a systematic review used in overviews of systematic reviews, meta-epidemiological studies and textbooks . BMC Med Res Methodol 19, 203 (2019) doi:10.1186/s12874-019-0855-0)

• Public Health + Methodologically sound studies from top journals in a browsable, searchable database
• Health Evidence Methodologically-sound reviews of health promotion and public health interventions
• Health Systems Evidence Continuously updated repository of syntheses of research evidence about governance, financial and delivery arrangements within health systems, and about implementation strategies that can support change in health systems. Health Systems Evidence also contains economic evaluations in these same domains, descriptions of health system reforms (with links to syntheses and economic evaluations when possible), and descriptions of health systems.
• Database of promoting health effectiveness reviews (DoPHER) Systematic and non-systematic reviews of effectiveness in health promotion and public health worldwide
• Campbell Collaboration Systematic Reviews Systematic reviews, evidence and gap maps, and methods research papers. Topics include social welfare, education, crime & justice, disability, and more. Use the link above or use the search feature on the Campbell website .
• << Previous: Articles: Searching PubMed
• Next: Organize Citations & Search Strategies >>
• Last Updated: May 22, 2024 3:44 PM
• URL: https://guides.lib.berkeley.edu/publichealth/litsearch

Loading metrics

Open Access

Peer-reviewed

Research Article

Functional connectivity changes in the brain of adolescents with internet addiction: A systematic literature review of imaging studies

Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Software, Validation, Visualization, Writing – original draft, Writing – review & editing

Affiliation Child and Adolescent Mental Health, Department of Brain Sciences, Great Ormond Street Institute of Child Health, University College London, London, United Kingdom

Roles Conceptualization, Supervision, Validation, Writing – review & editing

* E-mail: [email protected]

Affiliation Behavioural Brain Sciences Unit, Population Policy Practice Programme, Great Ormond Street Institute of Child Health, University College London, London, United Kingdom

• Max L. Y. Chang, 
• Irene O. Lee

• Published: June 4, 2024
• https://doi.org/10.1371/journal.pmen.0000022
• Peer Review
• Reader Comments

Internet usage has seen a stark global rise over the last few decades, particularly among adolescents and young people, who have also been diagnosed increasingly with internet addiction (IA). IA impacts several neural networks that influence an adolescent’s behaviour and development. This article issued a literature review on the resting-state and task-based functional magnetic resonance imaging (fMRI) studies to inspect the consequences of IA on the functional connectivity (FC) in the adolescent brain and its subsequent effects on their behaviour and development. A systematic search was conducted from two databases, PubMed and PsycINFO, to select eligible articles according to the inclusion and exclusion criteria. Eligibility criteria was especially stringent regarding the adolescent age range (10–19) and formal diagnosis of IA. Bias and quality of individual studies were evaluated. The fMRI results from 12 articles demonstrated that the effects of IA were seen throughout multiple neural networks: a mix of increases/decreases in FC in the default mode network; an overall decrease in FC in the executive control network; and no clear increase or decrease in FC within the salience network and reward pathway. The FC changes led to addictive behaviour and tendencies in adolescents. The subsequent behavioural changes are associated with the mechanisms relating to the areas of cognitive control, reward valuation, motor coordination, and the developing adolescent brain. Our results presented the FC alterations in numerous brain regions of adolescents with IA leading to the behavioural and developmental changes. Research on this topic had a low frequency with adolescent samples and were primarily produced in Asian countries. Future research studies of comparing results from Western adolescent samples provide more insight on therapeutic intervention.

Citation: Chang MLY, Lee IO (2024) Functional connectivity changes in the brain of adolescents with internet addiction: A systematic literature review of imaging studies. PLOS Ment Health 1(1): e0000022. https://doi.org/10.1371/journal.pmen.0000022

Editor: Kizito Omona, Uganda Martyrs University, UGANDA

Received: December 29, 2023; Accepted: March 18, 2024; Published: June 4, 2024

Copyright: © 2024 Chang, Lee. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the paper and its Supporting information files.

Funding: The authors received no specific funding for this work.

Competing interests: The authors have declared that no competing interests exist.

Introduction

The behavioural addiction brought on by excessive internet use has become a rising source of concern [ 1 ] since the last decade. According to clinical studies, individuals with Internet Addiction (IA) or Internet Gaming Disorder (IGD) may have a range of biopsychosocial effects and is classified as an impulse-control disorder owing to its resemblance to pathological gambling and substance addiction [ 2 , 3 ]. IA has been defined by researchers as a person’s inability to resist the urge to use the internet, which has negative effects on their psychological well-being as well as their social, academic, and professional lives [ 4 ]. The symptoms can have serious physical and interpersonal repercussions and are linked to mood modification, salience, tolerance, impulsivity, and conflict [ 5 ]. In severe circumstances, people may experience severe pain in their bodies or health issues like carpal tunnel syndrome, dry eyes, irregular eating and disrupted sleep [ 6 ]. Additionally, IA is significantly linked to comorbidities with other psychiatric disorders [ 7 ].

Stevens et al (2021) reviewed 53 studies including 17 countries and reported the global prevalence of IA was 3.05% [ 8 ]. Asian countries had a higher prevalence (5.1%) than European countries (2.7%) [ 8 ]. Strikingly, adolescents and young adults had a global IGD prevalence rate of 9.9% which matches previous literature that reported historically higher prevalence among adolescent populations compared to adults [ 8 , 9 ]. Over 80% of adolescent population in the UK, the USA, and Asia have direct access to the internet [ 10 ]. Children and adolescents frequently spend more time on media (possibly 7 hours and 22 minutes per day) than at school or sleeping [ 11 ]. Developing nations have also shown a sharp rise in teenage internet usage despite having lower internet penetration rates [ 10 ]. Concerns regarding the possible harms that overt internet use could do to adolescents and their development have arisen because of this surge, especially the significant impacts by the COVID-19 pandemic [ 12 ]. The growing prevalence and neurocognitive consequences of IA among adolescents makes this population a vital area of study [ 13 ].

Adolescence is a crucial developmental stage during which people go through significant changes in their biology, cognition, and personalities [ 14 ]. Adolescents’ emotional-behavioural functioning is hyperactivated, which creates risk of psychopathological vulnerability [ 15 ]. In accordance with clinical study results [ 16 ], this emotional hyperactivity is supported by a high level of neuronal plasticity. This plasticity enables teenagers to adapt to the numerous physical and emotional changes that occur during puberty as well as develop communication techniques and gain independence [ 16 ]. However, the strong neuronal plasticity is also associated with risk-taking and sensation seeking [ 17 ] which may lead to IA.

Despite the fact that the precise neuronal mechanisms underlying IA are still largely unclear, functional magnetic resonance imaging (fMRI) method has been used by scientists as an important framework to examine the neuropathological changes occurring in IA, particularly in the form of functional connectivity (FC) [ 18 ]. fMRI research study has shown that IA alters both the functional and structural makeup of the brain [ 3 ].

We hypothesise that IA has widespread neurological alteration effects rather than being limited to a few specific brain regions. Further hypothesis holds that according to these alterations of FC between the brain regions or certain neural networks, adolescents with IA would experience behavioural changes. An investigation of these domains could be useful for creating better procedures and standards as well as minimising the negative effects of overt internet use. This literature review aims to summarise and analyse the evidence of various imaging studies that have investigated the effects of IA on the FC in adolescents. This will be addressed through two research questions:

• How does internet addiction affect the functional connectivity in the adolescent brain?
• How is adolescent behaviour and development impacted by functional connectivity changes due to internet addiction?

The review protocol was conducted in line with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (see S1 Checklist ).

Search strategy and selection process

A systematic search was conducted up until April 2023 from two sources of database, PubMed and PsycINFO, using a range of terms relevant to the title and research questions (see full list of search terms in S1 Appendix ). All the searched articles can be accessed in the S1 Data . The eligible articles were selected according to the inclusion and exclusion criteria. Inclusion criteria used for the present review were: (i) participants in the studies with clinical diagnosis of IA; (ii) participants between the ages of 10 and 19; (iii) imaging research investigations; (iv) works published between January 2013 and April 2023; (v) written in English language; (vi) peer-reviewed papers and (vii) full text. The numbers of articles excluded due to not meeting the inclusion criteria are shown in Fig 1 . Each study’s title and abstract were screened for eligibility.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pmen.0000022.g001

Quality appraisal

Full texts of all potentially relevant studies were then retrieved and further appraised for eligibility. Furthermore, articles were critically appraised based on the GRADE (Grading of Recommendations, Assessment, Development, and Evaluations) framework to evaluate the individual study for both quality and bias. The subsequent quality levels were then appraised to each article and listed as either low, moderate, or high.

Data collection process

Data that satisfied the inclusion requirements was entered into an excel sheet for data extraction and further selection. An article’s author, publication year, country, age range, participant sample size, sex, area of interest, measures, outcome and article quality were all included in the data extraction spreadsheet. Studies looking at FC, for instance, were grouped, while studies looking at FC in specific area were further divided into sub-groups.

Data synthesis and analysis

Articles were classified according to their location in the brain as well as the network or pathway they were a part of to create a coherent narrative between the selected studies. Conclusions concerning various research trends relevant to particular groupings were drawn from these groupings and subgroupings. To maintain the offered information in a prominent manner, these assertions were entered into the data extraction excel spreadsheet.

With the search performed on the selected databases, 238 articles in total were identified (see Fig 1 ). 15 duplicated articles were eliminated, and another 6 items were removed for various other reasons. Title and abstract screening eliminated 184 articles because they were not in English (number of article, n, = 7), did not include imaging components (n = 47), had adult participants (n = 53), did not have a clinical diagnosis of IA (n = 19), did not address FC in the brain (n = 20), and were published outside the desired timeframe (n = 38). A further 21 papers were eliminated for failing to meet inclusion requirements after the remaining 33 articles underwent full-text eligibility screening. A total of 12 papers were deemed eligible for this review analysis.

Characteristics of the included studies, as depicted in the data extraction sheet in Table 1 provide information of the author(s), publication year, sample size, study location, age range, gender, area of interest, outcome, measures used and quality appraisal. Most of the studies in this review utilised resting state functional magnetic resonance imaging techniques (n = 7), with several studies demonstrating task-based fMRI procedures (n = 3), and the remaining studies utilising whole-brain imaging measures (n = 2). The studies were all conducted in Asiatic countries, specifically coming from China (8), Korea (3), and Indonesia (1). Sample sizes ranged from 12 to 31 participants with most of the imaging studies having comparable sample sizes. Majority of the studies included a mix of male and female participants (n = 8) with several studies having a male only participant pool (n = 3). All except one of the mixed gender studies had a majority male participant pool. One study did not disclose their data on the gender demographics of their experiment. Study years ranged from 2013–2022, with 2 studies in 2013, 3 studies in 2014, 3 studies in 2015, 1 study in 2017, 1 study in 2020, 1 study in 2021, and 1 study in 2022.

https://doi.org/10.1371/journal.pmen.0000022.t001

(1) How does internet addiction affect the functional connectivity in the adolescent brain?

The included studies were organised according to the brain region or network that they were observing. The specific networks affected by IA were the default mode network, executive control system, salience network and reward pathway. These networks are vital components of adolescent behaviour and development [ 31 ]. The studies in each section were then grouped into subsections according to their specific brain regions within their network.

Default mode network (DMN)/reward network.

Out of the 12 studies, 3 have specifically studied the default mode network (DMN), and 3 observed whole-brain FC that partially included components of the DMN. The effect of IA on the various centres of the DMN was not unilaterally the same. The findings illustrate a complex mix of increases and decreases in FC depending on the specific region in the DMN (see Table 2 and Fig 2 ). The alteration of FC in posterior cingulate cortex (PCC) in the DMN was the most frequently reported area in adolescents with IA, which involved in attentional processes [ 32 ], but Lee et al. (2020) additionally found alterations of FC in other brain regions, such as anterior insula cortex, a node in the DMN that controls the integration of motivational and cognitive processes [ 20 ].

https://doi.org/10.1371/journal.pmen.0000022.g002

The overall changes of functional connectivity in the brain network including default mode network (DMN), executive control network (ECN), salience network (SN) and reward network. IA = Internet Addiction, FC = Functional Connectivity.

https://doi.org/10.1371/journal.pmen.0000022.t002

Ding et al. (2013) revealed altered FC in the cerebellum, the middle temporal gyrus, and the medial prefrontal cortex (mPFC) [ 22 ]. They found that the bilateral inferior parietal lobule, left superior parietal lobule, and right inferior temporal gyrus had decreased FC, while the bilateral posterior lobe of the cerebellum and the medial temporal gyrus had increased FC [ 22 ]. The right middle temporal gyrus was found to have 111 cluster voxels (t = 3.52, p<0.05) and the right inferior parietal lobule was found to have 324 cluster voxels (t = -4.07, p<0.05) with an extent threshold of 54 voxels (figures above this threshold are deemed significant) [ 22 ]. Additionally, there was a negative correlation, with 95 cluster voxels (p<0.05) between the FC of the left superior parietal lobule and the PCC with the Chen Internet Addiction Scores (CIAS) which are used to determine the severity of IA [ 22 ]. On the other hand, in regions of the reward system, connection with the PCC was positively connected with CIAS scores [ 22 ]. The most significant was the right praecuneus with 219 cluster voxels (p<0.05) [ 22 ]. Wang et al. (2017) also discovered that adolescents with IA had 33% less FC in the left inferior parietal lobule and 20% less FC in the dorsal mPFC [ 24 ]. A potential connection between the effects of substance use and overt internet use is revealed by the generally decreased FC in these areas of the DMN of teenagers with drug addiction and IA [ 35 ].

The putamen was one of the main regions of reduced FC in adolescents with IA [ 19 ]. The putamen and the insula-operculum demonstrated significant group differences regarding functional connectivity with a cluster size of 251 and an extent threshold of 250 (Z = 3.40, p<0.05) [ 19 ]. The molecular mechanisms behind addiction disorders have been intimately connected to decreased striatal dopaminergic function [ 19 ], making this function crucial.

Executive Control Network (ECN).

5 studies out of 12 have specifically viewed parts of the executive control network (ECN) and 3 studies observed whole-brain FC. The effects of IA on the ECN’s constituent parts were consistent across all the studies examined for this analysis (see Table 2 and Fig 3 ). The results showed a notable decline in all the ECN’s major centres. Li et al. (2014) used fMRI imaging and a behavioural task to study response inhibition in adolescents with IA [ 25 ] and found decreased activation at the striatum and frontal gyrus, particularly a reduction in FC at inferior frontal gyrus, in the IA group compared to controls [ 25 ]. The inferior frontal gyrus showed a reduction in FC in comparison to the controls with a cluster size of 71 (t = 4.18, p<0.05) [ 25 ]. In addition, the frontal-basal ganglia pathways in the adolescents with IA showed little effective connection between areas and increased degrees of response inhibition [ 25 ].

https://doi.org/10.1371/journal.pmen.0000022.g003

Lin et al. (2015) found that adolescents with IA demonstrated disrupted corticostriatal FC compared to controls [ 33 ]. The corticostriatal circuitry experienced decreased connectivity with the caudate, bilateral anterior cingulate cortex (ACC), as well as the striatum and frontal gyrus [ 33 ]. The inferior ventral striatum showed significantly reduced FC with the subcallosal ACC and caudate head with cluster size of 101 (t = -4.64, p<0.05) [ 33 ]. Decreased FC in the caudate implies dysfunction of the corticostriatal-limbic circuitry involved in cognitive and emotional control [ 36 ]. The decrease in FC in both the striatum and frontal gyrus is related to inhibitory control, a common deficit seen with disruptions with the ECN [ 33 ].

The dorsolateral prefrontal cortex (DLPFC), ACC, and right supplementary motor area (SMA) of the prefrontal cortex were all found to have significantly decreased grey matter volume [ 29 ]. In addition, the DLPFC, insula, temporal cortices, as well as significant subcortical regions like the striatum and thalamus, showed decreased FC [ 29 ]. According to Tremblay (2009), the striatum plays a significant role in the processing of rewards, decision-making, and motivation [ 37 ]. Chen et al. (2020) reported that the IA group demonstrated increased impulsivity as well as decreased reaction inhibition using a Stroop colour-word task [ 26 ]. Furthermore, Chen et al. (2020) observed that the left DLPFC and dorsal striatum experienced a negative connection efficiency value, specifically demonstrating that the dorsal striatum activity suppressed the left DLPFC [ 27 ].

Salience network (SN).

Out of the 12 chosen studies, 3 studies specifically looked at the salience network (SN) and 3 studies have observed whole-brain FC. Relative to the DMN and ECN, the findings on the SN were slightly sparser. Despite this, adolescents with IA demonstrated a moderate decrease in FC, as well as other measures like fibre connectivity and cognitive control, when compared to healthy control (see Table 2 and Fig 4 ).

https://doi.org/10.1371/journal.pmen.0000022.g004

Xing et al. (2014) used both dorsal anterior cingulate cortex (dACC) and insula to test FC changes in the SN of adolescents with IA and found decreased structural connectivity in the SN as well as decreased fractional anisotropy (FA) that correlated to behaviour performance in the Stroop colour word-task [ 21 ]. They examined the dACC and insula to determine whether the SN’s disrupted connectivity may be linked to the SN’s disruption of regulation, which would explain the impaired cognitive control seen in adolescents with IA. However, researchers did not find significant FC differences in the SN when compared to the controls [ 21 ]. These results provided evidence for the structural changes in the interconnectivity within SN in adolescents with IA.

Wang et al. (2017) investigated network interactions between the DMN, ECN, SN and reward pathway in IA subjects [ 24 ] (see Fig 5 ), and found 40% reduction of FC between the DMN and specific regions of the SN, such as the insula, in comparison to the controls (p = 0.008) [ 24 ]. The anterior insula and dACC are two areas that are impacted by this altered FC [ 24 ]. This finding supports the idea that IA has similar neurobiological abnormalities with other addictive illnesses, which is in line with a study that discovered disruptive changes in the SN and DMN’s interaction in cocaine addiction [ 38 ]. The insula has also been linked to the intensity of symptoms and has been implicated in the development of IA [ 39 ].

“+” indicates an increase in behaivour; “-”indicates a decrease in behaviour; solid arrows indicate a direct network interaction; and the dotted arrows indicates a reduction in network interaction. This diagram depicts network interactions juxtaposed with engaging in internet related behaviours. Through the neural interactions, the diagram illustrates how the networks inhibit or amplify internet usage and vice versa. Furthermore, it demonstrates how the SN mediates both the DMN and ECN.

https://doi.org/10.1371/journal.pmen.0000022.g005

(2) How is adolescent behaviour and development impacted by functional connectivity changes due to internet addiction?

The findings that IA individuals demonstrate an overall decrease in FC in the DMN is supported by numerous research [ 24 ]. Drug addict populations also exhibited similar decline in FC in the DMN [ 40 ]. The disruption of attentional orientation and self-referential processing for both substance and behavioural addiction was then hypothesised to be caused by DMN anomalies in FC [ 41 ].

In adolescents with IA, decline of FC in the parietal lobule affects visuospatial task-related behaviour [ 22 ], short-term memory [ 42 ], and the ability of controlling attention or restraining motor responses during response inhibition tests [ 42 ]. Cue-induced gaming cravings are influenced by the DMN [ 43 ]. A visual processing area called the praecuneus links gaming cues to internal information [ 22 ]. A meta-analysis found that the posterior cingulate cortex activity of individuals with IA during cue-reactivity tasks was connected with their gaming time [ 44 ], suggesting that excessive gaming may impair DMN function and that individuals with IA exert more cognitive effort to control it. Findings for the behavioural consequences of FC changes in the DMN illustrate its underlying role in regulating impulsivity, self-monitoring, and cognitive control.

Furthermore, Ding et al. (2013) reported an activation of components of the reward pathway, including areas like the nucleus accumbens, praecuneus, SMA, caudate, and thalamus, in connection to the DMN [ 22 ]. The increased FC of the limbic and reward networks have been confirmed to be a major biomarker for IA [ 45 , 46 ]. The increased reinforcement in these networks increases the strength of reward stimuli and makes it more difficult for other networks, namely the ECN, to down-regulate the increased attention [ 29 ] (See Fig 5 ).

Executive control network (ECN).

The numerous IA-affected components in the ECN have a role in a variety of behaviours that are connected to both response inhibition and emotional regulation [ 47 ]. For instance, brain regions like the striatum, which are linked to impulsivity and the reward system, are heavily involved in the act of playing online games [ 47 ]. Online game play activates the striatum, which suppresses the left DLPFC in ECN [ 48 ]. As a result, people with IA may find it difficult to control their want to play online games [ 48 ]. This system thus causes impulsive and protracted gaming conduct, lack of inhibitory control leading to the continued use of internet in an overt manner despite a variety of negative effects, personal distress, and signs of psychological dependence [ 33 ] (See Fig 5 ).

Wang et al. (2017) report that disruptions in cognitive control networks within the ECN are frequently linked to characteristics of substance addiction [ 24 ]. With samples that were addicted to heroin and cocaine, previous studies discovered abnormal FC in the ECN and the PFC [ 49 ]. Electronic gaming is known to promote striatal dopamine release, similar to drug addiction [ 50 ]. According to Drgonova and Walther (2016), it is hypothesised that dopamine could stimulate the reward system of the striatum in the brain, leading to a loss of impulse control and a failure of prefrontal lobe executive inhibitory control [ 51 ]. In the end, IA’s resemblance to drug use disorders may point to vital biomarkers or underlying mechanisms that explain how cognitive control and impulsive behaviour are related.

A task-related fMRI study found that the decrease in FC between the left DLPFC and dorsal striatum was congruent with an increase in impulsivity in adolescents with IA [ 26 ]. The lack of response inhibition from the ECN results in a loss of control over internet usage and a reduced capacity to display goal-directed behaviour [ 33 ]. Previous studies have linked the alteration of the ECN in IA with higher cue reactivity and impaired ability to self-regulate internet specific stimuli [ 52 ].

Salience network (SN)/ other networks.

Xing et al. (2014) investigated the significance of the SN regarding cognitive control in teenagers with IA [ 21 ]. The SN, which is composed of the ACC and insula, has been demonstrated to control dynamic changes in other networks to modify cognitive performance [ 21 ]. The ACC is engaged in conflict monitoring and cognitive control, according to previous neuroimaging research [ 53 ]. The insula is a region that integrates interoceptive states into conscious feelings [ 54 ]. The results from Xing et al. (2014) showed declines in the SN regarding its structural connectivity and fractional anisotropy, even though they did not observe any appreciable change in FC in the IA participants [ 21 ]. Due to the small sample size, the results may have indicated that FC methods are not sensitive enough to detect the significant functional changes [ 21 ]. However, task performance behaviours associated with impaired cognitive control in adolescents with IA were correlated with these findings [ 21 ]. Our comprehension of the SN’s broader function in IA can be enhanced by this relationship.

Research study supports the idea that different psychological issues are caused by the functional reorganisation of expansive brain networks, such that strong association between SN and DMN may provide neurological underpinnings at the system level for the uncontrollable character of internet-using behaviours [ 24 ]. In the study by Wang et al. (2017), the decreased interconnectivity between the SN and DMN, comprising regions such the DLPFC and the insula, suggests that adolescents with IA may struggle to effectively inhibit DMN activity during internally focused processing, leading to poorly managed desires or preoccupations to use the internet [ 24 ] (See Fig 5 ). Subsequently, this may cause a failure to inhibit DMN activity as well as a restriction of ECN functionality [ 55 ]. As a result, the adolescent experiences an increased salience and sensitivity towards internet addicting cues making it difficult to avoid these triggers [ 56 ].

The primary aim of this review was to present a summary of how internet addiction impacts on the functional connectivity of adolescent brain. Subsequently, the influence of IA on the adolescent brain was compartmentalised into three sections: alterations of FC at various brain regions, specific FC relationships, and behavioural/developmental changes. Overall, the specific effects of IA on the adolescent brain were not completely clear, given the variety of FC changes. However, there were overarching behavioural, network and developmental trends that were supported that provided insight on adolescent development.

The first hypothesis that was held about this question was that IA was widespread and would be regionally similar to substance-use and gambling addiction. After conducting a review of the information in the chosen articles, the hypothesis was predictably supported. The regions of the brain affected by IA are widespread and influence multiple networks, mainly DMN, ECN, SN and reward pathway. In the DMN, there was a complex mix of increases and decreases within the network. However, in the ECN, the alterations of FC were more unilaterally decreased, but the findings of SN and reward pathway were not quite clear. Overall, the FC changes within adolescents with IA are very much network specific and lay a solid foundation from which to understand the subsequent behaviour changes that arise from the disorder.

The second hypothesis placed emphasis on the importance of between network interactions and within network interactions in the continuation of IA and the development of its behavioural symptoms. The results from the findings involving the networks, DMN, SN, ECN and reward system, support this hypothesis (see Fig 5 ). Studies confirm the influence of all these neural networks on reward valuation, impulsivity, salience to stimuli, cue reactivity and other changes that alter behaviour towards the internet use. Many of these changes are connected to the inherent nature of the adolescent brain.

There are multiple explanations that underlie the vulnerability of the adolescent brain towards IA related urges. Several of them have to do with the inherent nature and underlying mechanisms of the adolescent brain. Children’s emotional, social, and cognitive capacities grow exponentially during childhood and adolescence [ 57 ]. Early teenagers go through a process called “social reorientation” that is characterised by heightened sensitivity to social cues and peer connections [ 58 ]. Adolescents’ improvements in their social skills coincide with changes in their brains’ anatomical and functional organisation [ 59 ]. Functional hubs exhibit growing connectivity strength [ 60 ], suggesting increased functional integration during development. During this time, the brain’s functional networks change from an anatomically dominant structure to a scattered architecture [ 60 ].

The adolescent brain is very responsive to synaptic reorganisation and experience cues [ 61 ]. As a result, one of the distinguishing traits of the maturation of adolescent brains is the variation in neural network trajectory [ 62 ]. Important weaknesses of the adolescent brain that may explain the neurobiological change brought on by external stimuli are illustrated by features like the functional gaps between networks and the inadequate segregation of networks [ 62 ].

The implications of these findings towards adolescent behaviour are significant. Although the exact changes and mechanisms are not fully clear, the observed changes in functional connectivity have the capacity of influencing several aspects of adolescent development. For example, functional connectivity has been utilised to investigate attachment styles in adolescents [ 63 ]. It was observed that adolescent attachment styles were negatively associated with caudate-prefrontal connectivity, but positively with the putamen-visual area connectivity [ 63 ]. Both named areas were also influenced by the onset of internet addiction, possibly providing a connection between the two. Another study associated neighbourhood/socioeconomic disadvantage with functional connectivity alterations in the DMN and dorsal attention network [ 64 ]. The study also found multivariate brain behaviour relationships between the altered/disadvantaged functional connectivity and mental health and cognition [ 64 ]. This conclusion supports the notion that the functional connectivity alterations observed in IA are associated with specific adolescent behaviours as well as the fact that functional connectivity can be utilised as a platform onto which to compare various neurologic conditions.

Limitations/strengths

There were several limitations that were related to the conduction of the review as well as the data extracted from the articles. Firstly, the study followed a systematic literature review design when analysing the fMRI studies. The data pulled from these imaging studies were namely qualitative and were subject to bias contrasting the quantitative nature of statistical analysis. Components of the study, such as sample sizes, effect sizes, and demographics were not weighted or controlled. The second limitation brought up by a similar review was the lack of a universal consensus of terminology given IA [ 47 ]. Globally, authors writing about this topic use an array of terminology including online gaming addiction, internet addiction, internet gaming disorder, and problematic internet use. Often, authors use multiple terms interchangeably which makes it difficult to depict the subtle similarities and differences between the terms.

Reviewing the explicit limitations in each of the included studies, two major limitations were brought up in many of the articles. One was relating to the cross-sectional nature of the included studies. Due to the inherent qualities of a cross-sectional study, the studies did not provide clear evidence that IA played a causal role towards the development of the adolescent brain. While several biopsychosocial factors mediate these interactions, task-based measures that combine executive functions with imaging results reinforce the assumed connection between the two that is utilised by the papers studying IA. Another limitation regarded the small sample size of the included studies, which averaged to around 20 participants. The small sample size can influence the generalisation of the results as well as the effectiveness of statistical analyses. Ultimately, both included study specific limitations illustrate the need for future studies to clarify the causal relationship between the alterations of FC and the development of IA.

Another vital limitation was the limited number of studies applying imaging techniques for investigations on IA in adolescents were a uniformly Far East collection of studies. The reason for this was because the studies included in this review were the only fMRI studies that were found that adhered to the strict adolescent age restriction. The adolescent age range given by the WHO (10–19 years old) [ 65 ] was strictly followed. It is important to note that a multitude of studies found in the initial search utilised an older adolescent demographic that was slightly higher than the WHO age range and had a mean age that was outside of the limitations. As a result, the results of this review are biased and based on the 12 studies that met the inclusion and exclusion criteria.

Regarding the global nature of the research, although the journals that the studies were published in were all established western journals, the collection of studies were found to all originate from Asian countries, namely China and Korea. Subsequently, it pulls into question if the results and measures from these studies are generalisable towards a western population. As stated previously, Asian countries have a higher prevalence of IA, which may be the reasoning to why the majority of studies are from there [ 8 ]. However, in an additional search including other age groups, it was found that a high majority of all FC studies on IA were done in Asian countries. Interestingly, western papers studying fMRI FC were primarily focused on gambling and substance-use addiction disorders. The western papers on IA were less focused on fMRI FC but more on other components of IA such as sleep, game-genre, and other non-imaging related factors. This demonstrated an overall lack of western fMRI studies on IA. It is important to note that both western and eastern fMRI studies on IA presented an overall lack on children and adolescents in general.

Despite the several limitations, this review provided a clear reflection on the state of the data. The strengths of the review include the strict inclusion/exclusion criteria that filtered through studies and only included ones that contained a purely adolescent sample. As a result, the information presented in this review was specific to the review’s aims. Given the sparse nature of adolescent specific fMRI studies on the FC changes in IA, this review successfully provided a much-needed niche representation of adolescent specific results. Furthermore, the review provided a thorough functional explanation of the DMN, ECN, SN and reward pathway making it accessible to readers new to the topic.

Future directions and implications

Through the search process of the review, there were more imaging studies focused on older adolescence and adulthood. Furthermore, finding a review that covered a strictly adolescent population, focused on FC changes, and was specifically depicting IA, was proven difficult. Many related reviews, such as Tereshchenko and Kasparov (2019), looked at risk factors related to the biopsychosocial model, but did not tackle specific alterations in specific structural or functional changes in the brain [ 66 ]. Weinstein (2017) found similar structural and functional results as well as the role IA has in altering response inhibition and reward valuation in adolescents with IA [ 47 ]. Overall, the accumulated findings only paint an emerging pattern which aligns with similar substance-use and gambling disorders. Future studies require more specificity in depicting the interactions between neural networks, as well as more literature on adolescent and comorbid populations. One future field of interest is the incorporation of more task-based fMRI data. Advances in resting-state fMRI methods have yet to be reflected or confirmed in task-based fMRI methods [ 62 ]. Due to the fact that network connectivity is shaped by different tasks, it is critical to confirm that the findings of the resting state fMRI studies also apply to the task based ones [ 62 ]. Subsequently, work in this area will confirm if intrinsic connectivity networks function in resting state will function similarly during goal directed behaviour [ 62 ]. An elevated focus on adolescent populations as well as task-based fMRI methodology will help uncover to what extent adolescent network connectivity maturation facilitates behavioural and cognitive development [ 62 ].

A treatment implication is the potential usage of bupropion for the treatment of IA. Bupropion has been previously used to treat patients with gambling disorder and has been effective in decreasing overall gambling behaviour as well as money spent while gambling [ 67 ]. Bae et al. (2018) found a decrease in clinical symptoms of IA in line with a 12-week bupropion treatment [ 31 ]. The study found that bupropion altered the FC of both the DMN and ECN which in turn decreased impulsivity and attentional deficits for the individuals with IA [ 31 ]. Interventions like bupropion illustrate the importance of understanding the fundamental mechanisms that underlie disorders like IA.

The goal for this review was to summarise the current literature on functional connectivity changes in adolescents with internet addiction. The findings answered the primary research questions that were directed at FC alterations within several networks of the adolescent brain and how that influenced their behaviour and development. Overall, the research demonstrated several wide-ranging effects that influenced the DMN, SN, ECN, and reward centres. Additionally, the findings gave ground to important details such as the maturation of the adolescent brain, the high prevalence of Asian originated studies, and the importance of task-based studies in this field. The process of making this review allowed for a thorough understanding IA and adolescent brain interactions.

Given the influx of technology and media in the lives and education of children and adolescents, an increase in prevalence and focus on internet related behavioural changes is imperative towards future children/adolescent mental health. Events such as COVID-19 act to expose the consequences of extended internet usage on the development and lifestyle of specifically young people. While it is important for parents and older generations to be wary of these changes, it is important for them to develop a base understanding of the issue and not dismiss it as an all-bad or all-good scenario. Future research on IA will aim to better understand the causal relationship between IA and psychological symptoms that coincide with it. The current literature regarding functional connectivity changes in adolescents is limited and requires future studies to test with larger sample sizes, comorbid populations, and populations outside Far East Asia.

This review aimed to demonstrate the inner workings of how IA alters the connection between the primary behavioural networks in the adolescent brain. Predictably, the present answers merely paint an unfinished picture that does not necessarily depict internet usage as overwhelmingly positive or negative. Alternatively, the research points towards emerging patterns that can direct individuals on the consequences of certain variables or risk factors. A clearer depiction of the mechanisms of IA would allow physicians to screen and treat the onset of IA more effectively. Clinically, this could be in the form of more streamlined and accurate sessions of CBT or family therapy, targeting key symptoms of IA. Alternatively clinicians could potentially prescribe treatment such as bupropion to target FC in certain regions of the brain. Furthermore, parental education on IA is another possible avenue of prevention from a public health standpoint. Parents who are aware of the early signs and onset of IA will more effectively handle screen time, impulsivity, and minimize the risk factors surrounding IA.

Additionally, an increased attention towards internet related fMRI research is needed in the West, as mentioned previously. Despite cultural differences, Western countries may hold similarities to the eastern countries with a high prevalence of IA, like China and Korea, regarding the implications of the internet and IA. The increasing influence of the internet on the world may contribute to an overall increase in the global prevalence of IA. Nonetheless, the high saturation of eastern studies in this field should be replicated with a Western sample to determine if the same FC alterations occur. A growing interest in internet related research and education within the West will hopefully lead to the knowledge of healthier internet habits and coping strategies among parents with children and adolescents. Furthermore, IA research has the potential to become a crucial proxy for which to study adolescent brain maturation and development.

Supporting information

S1 checklist. prisma checklist..

https://doi.org/10.1371/journal.pmen.0000022.s001

S1 Appendix. Search strategies with all the terms.

https://doi.org/10.1371/journal.pmen.0000022.s002

S1 Data. Article screening records with details of categorized content.

https://doi.org/10.1371/journal.pmen.0000022.s003

Acknowledgments

The authors thank https://www.stockio.com/free-clipart/brain-01 (with attribution to Stockio.com); and https://www.rawpixel.com/image/6442258/png-sticker-vintage for the free images used to create Figs 2 – 4 .

• View Article
• PubMed/NCBI
• Google Scholar
• 2. Association AP. Diagnostic and statistical manual of mental disorders: DSM-5. 5 ed. Washington, D.C.: American Psychiatric Publishing; 2013.
• 10. Stats IW. World Internet Users Statistics and World Population Stats 2013 [ http://www.internetworldstats.com/stats.htm .
• 11. Rideout VJR M. B. The common sense census: media use by tweens and teens. San Francisco, CA: Common Sense Media; 2019.
• 37. Tremblay L. The Ventral Striatum. Handbook of Reward and Decision Making: Academic Press; 2009.
• 57. Bhana A. Middle childhood and pre-adolescence. Promoting mental health in scarce-resource contexts: emerging evidence and practice. Cape Town: HSRC Press; 2010. p. 124–42.
• 65. Organization WH. Adolescent Health 2023 [ https://www.who.int/health-topics/adolescent-health#tab=tab_1 .

• Open access
• Published: 04 June 2024

What are medical students taught about persistent physical symptoms? A scoping review of the literature

• Catie Nagel 1 ,
• Chloe Queenan 1 &
• Chris Burton 1  

BMC Medical Education volume  24 , Article number:  618 ( 2024 ) Cite this article

133 Accesses

Metrics details

Persistent Physical Symptoms (PPS) include symptoms such as chronic pain, and syndromes such as chronic fatigue. They are common, but are often inadequately managed, causing distress and higher costs for health care systems. A lack of teaching about PPS has been recognised as a contributing factor to poor management.

The authors conducted a scoping review of the literature, including all studies published before 31 March 2023. Systematic methods were used to determine what teaching on PPS was taking place for medical undergraduates. Studies were restricted to publications in English and needed to include undergraduate medical students. Teaching about cancer pain was excluded. After descriptive data was extracted, a narrative synthesis was undertaken to analyse qualitative findings.

A total of 1116 studies were found, after exclusion, from 3 databases. A further 28 studies were found by searching the grey literature and by citation analysis. After screening for relevance, a total of 57 studies were included in the review. The most commonly taught condition was chronic non-cancer pain, but overall, there was a widespread lack of teaching and learning on PPS. Several factors contributed to this lack including: educators and learners viewing the topic as awkward, learners feeling that there was no science behind the symptoms, and the topic being overlooked in the taught curriculum. The gap between the taught curriculum and learners’ experiences in practice was addressed through informal sources and this risked stigmatising attitudes towards sufferers of PPS.

Faculties need to find ways to integrate more teaching on PPS and address the barriers outlined above. Teaching on chronic non-cancer pain, which is built on a science of symptoms, can be used as an exemplar for teaching on PPS more widely. Any future teaching interventions should be robustly evaluated to ensure improvements for learners and patients.

Peer Review reports

Persistent Physical Symptoms (PPS) are symptoms which are disproportionate to currently recognised pathology and are common in all fields of medicine. The term encompasses single symptoms such as pain, dizziness or fatigue, and established syndromes including fibromyalgia and irritable bowel syndrome. It is increasingly understood that PPS arise from complex interactions between the brain and body [ 1 , 2 ]. While historically terms such as “medically unexplained symptoms” have been in common use, most symptoms can actually be explained [ 3 ] and PPS is a more acceptable term to patients [ 4 ].

PPS are common and present to nearly every medical specialty. They represent the primary reason for presentation in around 45% of general practice consultations and between 30 and 70% of presentations to neurology, gynaecology, and rheumatology outpatient clinics [ 5 ]. People with PPS suffer unduly in a medical system that is predisposed to ‘body part medicine,’ [ 6 ] resulting in what Balint referred to as the “collusion of anonymity.” [ 7 ] In other words, patients who pass from specialist to specialist, without any doctor taking full responsibility for holistic care. Patients with PPS consult more frequently [ 8 ] and tend to have a higher rate of referral to secondary care [ 9 ]. This is costly, both in financial terms and in terms of the emotional work for patients and clinicians [ 10 , 11 ]. Patients with PPS often have a poor experience of the health system and can be left feeling marginalised and even stigmatised [ 12 ].

Doctors find it difficult to consult and manage patients with persistent physical symptoms [ 8 ]. The absence of a common language of explanation to reconcile patients’ lived experience with doctors’ biomedical models, is particularly problematic [ 13 ]. It is plausible that difficulties may arise, or be perpetuated by, issues in each of the three domains of learning: cognitive (knowledge), psychomotor (skills) and affective (attitudes) [ 14 ].

The shifting perspectives, particularly around “medically unexplained symptoms” may account for historical uncertainty, however recent adoption of more consistent language and underlying models of symptoms mean that a common curriculum should be possible [ 15 ]. It is the authors’ experience that little teaching and learning at the undergraduate level has previously taken place on this topic. We wanted to find out if this was still the case, by reviewing the current medical education literature.

We carried out a scoping review with narrative synthesis following the approach of Arksey & O’Malley [ 16 ]. The PRISMA-ScR guidelines were used to structure reporting [ 17 ].

Research questions

The aim of the review was to explore the published literature regarding undergraduate medical teaching and learning on persistent physical symptoms. The specific research questions were:

What teaching and learning on persistent physical symptoms has been described for medical undergraduates?

What teaching methods have been used and how have these been evaluated?

Search strategy

We used a Population, Concept, and Context (PCC) framework to structure a systematic search. The population was undergraduate medical students, the concept was persistent physical symptoms, and the context was teaching and learning. A variety of synonyms were used in order to be inclusive, given the constant evolution of terms for persistent physical symptoms. We used adjacency searching and truncation methods in order to broaden the search as widely as possible and to account for different spellings of words or use of phrases across the international context. Search concepts were then combined using Boolean operators. No date range was used, so all studies before 31 March 2023 were included. Inclusion criteria were: studies relating to the teaching and learning of Persistent Physical Symptoms; medical students included in the population; available in the English language. Exclusion criteria were: studies about cancer or terminal pain without the inclusion of other forms of chronic or persistent pain; population not including medical students; letters to the editor, and papers which were not available in the English language. See Table  1 for the full search strategy.

Sources of evidence

Two authors searched for published literature in MEDLINE, PsycINFO, and Web of Science. Additionally, we searched Google and Google Scholar in order to include any grey literature or sources that had not been picked up by the previous search method. We employed citation analysis, by following backward citations from included papers and analysed the citations of any existing literature reviews.

Study selection

We used a two-stage screening process to identify eligible papers: first at title and abstract level and then at full text. This method was undertaken separately by two reviewers.

Literature reviews were excluded to avoid duplicated representation of primary data, but citations in these reviews were analysed to ensure consistent inclusion of studies and to check for any additional sources.

Charting the data: summary and synthesis

Summary findings for each full text article were charted to determine the most relevant items for extraction. This was an iterative process given the high degree of heterogeneity between the studies. Charting was conducted by two reviewers independently. Discrepancies in charting and data extraction were discussed in review meetings and a consensus was reached regarding which data to include for analysis.

Reviewers extracted descriptive data including: country of origin, whether the study was experimental or observational, the characteristics of the study participants, and whether any teaching intervention was evaluated. Other study characteristics were noted, such as the symptom or syndrome represented, as well as the type of study or intervention.

The expectation was that there would be a lack of teaching and learning on the subject of persistent physical symptoms. For this reason, the scoping review aimed to capture the greatest breadth of studies, rather than exclude studies based on quality criteria. If a teaching intervention was used, we did look at whether this was evaluated using a validated tool.

Following the extraction of descriptive data, a narrative synthesis was undertaken to identify other key findings. An inductive, iterative approach was taken in order to identify themes relating back to the research question. Manual coding was undertaken by two authors independently, followed by a discussion with all authors to arrive at an interpretation of the findings.

Search strategy, study selection, and data extraction

Searches identified 1390 unique titles. Studies were limited to English language and human participants, leading to 274 being excluded. First stage screening excluded a further 1080 studies. It was not possible to retrieve one study and six were excluded on full text. Ten further records were identified through a grey literature search using Google and Google Scholar and 18 were found through citation searching, three of which were from a previous literature review [ 15 ]. This resulted in 57 publications for inclusion in the review. See the PRISMA flow diagram in Fig.  1 for a summary of these findings.

PRISMA Diagram

Adapted from Page MJ, et al. [ 17 ]

Descriptive analysis

Study types.

The studies included for review were highly heterogeneous in their nature. 15/57 (26%) studies employed a teaching intervention, with the remaining either being observational or qualitative. 8/57 (14%) studies described or evaluated the teaching curriculum, 13/57 (22%), included an assessment of the current level of learner knowledge. 9/57 (16%) used qualitative methods with learners and 6/57 (11%) with medical educators. One literature review on assessing knowledge, perceptions and attitudes to pain was found [ 15 ]. The citations of this review were checked and the three new sources [ 18 , 19 , 20 ] were included for review. Sources within this literature review that did not meet the eligibility criteria were excluded. The findings of the review itself were noted for congruity, but not formally analysed.

Study characteristics

23/57(40%) of studies took place in USA and 13/57 (23%) in the UK. Six studies took place in Scandinavia and four in Canada, four in Australia and one in New Zealand, India, and Nigeria respectively. Some studies were based in more than one country e.g. Australia and New Zealand [ 21 ]. Publication dates ranged from 1992 to 2022. See Table  2 for a summary of the descriptive data.

Teaching and learning methods

A wide range of teaching and learning methods were discussed in the literature. These are fully described in Table  3 , but included lectures, workshops, reflective practice, and forum theatre.

Evaluation of teaching studies

Four studies used validated tools to assess learner attitudes towards patients with PPS, but only one used such a tool to evaluate a teaching intervention [ 22 ]. Morris, Rankin, and Briggs used the HC-PAIRS attitudinal questionnaire to assess learner attitudes towards patients with chronic low back pain [ 18 , 23 , 24 ]. Whereas Friedberg et al. [ 22 ] used the Chronic Fatigue Syndrome Attitudes Test (CFSAT) and paired t-test to analyse learner attitudes before and after a teaching intervention. The remaining educational interventions either did not use a validated tool for evaluation or were not formally evaluated. See Table  3 for more details.

Thematic synthesis

All studies identified a lack of teaching about persistent physical symptoms (PPS) at undergraduate level. The narrative synthesis identified four themes: An awkward problem, an absence of science, being easily overlooked, and a hidden curriculum.

An awkward problem

PPS was consistently viewed as an awkward problem. Medical educators and learners found it difficult to understand, particularly when referring to the symptoms as ‘unexplained.’ Some educators described PPS as too complex or too confusing, even ‘dangerous’ to introduce at an undergraduate level and stated the need to focus on the easily ‘explainable.’ [ 25 ] Chronic non-cancer pain was the dominant condition represented in the literature, but despite theoretical concepts of chronic pain being more established, learners found the subject challenging, even ‘unpleasant.’ [ 26 , 27 ].

The absence of science

Four studies highlighted that learners infer patients with PPS have ‘no science’ behind their symptoms. In the study by Vasanthy [ 28 ], clinical role models in Kerala were found to have a ‘nihilistic’ attitude towards people with fibromyalgia and regarded the condition as benign and unimportant. This finding was echoed by UK studies [ 29 , 30 , 31 ] where the impact of a lack of teaching and negative role modelling was evident:

“You can’t really train someone for it because there is no science behind it” [ 30 ].

One final year medical student stated that fibromyalgia was “not a medical issue” intimating that it had no place in the taught curriculum [ 29 ]. Learners understood the need to be supportive and for good communication, but only as a way of achieving relational congruence, not epistemic congruence [ 8 ]. Terminology may be important and in one study learners’ attitudes towards PPS varied depending on the diagnostic label [ 32 ]. As an example, learners thought that people with myalgic encephalopathy were less likely to recover than those with chronic fatigue syndrome [ 32 ].

Easily overlooked

Even without the overt attitudinal barriers described in some studies, PPS as a topic is overlooked in undergraduate medical education. The most common barrier was an already overloaded teaching curriculum [ 25 , 33 ]. PPS was not deemed a priority area by educational leaders and [ 33 ] even when they recognised its importance, they cited a lack of ownership of the topic and a lack of coordination between teaching specialties as a barrier to implementing teaching. This was in contrast to chronic non-cancer pain teaching which usually did have clear ownership by pain specialists and established interdisciplinary relationships [ 34 ]. The experience of learners in the clinical setting was that they were shielded from patients with PPS or directed towards patients with other more easily defined clinical problems [ 28 ].

Stigma and the hidden curriculum

Given the vacuum of formal teaching, learners were taking on stigmatised messages about sufferers of PPS, frequently from role models in the clinical placement setting. Stenhoff and colleagues described a cycle of negativity created by the lack of teaching on the subject of chronic fatigue, which resulted in negative behaviour by clinical role models, in turn perpetuating negative attitudes in the next generation of learners [ 31 ]. Whilst learners recognised the problematic nature of the attitudes towards people with PPS, they lacked the tools to challenge negative stereotypes [ 29 , 30 , 35 ]. Learners experienced a mismatch between formal teaching on the topic and their experience on placement, where these conditions were frequently encountered. They addressed the gap by seeking information about PPS from informal sources, such as their own or their families’ experiences or from the internet [ 36 ]. This lack of explicit teaching and the influence of informal sources has been termed by some authors as the ‘hidden curriculum’ [ 29 , 30 , 31 , 36 ] and this has had a significant impact on learners’ attitudes towards people suffering with PPS.

Suggestions for improvement: relationship to domains of learning

The findings of the narrative synthesis map onto Bloom’s revised three domains of learning [ 14 ].

Knowledge (cognitive)

A number of studies demonstrate success in teaching on the topic of chronic non-cancer pain. Teaching interventions tended to include a foundation of knowledge such as teaching on pain mechanisms, pharmacology, and pain management [ 37 , 38 ]. Such theory-driven interventions led to improved scores on assessment [ 39 ]. Methods of teaching should be considered in the explicitly taught curriculum. Authors recommended an integrated approach [ 40 , 41 ] and one which drew on the skills and knowledge from a variety of disciplines [ 37 , 42 ]. Curriculum mapping was recommended by Howman et al. [ 33 ] in order to identify ways in which this integration could be implemented. The need for an holistic approach which emphasises the importance of empathy [ 41 ] and the biopsychosocial was also widely recognised [ 43 , 44 , 45 , 46 ]. Learners cited a lack of assessment as an indicator that PPS was either unimportant or uncommon [ 29 , 33 ] and therefore any teaching intervention should include assessment in order to drive learning and engagement.

Skills (psychomotor)

Learners valued the addition of skills-based teaching and engaged best with teaching that was experiential [ 47 ] and included either patients with PPS or simulated patients [ 45 , 48 , 49 ]. In one study the focus of the teaching was on interactive, practical teaching for emotionally demanding consultations and the skills taught in such a programme could be transferable to the PPS context [ 49 ]. Approaches to help learners find a common language of explanation [ 13 ] will not only bridge the epistemic gap between clinicians and patients [ 8 ], but should give learners greater confidence and satisfaction in consultations where PPS are the focus.

Attitudes (affective) and the role of reflection

Reflection is a key transferable skill that graduates should acquire as part of their undergraduate training [ 50 ]. Both learners and educators voiced a great deal of anxiety regarding teaching and managing patients with PPS. Some authors utilised reflective logs and visual art as a way of teaching about chronic pain [ 51 ] and learners valued the deep insights provided by this method. Skills in reflection might help to ameliorate the negative emotions felt by learners, especially if combined with a taught framework that helps them understand concepts such as internal bias and cognitive dissonance [ 52 ].

Summary of main findings

This review found that teaching on persistent physical symptoms in undergraduate medical education is inconsistent and incomplete. We identified four important themes: an awkward problem, the absence of science, easily overlooked, and the hidden curriculum. Mapping these to teaching and learning domains provides a coherent framework for undergraduate teaching of these common conditions. Where teaching does take place, this is more frequently on the topic of chronic non-cancer pain. A number of studies have demonstrated improved knowledge [ 39 ], skills [ 49 ], and attitudes [ 51 ] as a result of this teaching [ 34 , 47 ], but high quality evaluation of such teaching and learning is lacking.

Strengths and limitations

This scoping review has addressed a gap in the literature. By undertaking a search of three databases, the grey literature, and citation analysis, a wide range of sources were included for initial screening. Two researchers independently undertook the search strategy before comparing findings which has helped to ensure a robust and systematic approach. Narrative synthesis was undertaken by three researchers, one with expertise in the field of persistent physical symptoms.

The majority of the studies identified were from the USA and UK. Papers that were not accessible in English were excluded, which may explain this finding. Where teaching and learning evaluations had taken place, this was on a small scale usually within one institution. Only one study [ 22 ] used a validated tool to evaluate the efficacy of the teaching intervention.

Implications for practice, policy, and research

There is a lack of teaching on PPS in undergraduate medical education. As a result, medical graduates are ill-equipped to recognise, consult for, and manage this group of conditions. Given the prevalence of PPS across medical specialties this is a priority area that needs to be addressed, whilst acknowledging the barriers that exist to implementation.

The solutions offered up in the literature include the need to consider whole-person care, in order to avoid fragmentation and the “collusion of anonymity” [ 7 ] described above. For this reason, teaching on PPS should be integrated into the core curriculum and draw on a variety of disciplines.

A better understanding of the science behind PPS [ 1 , 2 ] is needed for both educators and learners. There is also a need to move learners beyond reductionist models of communication skills towards more theory-driven approaches of person-centredness, as identified by Bansal [ 53 ]. We need to convey to learners that skilled communication is not about platitudes, but can make a difference to recovery and addresses the current epistemic gap between clinicians and their patients [ 8 , 13 ].

Future educational research should focus on the most effective methods to improve the knowledge base of both educators and students and how best to evaluate the success of future teaching interventions. Skills in person-centred communication and explanation [ 3 ] need to be taught, alongside those in reflection and challenging prejudice.

We identified four important themes which underpin the challenges of teaching medical undergraduates about persistent physical symptoms. Educational faculties need to find ways to integrate teaching into current programmes and work around the existing barriers to successful implementation and evaluation of teaching about these common and limiting conditions. Examples of successful teaching on chronic non-cancer pain were found in the literature. These tended to articulate the science behind symptoms and often included experiential elements. Such examples should be used to inform an approach for teaching about other forms of PPS. Importantly, robust evaluation that accounts for the complexity of the taught environment is needed to ensure our teaching is making a difference, both for our learners and the patients they will go on to encounter.

Data availability

All data generated or analysed during this study are included in this published article and its supplementary information files.

Henningsen P, et al. Persistent physical symptoms as Perceptual Dysregulation: a Neuropsychobehavioral Model and its clinical implications. Psychosom Med. 2018;80(5):422–31.

Article   Google Scholar  

Burton C et al. Functional somatic disorders: discussion paper for a new common classification for research and clinical use BMC Medicine, 2020;18(1).

Morton L, et al. A taxonomy of explanations in a general practitioner clinic for patients with persistent medically unexplained physical symptoms. Patient Educ Couns. 2017;100(2):224–30.

Picariello F, Ali S, Moss-Morris R, Chalder T. The most popular terms for medically unexplained symptoms: the views of CFS patients. J Psychosom Res. 2015;78(5):420–6.

Haller H, Cramer H, Lauche R, Dobos G. Somatoform disorders and medically unexplained symptoms in primary care. Deutsches Ärzteblatt international, 2015.

Oldham J. Reform reform: an essay by John Oldham. BMJ: Br Med J. 2013;347:f6716.

Balint M. The doctor, his patient and the illness . 2000, Edinburgh: Edinburgh: Churchill Livingstone, 2000.

Johansen M-L, Risor MB. What is the problem with medically unexplained symptoms for GPs? A meta -synthesis of qualitative studies. Patient Educ Couns. 2017;100(4):647–54.

Verhaak PFM. Persistent presentation of medically unexplained symptoms in general practice. Fam Pract. 2006;23(4):414–20.

Mcgorm K, et al. Patients repeatedly referred to secondary care with symptoms unexplained by organic disease: prevalence, characteristics and referral pattern. Fam Pract. 2010;27(5):479–86.

Barsky AJ, Orav EJ, Bates DW. Somatization increases medical utilization and costs Independent of Psychiatric and Medical Comorbidity. Arch Gen Psychiatry. 2005;62(8):903.

Polakovská L, Řiháček T. What is it like to live with medically unexplained physical symptoms? A qualitative meta-summary. Psychology & Health. 2021:pp. 1–17.

Salmon P. Conflict, collusion or collaboration in consultations about medically unexplained symptoms: the need for a curriculum of medical explanation. Patient Educ Couns. 2007;67(3):246–54.

Anderson L. A taxonomy for learning, teaching, and assessing: a revision of Bloom’s . Pearson new international ed. 2014, Harlow, Essex: Pearson.

Ung A, Salamonson Y, Hu W, Gallego G. Assessing knowledge, perceptions and attitudes to pain management among medical and nursing students: a review of the literature. Br J Pain. 2016;10(1):8–21.

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Page MJ et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ, 2021:p. n71.

Briggs AM, et al. Low back pain-related beliefs and likely practice behaviours among final-year cross-discipline health students. Eur J Pain. 2013;17(5):766–75.

Murinson BB, et al. A new program in pain medicine for medical students: integrating core curriculum knowledge with emotional and reflective development. Pain Med. 2011;12(2):186–95.

Watt-Watson J, et al. An integrated undergraduate pain curriculum, based on IASP curricula, for six Health Science Faculties. Pain. 2004;110(1):140–8.

Shipton EE, et al. Pain medicine content, teaching and assessment in medical school curricula in Australia and New Zealand. Volume 18. BMC Medical Education; 2018:1.

Friedberg F, Sohl SJ, Halperin PJ. Teaching medical students about medically unexplained illnesses: a preliminary study. Med Teach. 2008;30(6):618–21.

Morris H, Ryan C, Lauchlan D, Field M. Do medical student attitudes towards patients with chronic low back pain improve during training? A cross-sectional study. BMC Med Educ. 2012;12(1):10.

Rankin L, Fowler CJ, Stålnacke B-M, Gallego G. What influences chronic pain management? A best–worst scaling experiment with final year medical students and general practitioners. Br J Pain. 2019;13(4):214–25.

Joyce E, et al. Training tomorrow’s doctors to explain ‘medically unexplained’ physical symptoms: an examination of UK medical educators’ views of barriers and solutions. Patient Educ Couns. 2018;101(5):878–84.

Corrigan C, et al. What can we learn from First-Year Medical Students’ perceptions of Pain in the primary care setting? Pain Med. 2011;12(8):1216–22.

Wilson JF, et al. Medical students’ attitudes toward pain before and after a brief course on pain. Pain. 1992;50(3):251–6.

Vasanthy B, Parameswaran Nair VC. Fibromyalgia: perspective of patients, medical students and professionals. Journal of evidence based Medicine and Healthcare. J Evid Based Med Healthc. 2018;5(34):2463–7.

Silverwood V et al. ‘If it’s a medical issue I would have covered it by now’: learning about fibromyalgia through the hidden curriculum: a qualitative study. BMC Med Educ, 2017;17(1).

Shattock L, et al. They’ve just got symptoms without science’: medical trainees’ acquisition of negative attitudes towards patients with medically unexplained symptoms. Patient Educ Couns. 2013;91(2):249–54.

Stenhoff AL, Sadreddini S, Peters S, Wearden A. Understanding medical students’ views of chronic fatigue syndrome: a qualitative study. J Health Psychol. 2015;20(2):198–209.

Jason LA, et al. Evaluating attributions for an illness based upon the name: chronic fatigue syndrome, myalgic Encephalopathy and Florence Nightingale Disease. Am J Community Psychol. 2002;30(1):133–48.

Howman M, et al. Teaching about medically unexplained symptoms at medical schools in the United Kingdom. Med Teach. 2012;34(4):327–9.

Vargovich AM, et al. Difficult conversations: Training Medical students to assess, educate, and treat the patient with Chronic Pain. Acad Psychiatry. 2019;43(5):494–8.

Yon K, et al. Improving teaching about medically unexplained symptoms for newly qualified doctors in the UK: findings from a questionnaire survey and expert workshop. BMJ Open. 2017;7(4):e014720.

Emorinken A, et al. Assessment of Undergraduate Medical Students Knowledge and Awareness of Fibromyalgia. J Evol Med Dent Sci. 2022;11(5):551–6.

Comer L. Content analysis of chronic pain content at three undergraduate medical schools in Ontario. Can J Pain. 2017;1(1):75–83.

Kolber BJ, Janjic JM, Pollock JA, Tidgewell KJ. Summer undergraduate research: a new pipeline for pain clinical practice and research. BMC Med Educ, 2016;16(1).

Weiner DK, et al. E-Learning Module on Chronic Low Back Pain in older adults: evidence of Effect on Medical Student Objective Structured Clinical Examination performance. J Am Geriatr Soc. 2014;62(6):1161–7.

Pöyhiä R, Niemi-Murola L, Kalso E. The outcome of pain related undergraduate teaching in Finnish medical faculties. Pain. 2005;115(3):234–7.

Murinson BB, et al. Recommendations for a New Curriculum in Pain Medicine for Medical students: toward a Career distinguished by competence and Compassion. Pain Med. 2013;14(3):345–50.

Morley-Forster P et al. Mitigating the risk of opioid abuse through a balanced undergraduate pain medicine curriculum. J Pain Res, 2013:p. 791.

Wojtowicz AA, et al. Perceptions of clinical training in biopsychosocial treatment of pediatric functional abdominal pain: a survey of medical students. Clin Pract Pediatr Psychol. 2020;8:37–44.

Google Scholar  

Dwyer CPM-P, Phoebe E, Durand H, Gormley EM, Slattery BW, Harney OM, MacNeela P, McGuire BE. Factors influencing the application of a Biopsychosocial Perspective in Clinical Judgement of Chronic Pain: interactive management with medical students. Pain Physician. 2017;20(6):E951–60.

Ali N, Thomson DI. A comparison of the knowledge of chronic pain and its management between final year physiotherapy and medical students. Eur J Pain. 2009;13(1):38–50.

Tauben DJ, Loeser JD. Pain Education at the University of Washington School of Medicine. J Pain. 2013;14(5):431–7.

Stevens DL, et al. Medical students retain pain assessment and management skills long after an experiential curriculum: a controlled study. Pain. 2009;145(3):319–24.

Bradner M, et al. Chronic non-malignant pain: it’s complicated. Clin Teach. 2019;16(5):530–2.

Baessler F, et al. Are we preparing future doctors to deal with emotionally challenging situations? Analysis of a medical curriculum. Patient Educ Couns. 2019;102(7):1304–12.

GMC, Outcomes for Graduates . 2018.

Lempp H, Potter J, Petit P, Hester J. Exploring chronic pain with patients: medicine meets art. Med Educ. 2010;44(11):1139–40.

Syed M. Black box thinking: marginal gains and the secrets of high performance. London: John Murray; 2016.

Bansal A, et al. Optimising planned medical education strategies to develop learners’ person-centredness: A realist review. Medical Education, 2021.

Turner GH, Weiner DK. Essential components of a Medical Student Curriculum on Chronic Pain Management in older adults: results of a modified Delphi process. Pain Med. 2002;3(3):240–52.

Niemi-Murola L, et al. Training medical students to manage a chronic pain patient: both knowledge and communication skills are needed. Eur J Pain. 2006;10(2):167–167.

Saypol B, Schmulson DD. A review of three educational projects using interactive theater to improve physician-patient communication when treating patients with irritable bowel syndrome. Rev Esp Enferm Dig. 2015;107(5):268–73.

Bradshaw YS, et al. Deconstructing one Medical School’s Pain Curriculum: II. Partnering with medical students on an evidence-guided redesign. Pain Med. 2017;18(4):664–79.

Leeds FS, et al. A patient-narrative Video Approach to Teaching Fibromyalgia. J Med Educ Curric Dev. 2020;7:238212052094706.

Gadde U, et al. Implementing an interactive introduction to complementary medicine for chronic Pain Management into the Medical School Curriculum. MedEdPORTAL. 2020;16(1):11056.

Campbell WI. What do medical-students know about chronic pain and its management. Ulster Med J. 1992;61(2):139–43.

Chibnall JT, Tait RC, Ross LR. J Behav Med. 1997;20(3):257–71.

Niemi-Murola L, Nieminen JT, Kalso E, Pöyhiä R. Medical undergraduate students’ beliefs and attitudes toward pain - how do they mature? Eur J Pain. 2007;11(6):700–6.

Amber KT, Brooks L, Chee J, Ference TS. Assessing the perceptions of Fibromyalgia Syndrome in United States among Academic Physicians and Medical students: where are we and where are we headed? J Musculoskelet Pain. 2014;22(1):13–9.

Amber KT, Brooks L, Ference TS. Does Improved confidence in a Disease relate to increased knowledge? Our experience with medical students: table 1. Pain Med. 2014;15(3):483–4.

Adillón C, Lozano È, Salvat I. Comparison of pain neurophysiology knowledge among health sciences students: a cross-sectional study. BMC Res Notes, 2015. 8(1).

Argyra E, et al. How does an undergraduate Pain Course Influence Future Physicians’ awareness of Chronic Pain concepts? A comparative study. Pain Med. 2015;16(2):301–11.

Briggs EV, et al. Current pain education within undergraduate medical studies across Europe: advancing the provision of Pain Education and Learning (APPEAL) study. BMJ Open. 2015;5(8):e006984.

Hollingshead NA et al. Examining influential factors in providers’ chronic pain treatment decisions: a comparison of physicians and medical students. BMC Med Educ, 2015. 15(1).

Rankin L, Stålnacke B-M, Fowler CJ, Gallego G. Differences in Swedish and Australian medical student attitudes and beliefs about chronic pain, its management, and the way it is taught. Scandinavian J Pain. 2018;18(3):533–44.

Cristóvão I, Reis-Pina P. O Ensino Da Dor Crónica em Portugal: as Perspectivas Dos Estudantes De Medicina E dos internos do Ano Comum. Acta Med Port. 2019;32(5):338.

Gustafsson Sendén M, Renström EA. Gender bias in assessment of future work ability among pain patients – an experimental vignette study of medical students’ assessment. Scandinavian J Pain. 2019;19(2):407–14.

Lechowicz K, et al. Acute and Chronic Pain Learning and Teaching in Medical School—An observational cross-sectional study regarding Preparation and Self-confidence of clinical and Pre-clinical Medical Students. Medicina. 2019;55(9):533.

Storrar A, Rayment D, Mallam E. 47 undergraduate teaching and perceptions of functional neurological disorders. Members’ POSTER abstracts. BMJ Publishing Group Ltd; 2019.

Muirhead N, Muirhead J, Lavery G, Marsh B. Medical School Education on myalgic encephalomyelitis. Medicina. 2021;57(6):542.

Simons J, et al. Disorders of gut-brain interaction: highly prevalent and burdensome yet under‐taught within medical education. United European Gastroenterology Journal, 2022.

Lambson R. Chronic fatigue syndrome: where do your views lie? An experience from a UK Medical Student. Int J Med Students. 2015;3(2):117–8.

Raber I, et al. Qualitative Assessment of Clerkship Students’ perspectives of the topics of Pain and Addiction in their preclinical curriculum. Acad Psychiatry. 2018;42(5):664–7.

Rice K, et al. Medical trainees’ experiences of Treating People with Chronic Pain: a lost opportunity for Medical Education. Acad Med. 2018;93(5):775–80.

Sallay V, et al. Medical educators’ experiences on medically unexplained symptoms and intercultural communication—an expert focus group study. Volume 22. BMC Medical Education. 2022;1.

Download references

Acknowledgements

Not applicable.

CN is undertaking an In Practice Fellowship which has been funded by the NIHR (Reference number NIHR301019). This is an educational award and does not fund direct research costs.

Author information

Authors and affiliations.

Primary Care Research Group, Division of Population Health, School of Medicine, University of Sheffield, Regent Court, 30 Regent Street, S1 4DA, Sheffield, UK

Catie Nagel, Chloe Queenan & Chris Burton

You can also search for this author in PubMed   Google Scholar

Contributions

Chris Burton: review and editing (equal); formal analysis (supporting). Catie Nagel: Conceptualisation (lead); Methodology (lead); writing – original draft (lead); formal analysis (lead); writing – review and editing (lead). Chloe Queenan: review and editing (supporting); formal analysis (supporting).

Corresponding author

Correspondence to Catie Nagel .

Ethics declarations

Ethics approval and consent to participate, authors’ information, consent for publication, competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Nagel, C., Queenan, C. & Burton, C. What are medical students taught about persistent physical symptoms? A scoping review of the literature. BMC Med Educ 24 , 618 (2024). https://doi.org/10.1186/s12909-024-05610-z

Download citation

Received : 03 November 2023

Accepted : 27 May 2024

Published : 04 June 2024

DOI : https://doi.org/10.1186/s12909-024-05610-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Scoping review
• Narrative synthesis
• Undergraduate
• Persistent physical symptoms
• Medically unexplained symptoms
• Chronic pain

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Lippincott Open Access

Integrity of Databases for Literature Searches in Nursing

The quality of literature used as the foundation to any research or scholarly project is critical. The purpose of this study was to analyze the extent to which predatory nursing journals were included in credible databases, MEDLINE, Cumulative Index to Nursing and Allied Health Literature (CINAHL), and Scopus, commonly used by nurse scholars when searching for information. Findings indicated that no predatory nursing journals were currently indexed in MEDLINE or CINAHL, and only one journal was in Scopus. Citations to articles published in predatory nursing journals are not likely found in a search using these curated databases but rather through Google or Google Scholar search engines.

Research, evidence-based practice, quality improvement studies, and other scholarly projects typically begin with a literature review. In research, the review of the literature describes existing knowledge about the topic, reveals gaps and further research questions to be answered, and provides a rationale for engaging in a new study. In evidence-based practice, the literature review provides evidence to answer clinical questions and make informed decisions. Quality improvement studies also begin with a search of the literature to gather available knowledge about a problem and explore interventions used in other settings. The appearance of journals that are published by predatory publishers has introduced the danger that reviews of the literature include inadequate, poorly designed, and low-quality information being used as “evidence”—raising the possibility of risky and harmful practice. Researchers and authors should be confident in the literature they cite; readers should have assurance that the literature review is based on sound, authoritative sources. When predatory journals are cited, that trust is eroded. No matter what type of study or project is being done, the quality of literature is critical for the development of nursing knowledge and for providing up-to-date information, concepts, theories, and approaches to care. 1

An effective literature review requires searching various reliable and credible databases such as MEDLINE (through PubMed or Ovid) and the Cumulative Index to Nursing and Allied Health Literature (CINAHL), among others that are relevant to the topic. The ease of searching using a web browser (now commonly referred to as “googling”) has increased the risk of finding sources published in predatory and low-quality journals that have not met the standards of research and scholarship that can be trusted as credible and reliable evidence.

The purpose of this article is to present an analysis of the extent to which predatory nursing journals are included in MEDLINE, CINAHL, and Scopus databases, used by nurse researchers and other nurses when searching for information, and in the Directory of Open Access Journals. This directory indexes “high-quality, open access, peer-reviewed journals” and should not include any predatory journals. 2

Statement of Significance

What is known or assumed to be true about this topic?

The quality of nursing literature used is vital for the development of research studies, application of evidence in clinical settings, and other scholarly projects. Nurse scholars need to be confident as they search the literature that they are accessing sound information sources and not articles from predatory nursing journals, which do not adhere to quality and ethical publishing standards. Citations of articles in predatory nursing journals may be found when searching Google and Google Scholar, making these citations easy to access but potentially resulting in the integration of poor quality research into the nursing literature. On the other hand, searches through credible databases—MEDLINE, CINAHL, and Scopus—are less likely to yield citations from predatory publications.

What this article adds:

This study helps validate the trustworthiness of these databases for conducting searches in nursing.

PREDATORY JOURNALS

Many studies have documented the problem of predatory journals. These journals do not adhere to quality and ethical publishing standards, often use deceptive language in emails to encourage authors to submit their manuscripts to them, are open access but may not be transparent with the article processing charge, may have quick but questionable peer review, and may publish inaccurate information on their Web sites such as impact factor and indexing. 3 – 6 Predatory publishing is an issue in many fields including nursing. In a recent study, 127 predatory journals were identified in nursing. 7

Citations acknowledge the ideas of others and give credit to the authors of the original work. When articles are cited in a subsequent publication, those citations disseminate the information beyond the original source, and the article in which it is cited might in turn be referenced again, transferring knowledge from one source to yet another. When articles in predatory journals are cited, the same process occurs. Those citations transfer knowledge from the predatory publication beyond that source. Studies have found that authors are citing articles published in predatory journals in nursing as well as other fields. 7 – 10 Nurse scholars need to be confident as they search the literature that they are accessing sound information sources and not articles from predatory journals.

NATIONAL LIBRARY OF MEDICINE INFORMATION RESOURCES

The National Library of Medicine (NLM) supports researchers and clinicians through its multiple health information resources including PubMed, MEDLINE, and PubMed Central (PMC). PubMed serves as the search engine to access the MEDLINE database, PMC, and books, chapters, and other documents that are indexed by the NLM. PubMed is free and publicly available: by using PubMed, researchers can search more than 30 million citations to the biomedical literature. 11 The majority of records in PubMed are from MEDLINE, which has citations from more than 5200 scholarly journals. For inclusion in MEDLINE, journals are assessed for their quality by the Literature Selection Technical Review Committee. 12 Five areas are included in this assessment: scope of the journal (ie, in a biomedical subject); quality of the content (validity, importance of the content, originality, and contribution of the journal to the coverage of the field); editorial standards and practices; production quality (eg, layout and graphics); and audience (content addresses health care professionals).

PMC includes journal citations and full-text articles that are selected by the NLM for digital archiving. To be included in PMC, journals are evaluated for their scope and scientific, editorial, and technical quality. 13 Journals considered for inclusion are evaluated by independent individuals both inside and outside PMC. 14 PMC serves as the repository for articles to meet the compliance requirements of the National Institutes of Health (NIH) and other funding agencies for public access to funded research. About 12% of the articles in PMC are deposited by individual authors to be in compliance with funders and 64% by publishers, scholarly societies, and other groups. 15 Beginning in June 2020, as a pilot program, preprints reporting research funded by the NIH also can be deposited in PMC. 16

CINAHL AND SCOPUS

The journal assessment and indexing processes for CINAHL and Scopus are similar to those used by the NLM. However, as private corporations, EBSCO (CINAHL) and Elsevier (Scopus) are not required to make journal selection processes publicly available or explicit. CINAHL has an advisory board for journal selection. A CINAHL representative provided the following criteria for indexing of journals in CINAHL: high impact factor; usage in reputable subject indexes (eg, the NLM catalog); peer-reviewed journals covered by other databases (eg, Web of Science and Scopus); top-ranked journals by industry studies; and article quality (avoiding low-quality journals) (personal communication, October 19, 2020).

Elsevier's Scopus provides a webpage referring to the journal selection and assessment processes. Journals being considered for indexing in Scopus are evaluated by the Content Selection and Advisory Board and must meet the following criteria: peer-reviewed with a publicly available description of the peer review process; published on a regular basis; has a registered International Standard Serial Number (ISSN); includes references in Roman (Latin) script; has English language titles and abstracts; and has publicly available publication ethics and publication malpractice statements. 17

LITERATURE REVIEW

Studies have shown that in health care fields, researchers, clinicians, faculty, and students regularly search MEDLINE for their research and other scholarly and clinical information. 18 – 21 De Groote et al 18 found that 81% of health science faculty used MEDLINE to locate articles for their research. MEDLINE was used by the majority of faculty in each individual health care field including nursing (75%) and medicine (87.5%) for searching the literature and finding articles. In another study of 15 different resources, medical faculty and residents reported that PubMed was used most frequently for searching the databases of the NLM, primarily MEDLINE. 20 Few studies have focused on the search practices of nurses. In a review of the literature, Alving et al 22 found that hospital nurses primarily searched Google for information on evidence-based nursing. They used Google more than bibliographic databases.

The quality of content that is retrieved when using PubMed as a search engine is important considering its widespread use for accessing scholarly and clinical information in nursing and other fields. Manca et al 23 reported that articles published in predatory journals were being retrieved when conducting searches using PubMed and were a concern for researchers. Based on their studies of predatory journals in neurology 24 and rehabilitation, 25 they concluded that predatory journals “leaked into PubMed” through PMC because of less stringent criteria for inclusion of journals. 23 Citations to articles from predatory journals then could be found using the PubMed search engine. However, in a letter to the editor, Topper et al 26 from the NLM clarified that individual articles published in predatory journals might be deposited in PMC to meet the requirements of research funding and be searchable in PubMed. Topper and colleagues make a clear distinction between journals indexed in MEDLINE or PMC and citations of individual articles that were deposited in PMC to meet funder requirements.

The aim of this study was to determine whether predatory nursing journals were included in databases used by nurse researchers and other nurses when searching for information. These databases included MEDLINE (searched via PubMed), CINAHL (EBSCO), and Scopus (Elsevier) and in the Directory of Open Access Journals.

In an earlier study, 127 predatory nursing journals were identified and assessed for characteristics of predatory publications. That dataset was used for the current study. For each predatory nursing journal, information was retrieved from the NLM Catalog, Ulrichsweb, and journal and publisher Web sites. Ulrichsweb 27 provides bibliographic and publisher information on academic and scholarly journals, open access journals, peer-reviewed titles, magazines, newspapers, and other publications. Journal titles of the predatory journals were often similar to nonpredatory journals and could be easily mistaken. To ensure accuracy, the information for each journal was checked for consistency between these sources using the ISSN, exact journal title, and publisher name. The purpose of an ISSN is to identify a publication and distinguish it from other publications with similar names. An ISSN is mandatory for all publications in many countries and having one assigned is considered a journal best practice. 28 For each predatory journal, the following data were collected if available: complete journal title; abbreviated journal title; acronym; ISSN (electronic and/or print); DOI prefix; publisher name and Web site URL; NLM index status; number of predatory journal articles cited in MEDLINE and PMC (when searching using PubMed), in CINAHL, and in Scopus; if the journal was indexed in the Directory of Open Access Journals; status in Ulrichsweb; and Google Scholar profile URL.

Counts of articles cited were checked individually by journal title, publisher, and/or ISSN. Once ISSNs (both electronic and print where available) were assembled, a search algorithm was created, which included all retrieved journal ISSNs. MEDLINE was searched via PubMed using a combination of NLM journal title abbreviations and ISSNs. CINAHL, Scopus, and the Directory of Open Access Journals were searched using a combination of ISSN, journal title abbreviation, full title, and publisher. Results were visually inspected for accuracy and alignment with dataset fields.

Data analysis

Data were collected between January and April 2020. Data were entered into an Excel spreadsheet and organized by predatory journal name; abbreviated journal title; acronym; ISSN (electronic, print); DOI prefix; Web site URL; entry in NLM Catalog (yes/no); index status; number of articles cited in PubMed, CINAHL, and Scopus; Directory of Open Access Journals (included/not included); Ulrichsweb status (active/ceased); publisher; and Google Scholar profile URL. Frequencies and medians are reported.

Of the 127 predatory nursing journals in the dataset, only 102 had ISSNs to use for the search. Eighteen of the journals had records in the NLM Catalog, but only 2 of those had ever been indexed in MEDLINE, and neither are currently indexed. These 2 journals had been published earlier by a reputable publisher but then were sold to one of the large predatory publishers. The NLM Catalog record for these journals indicates that citations of articles from them appeared in MEDLINE through 2014 for one of the journals and 2018 for other, but following their transition to the new publisher are no longer included. Consistent with the MEDLINE results, these same 2 journals had been indexed in Scopus as well. Citations of articles from one of these journals were added to Scopus up to 2014, with no articles cited thereafter. Articles from the second journal continue to be added through 2020. One additional journal from the predatory journal dataset is currently in Scopus, however, only through 2014. None of the predatory nursing journals were indexed in CINAHL based on full journal title, title abbreviation, ISSN, or publisher. Two journals in the dataset were found in the Directory of Open Access Journals.

When searching PubMed, we found citations of articles from 16 predatory nursing journals. The number of citations ranged from 1 to 372 citations (from one of the journals indexed earlier in MEDLINE but sold to a predatory publisher). The second highest number of citations (n = 168) was of articles from a predatory nursing journal that had been depositing articles in PMC (and thus were retrievable when searching PubMed) but is no longer adding new material to PMC. The other citations were of articles deposited in PMC to meet requirements of NIH and other research funding. The predatory journals in which these articles were published, however, are not indexed in MEDLINE or PMC.

There were no articles from predatory nursing journals cited in CINAHL. Scopus has citations from the 2 predatory nursing journals that are no longer indexed there: 616 that were published in one of the journals and 120 from the other. Articles from a third predatory nursing journal in the study dataset, which is currently indexed in Scopus, totaled 173 (see Table).

Abbreviation: CINAHL, Cumulative Index to Nursing and Allied Health Literature.

a Predatory nursing journals with 3 or more citations to articles.

b Search using PubMed.

This analysis documented that none of the predatory nursing journals in the study dataset were currently indexed in MEDLINE or CINAHL, and only one journal is still in Scopus. Most of the citations of articles from predatory journals found in a search of these databases are from earlier years before the journals were sold to one of the large predatory publishers. Other citations are to articles deposited in PMC in compliance with research funder requirements.

By using PubMed as a search engine and entry point to the databases of the NLM, researchers can search millions of records included in MEDLINE, or in process for inclusion, and articles from PMC deposited by publishers or authors for compliance with funders. Six million records, and about 5500 journals, can be searched in CINAHL Complete, 29 and Scopus, the largest of the proprietary databases, provides access to 24000 journals and 60 million records. 30 Results from this study show that very few articles published in predatory nursing journals find their way into a search done using PubMed and Scopus and none into CINAHL.

In a prior study, 814 citations of articles in predatory nursing journals were found in articles published in nonpredatory nursing journals. 7 Based on this current study, the conclusion can be made that these citations are not coming from searches in MEDLINE/PubMed, CINAHL, or Scopus and are likely from searches done using Google or Google Scholar as the search engine. The databases examined in this study are curated by organizations with a vested interest in maintaining and improving the quality of the research literature in those databases.

Searching multiple databases using different search engines can be frustrating and time consuming. There is overlap among MEDLINE, CINAHL, and Scopus. However, these are curated databases and, as this study found, are unlikely to return many, if any, predatory citations as part of the search results. Still, it falls on the searcher to eliminate duplicates and redundant citations. Further, certain types of literature, such as theses, dissertations, and fugitive (or “gray” literature), 31 are unlikely to be found in any of these databases, even though those citations may be important or relevant sources. Given this, it is easy to understand the intuitive appeal of Google Scholar, which provides “one stop shopping”: “From one place, you can search across many disciplines and sources: articles, theses, books, abstracts and court opinions, from academic publishers, professional societies, online repositories, universities and other web sites. Google Scholar helps you find relevant work across the world of scholarly research.” 32 Google and Google Scholar were founded with a mission to become the most comprehensive search engines in the world. While this allows someone to scour the World Wide Web and Internet for some of the most obscure facts available, at the same time, little is done to verify or validate the results that are returned. Thus, it falls on the searcher to be diligent and evaluate the results of a Google or Google Scholar search, which will include citations of articles in predatory journals. This is easily confirmed by the fact that many predatory journal Web sites promote the Google Scholar logo as a sign of indexing or a badge of legitimacy.

Another vexing issue that was revealed in this study is that of reputable journals that have been bought by predatory publishers. This study found 2 journals in this category. Brown 33 reported on 16 medical specialty journals that were purchased from 2 Canadian commercial publishers by a predatory publisher. In all these cases, it is the same predatory publisher, although some of the purchases were made under a different business imprint, adding further confusion to an already muddied situation. Jeffrey Beall, who coined the term “predatory publisher” and maintained the blog “Scholarly Open Access” for almost a decade, was quoted by Brown 33 : “[The company] is not only buying journals, it is buying metrics and indexing, such as the journals' impact factors and listing in Scopus and PubMed, in order to look legitimate.” One positive finding from this study was that the 2 purchased journals that were identified were quickly de-accessioned by the NLM and are no longer indexed in MEDLINE, although citations from their pre-predatory era remain intact.

Recommendations

All of this presents a confusing picture, but it is possible to make some specific recommendations to aid researchers, clinicians, faculty, and students in their literature searches. First, become familiar with the journals and publications in your field. This is a basic foundation of scholarship. As you read articles, remember where they were published, learn journal titles, and focus on sources as well as the content. As you come across predatory journals in nursing and health care, make note of them and learn their titles too. Remember that many predatory journals adopt names that are intended to be confusing and may differ from a legitimate journal by only one letter, such as “Africa” and “African.”

Second, consider carefully how to approach your search from the outset. If you choose to start with MEDLINE (searched via PubMed), CINAHL, or Scopus, then you can have some assurance that the results will not return citations from predatory journals—although you should still verify every citation that you receive. On the other hand, Google and Google Scholar can be a “quick and easy” way to get started but will require that you carefully review and evaluate the results. If you need to venture to other more specialized databases, such as PsycInfo or ERIC (Education Resources Information Center), it is important to carefully inspect the results that you receive. To reduce the risk of including a predatory journal article in research, nursing scholars should use reputable bibliographic databases, which have clear criteria for journal indexing, for their searches.

Third, when you come across a journal title that is not familiar, take time to research it, visit the journal Web site and evaluate the information at the Web site, and determine whether it is a credible source to include in your results. If something seems irregular, then it is worth your time to do more investigating—either on your own or by enlisting the help of a knowledgeable colleague or librarian. Journals change publishers all the time, and while most of these business transfers are benign and probably will not impact you as an end consumer of the literature, that is not always the case. Likewise, the major publishers in the world today are large, multinational conglomerates that regularly spin off or purchase other companies. While this probably will not impact you on a day-to-day basis, it is important to investigate any irregularities when conducting a search of the literature.

Last, because these issues are complex and multifaceted, it is always wise to consult with a librarian who can assist you in every step of the search process. Their knowledge and expertise in information literacy, data sources, and searching techniques can help to ensure that you find the information you need from sources that are reliable and credible.

Researchers, clinicians, faculty, and students need to be careful not to include citations from predatory sources in their literature searches and articles. Predatory journals publish low-quality studies and citing this work erodes the scholarly literature in nursing. The findings of this study offer some reassurance to those who search the professional nursing literature: if you begin a search in a database such as MEDLINE, CINAHL, or Scopus, then the results will probably not include citations to predatory publications. Google and Google Scholar searches, however, may very well include predatory citations, and in that case, it is the searcher's responsibility to carefully evaluate the output and discard findings from nonlegitimate sources. Enlisting the help of a librarian is always beneficial and highly recommended.

Peggy L. Chinn, PhD, RN, FAAN, Editor, Advances in Nursing Science , is a member of our research team and contributed to the study and preparation of the manuscript.

The authors have disclosed that they have no significant relationships with, or financial interest in, any commercial companies pertaining to this article.

Home > PAIJ > Vol. 7 (2024) > Iss. 1

Effects and Nursing Considerations for Equine-Assisted Activities and Therapies for Children with Autism Spectrum Disorders: A Literature Review

Namiko Kawamura , Siga University of Medical Science Follow Mayu Sakamoto , Shiga University of Medical Science Follow Kayoko Machida , Sapporo City University Follow

This literature review aimed to analyze the effects and nursing challenges associated with equine-assisted activities and therapies (EAATs) for children with autism spectrum disorders (ASD). The study utilized the PubMed, CINAHL, and MEDLINE databases to identify 24 relevant articles. The effective contents were classified into two major categories: effects on interpersonal relationships, and effects attributable to the physical and emotional aspects of the lives of the children. The medical staff involved were mainly occupational therapists, followed by physical therapists and speech-language pathologists. The included studies also mention the involvement of trained equine therapists and volunteers, but not the involvement of nurses.

Considering the unique characteristics of EAATs in various settings and the individual needs of the recipients of the therapy, this study highlights the importance of tailoring therapy to individual needs. Nurses should be aware of the potential benefits of EAATs in improving the overall well-being of children with ASD and should consider collaborating with other health care professionals to provide comprehensive care.

Recommended Citation

Kawamura, Namiko; Sakamoto, Mayu; and Machida, Kayoko (2024) "Effects and Nursing Considerations for Equine-Assisted Activities and Therapies for Children with Autism Spectrum Disorders: A Literature Review," People and Animals: The International Journal of Research and Practice : Vol. 7 : Iss. 1, Article 9. Available at: https://docs.lib.purdue.edu/paij/vol7/iss1/9

Included in

Occupational and Environmental Health Nursing Commons , Psychiatric and Mental Health Nursing Commons

• Journal Home
• About This Journal
• Aims & Scope
• Editorial Board
• Submit Article
• Most Popular Papers
• Receive Email Notices or RSS

Advanced Search

ISSN: 2575-9078

Supported by Purina

Home | About | FAQ | My Account | Accessibility Statement

Privacy Copyright