Reporting quality of systematic review abstracts in articles hand and wrist pathology: a review

Introduction

The biomedical evidence base is expanding at an exponential rate with data repositories such as PubMed currently housing more than 23 million references from more than 5600 journals.¹ In addition, 8000 systematic reviews are indexed each year, making it increasingly difficult for clinicians to remain contemporary with either primary or synthesised evidence.² One way in which clinicians have responded to this issue is to subscribe to a journal’s free abstract service or to read abstracts within free digital repositories, in order to obtain a brief synopsis of the information. The cost of article access can be prohibitive to clinicians without academic or institutional affiliations, especially for large or popular journals. Although a thorough evaluation of the full text of an article is preferable, there is some evidence that readers are more than twice as likely to read only the abstract, especially when they are unable to gain access to the full text.^3,4 It is therefore possible that important healthcare decisions are being based on information that is contained within abstracts.⁵ Recognising this, a number of reporting guidelines have been developed in order to improve the quality of abstract reporting.^3,6

The preferred reporting items for systematic reviews and meta-analyses for abstracts (PRISMA-A) checklist was developed in 2013 as an extension of the PRISMA statement with the aim of providing more focused guidance on writing abstracts for systematic reviews.⁷ Although the PRISMA-A checklist was not intentionally developed to improve how systematic reviews and meta-analyses are planned and carried out, it is implicit that setting standards in reporting could have an impact on how they are undertaken.^7,8 The PRISMA-A checklist focuses on accurate representation of a systematic review in an abstract with emphasis placed on evaluations of interventions where meta-analyses have been conducted.³ The checklist covers the title, objectives, eligibility criteria, information sources, risk of bias assessment, included studies, results synthesis and findings, strengths and limitations, interpretation of findings, funding sources and protocol registration.

The PRISMA-A checklist has been adapted to evaluate the quality of systematic review abstract reporting in conference proceedings as well as in dental, general medical, oncological and mental health literature.^9–14 However, it has not previously been used to examine the quality of abstract reporting within plastic surgery. The aim of this study was to evaluate the quality of abstract reporting in systematic reviews pertaining to surgical management of hand and wrist pathology and to identify characteristics associated with improved reporting.

Materials and methods

We used the Ovid SP platform to search MEDLINE®, EMBASE and the Cochrane Library from inception to December 2017 to identify systematic reviews addressing surgical interventions for hand and wrist pathology. In MEDLINE®, we used medical subject headings (MeSH) and free text terms related to ‘hand’ or ‘wrist’ or ‘upper limb’ and ‘systematic review’ or ‘meta-analysis’. The reference lists of included articles were also examined. The search strategy used for this systematic review was adapted from our previous study that assessed the methodological and reporting quality of systematic reviews in hand and wrist pathology.¹⁵

Study selection criteria and procedures

Our inclusion criteria were as follows:

1. Systematic review alone or in combination with meta-analysis, as defined in the title, abstract or main text;

2. A specific focus on pathology of the hand and/ or wrist; and

3. Inclusion of a greater number of studies pertaining to surgical rather than non-surgical interventions. Surgical interventions were defined as invasive procedures including endoscopic or open repair, while non-surgical interventions were defined as non-invasive treatment such as cast splinting.

Our exclusion criteria were:

1. Narrative reviews, guidelines, editorials and opinion pieces;

2. Studies that were not available in English or full text; and

3. studies that addressed the upper limbs excluding hand and wrist (for example, elbow, shoulder). Articles that were not available in English were excluded as we did not have access to translation services. Two authors (JW, AS) screened eligible titles and abstracts against the inclusion criteria. Full-text articles were retrieved and reviewed independently by two authors (JW, AS) for final inclusion. Differences of opinion were resolved by discussion with a third author (TO). Figure 1 details the study selection process.

Fig 1.Outline of study selection process (PRISMA flow diagram).

Source: Moher D1, Liberati A, Tetzlaff J, Altman DG; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009 Jul 21;6(7):e1000097. DOI: 10.1371/journal.pmed.1000097. For more information visit: www.prisma-statement.org

Methodological assessment

Each PRISMA-A item was assessed by two authors (AS, TO) and recorded with a ‘yes’, ‘no’ or ‘not applicable’ response using an electronic data collection form. A single point score was given to each item if reporting was adequate, and no points were given if reporting was not stated or not applicable. Consequently, the possible score for each individual systematic review abstract ranged from 0 to 12.

Data extraction

Two authors (AS, TO) independently extracted the general characteristics of each systematic review such as author details (number, country of origin), study descriptors (review category, disease state, intervention, number of studies), bibliographical features (journal name and impact factor [IF], year of publication, word limit for abstracts, structured format [introduction, methods, results and discussion—IMRAD] and whether the journal endorsed the PRISMA statement). These characteristics were selected as descriptive comparators and have been reported in other studies as being associated with improved review quality.^14–16

Statistical methods

Descriptive statistics were summarised as the mean (standard deviation—SD) for continuous variables and as the frequency (percentage) for categorical variables. A univariable linear regression model was used to investigate the association between study characteristics and the PRISMA-A score. Multivariable linear regression models were constructed to determine significant predictors for each variable. All multivariable models were adjusted for abstract length, abstract structure, whether a meta-analysis was included, impact factor and whether publication occurred before or after the PRISMA-A checklist appeared. Effect estimates were presented as mean differences (MD) and 95 per cent confidence intervals (95% CI).

Results

The initial search strategy resulted in 810 studies. After removing the duplicate articles, independent review of titles and abstracts by two authors (AS, TO) identified 201 potentially relevant articles. Of these, 87 were excluded because they failed to meet the inclusion criteria. Therefore, a total of 114 studies were included in this study.

Study characteristics

The characteristics of the included studies are shown in Table 1. Interventions for fracture management (37.7%, n = 43) and arthritis (16.7%, n = 19) were strongly represented in the included reviews. Many authors originated from the Americas (43.9%, n = 50), followed by Europe (36.8%, n = 42) and Australasia (19.3%, n = 22). The majority of systematic reviews had fewer than six authors (76.3%, n = 87). There were 47 systematic reviews (41%) that included a meta-analysis.

PRISMA-A checklist assessment

The mean (SD) overall score was 3.6 (1.7), with both the highest (8) and lowest (0) scores observed in hand-specific journals. The proportion of articles adhering to each of the PRISMA items is presented in Table 2. The checklist items that were most reported were title (item 1, 83.3%), study objectives (item 2, 50%) and interpretation of findings (item 10, 58.8%). The remaining items were reported adequately in less than 40 per cent of studies, with the lowest-scoring checklist components relating to risk of bias (item 5, 16.7%), registration (item 12, 0.9%) and funding source (item 11, 0%).

Table 3 displays the univariable and multivariable relationship between study demographics and PRISMA-A score. In the univariable analysis, our results showed higher PRISMA-A scores when systematic reviews incorporated a meta-analysis and an abstract greater than 300 words and were published in the Cochrane Library. Conversely, lower PRISMA-A scores were seen with systematic reviews published in hand and wrist journals, reviews addressing specific clinical topics or abstracts using an unstructured format.

In the multivariable model, our results showed improvements in PRISMA-A scores when the systematic review incorporated an abstract greater than 300 words (MD = 3.1; 95% CI [1.5, 4.6]; p < 0.001) and meta-analysis (MD = 0.64, 95% CI [0.05, 1.22]; p = 0.034). Conversely, lower PRISMA-A scores were observed in systematic reviews published in hand and wrist journals (MD = –0.97, 95% CI [–1.89, –0.04]; p = 0.04) using an unstructured format (MD = 0.65, 95% CI [–1.28, –0.02]; p = 0.044).

Discussion

Summary of findings

The present study investigated the completeness of systematic review abstracts published in the field of hand and wrist pathology using the PRISMA-A checklist. Only three PRISMA-A items were commonly reported more than 50 per cent of the time, with the remaining nine items being infrequently met. While the reporting of abstracts included some description about recording an appropriate title, along with study objectives and details about overall interpretation, the majority of abstracts failed to report adequately on their eligibility criteria, information sources, risk of bias, included studies, methods used for evidence synthesis, description of effect, strengths and limitations, study registration and funding sources. We identified a number of study characteristics that were associated with improved abstract reporting quality. Multivariable analysis revealed that systematic reviews that included a meta-analysis or had an abstract length greater than 300 words had a greater mean score, while publications in hand- or wrist-specific journals had a lower mean score.

Agreements and disagreements with other reviews

While we could not find existing studies that specifically assessed the reporting quality of abstracts with a focus on pathology of the hand and wrist, a number of studies have investigated abstract reporting in other specialties. In a study by Gomez-Garcia and colleagues,¹⁷ which investigated adherence to reporting standards and methodological quality using multiple tools including PRISMA-A in 139 systematic reviews on psoriasis interventions, the authors found items such as study objectives, description of effect and overall interpretation to be present in almost all abstracts. However, consistent with the findings of this study, most abstracts failed to adequately report risk of bias, strengths and limitations, study registration details and disclosure of sources of funding. Similarly, Hopewell and colleagues⁹ evaluated the quality of reporting of systematic review abstracts presented at scientific medical conferences and found that many failed to report the date of search, assessment of risk of bias, and the result for the main efficacy outcomes including the number of studies and participants, harms, strengths and limitations of the evidence, and funding source. Koller and colleagues¹³ also evaluated the quality of reporting in the abstracts of 182 systematic reviews and found areas that were most lacking included the search process, study registration details, strengths and limitations, and funding sources. Similarly, Bigna, Um and Nansseu¹² found the quality of 84 systematic review abstracts to be deteriorating over time with respect to strengths and limitations and funding, whereas Tsou and Treadwell’s¹¹ assessment of 200 randomly selected abstracts of systematic reviews of health interventions noted that less than 50 per cent of abstracts contained a risk of bias assessment.

Study strengths and limitations

This study assessed the reporting quality of systematic review abstracts in hand and wrist surgery, which is a highly relevant topic given the large numbers of studies in the field being published annually. The methodological strengths of our study included a comprehensive search strategy incorporating multiple electronic databases and plastic surgery journals. This allowed us to identify a large number of relevant studies based on our inclusion criteria and also showed that the search strategy was easily replicable. Furthermore, we performed independent study selection, data extraction and PRISMA-A scoring, reducing the risk of human error during this process.

There are also some limitations to this study. Our sample included studies that exclusively focused on a subsection of plastic and orthopaedic surgery literature. As such, the results may not be applicable to other areas of surgery or those relating to medical interventions. We also excluded non-English systematic reviews, which may have led to selection bias. However, very few relevant articles in other languages were noted during our search screening process, and we believe that their exclusion is unlikely to affect our results. In addition, we noted that the PRISMA-A checklist contained items (for example, item 5 risk of bias) that could only be applied to systematic reviews that included a randomised controlled trial (RCT). In our sample, many studies contained few or no RCTs, which may have impacted their ability to satisfy certain items in the checklist. However, the inability of a small proportion of studies to adhere to certain elements of the checklist does not explain the overall suboptimal quality of systematic review abstract reporting observed in this study.

Implications for clinical research and practice

The results of this study indicate that the overall completeness of abstracts within systematic reviews focusing on the pathology of the hand and wrist requires improvement. While a number of journals endorse the PRISMA tool to aid with reporting of systematic reviews, the abstract extension (PRISMA-A) is less widely used. Its endorsement could help authors to become aware of the fundamental elements of abstracts and thereby improve the quality of reporting. Our findings indicated that abstracts that were structured and allowed for a greater than 300-word limit were associated with higher PRISMA-A scores. We suggest that journal editors consider exercising lenience with systematic review abstract word limits to enable authors to better adhere to reporting guidelines.¹⁰

Conclusions

Abstracts of systematic reviews published in hand and wrist pathology literature demonstrate suboptimal reporting quality as assessed by the PRISMA-A checklist. Given that abstracts are the first-line tool for authors to convey relevant information within the full text to readers, it is important that efforts be made to improve their reporting. This could be done by endorsement of PRISMA-A guidelines by authors and journals and by reducing constraints on abstract length.

Conflict of interest

The authors have no conflicts of interest to disclose.

Funding declaration

The authors received no financial support for the research, authorship and/or publication of this article.