Advertisement

Quality of reporting of meta-analyses: the QUOROM statement. Will it help?

      The number of meta-analyses of randomized clinical trials (RCTs) has increased markedly in recent years. This method of aggregating different and often conflicting results from similar randomized clinical trials has achieved a significant position in providing useful information for evidence-based medicine.
      In parallel with this development there has been an increased focus on methodology in this field. Several aspects have been discussed and explored including the influence of study design, combinability of trial results, control of bias, statistical analysis, and applicability of the results [
      • Sacks H.S
      • Berrier J
      • Reitman D
      • Ancona-Berk V.A
      • Chalmers T.C
      Meta-analyses of randomized controlled trials.
      ].
      As a sequel to the CONSORT initiative to improve quality of reporting of RCTs [
      • Begg C
      • Cho M
      • Eastwood S
      • Horton R
      • Moher D
      • Olkin I
      • et al.
      Improving the quality of reporting of randomized controlled trials. The CONSORT statement.
      ], a recent report has been published in the Lancet presenting a checklist to ensure high quality of reporting of meta-analyses (the QUOROM statement) [
      • Moher D
      • Cook D.J
      • Eastwood S
      • Olkin I
      • Rennie D
      • Stroup D.F
      Improving the quality of reports of meta-analyses of randomized controlled trials: the QUOROM statement.
      ]. This was the result of a 2-day conference in 1996 with the participation of 30 experts (clinical epidemiologists, clinicians, statisticians and researchers conducting meta-analyses as well as editors interested in meta-analysis). Since documentation could only be found for eight of the 18 items dealt with, the authors do not consider the QUOROM statement to be the final truth. Thus the authors invite interested readers, reviewers, researchers and editors to use the QUOROM statement and to generate ideas for its improvement. The QUOROM statement is available on The Lancet’s website: htp://www.thelancet.com.
      The QUOROM statement [
      • Moher D
      • Cook D.J
      • Eastwood S
      • Olkin I
      • Rennie D
      • Stroup D.F
      Improving the quality of reports of meta-analyses of randomized controlled trials: the QUOROM statement.
      ] comprises the following 18 items:
      • 1. The title should identity the report as a meta-analysis (or systematic review) of RCTs.
      • 2. The abstract should use a structured format applying these sections:
        • 3. Objectives: describing the clinical question explicitly.
        • 4. Data sources: describing the databases (i.e. list) and other information sources.
        • 5. Review methods: describing the selection criteria (i.e. population, intervention, outcome, and study design); methods for validity assessment, data abstraction, and study characteristics, and quantitative data synthesis in sufficient detail to permit replication.
        • 6. Results: describing the characteristics of the RCTs included and excluded, qualitative and quantitative findings (i.e. point estimates and confidence intervals) and subgroup analyses.
        • 7. Conclusion: describing the main results.
      • 8. Introduction describing the explicit clinical problem, biological rationale for the intervention, and rationale for review.
      • Methods
        • 9. Searching: describing the information sources in detail (e.g. databases, registers, personal files, expert informants, agencies, hand-searching), and any restrictions (years considered, publication status, language of publication).
        • 10. Selection: describing the inclusion and exclusion criteria (defining population, intervention, principal outcomes, and study design).
        • 11. Validity assessment: describing the criteria and process used (e.g. masked conditions, quality assessment, and their findings).
        • 12. Data abstraction: describing the process or processes used (e.g. completed independently, in duplicate).
        • 13. Study characteristics: describing the type of study design, participants’ characteristics, details of intervention. outcome definitions, affiliations, and how clinical heterogeneity was assessed.
        • 14. Quantitative data synthesis: describing the principal measures of effect (e.g. relative risk), method of combining results (statistical testing and confidence intervals), handling of missing data, how statistical heterogeneity was assessed; a rationale for any a priori sensitivity and subgroup analyses; and any assessment of publication bias.
      • Results
        • 15. Trial flow: providing a meta-analysis profile summarising trial flow: (A) potentially relevant RCTs identified and screened for retrieval; (B) RCTs retrieved for more detailed evaluation; (C) Potentially appropriate RCTs to be included; (D) RCTs included; (E) RCTs with usable information, indicating for each step the number of RCTs and the reasons for exclusion.
        • 16. Study characteristics: presenting descriptive data for each trial (e.g. age, sample size, intervention, dose, duration, follow-up period).
        • 17. Quantitative data synthesis: report agreement on the selection and validity assessment; present simple summary results (for each treatment group in each trial, for each primary outcome); present data needed to calculate effect sizes and confidence intervals in intention-to-treat analyses (e.g. 2×2 tables of counts, means and SDs, proportions).
      • 18. Discussion: Summarize key findings, discuss clinical inferences based on internal and external validity; interpret the results in light of the totality of available evidence; describe potential biases in the review process (e.g. publication bias); and suggest a future research agenda.
      How is the quality of reporting of meta-analyses in hepatology according to the QUOROM statement? To get an indication of this, the 15 latest hepatologic meta-analyses [
      • Poynard T
      • Leroy V
      • Cohard M
      • Thevenot T
      • Mathurin P
      • Opolon P
      • Zarski J.P
      Meta-analysis of interferon randomized trials in the treatment of viral hepatitis C: effects of dose and duration.
      ,
      • Carithers Jr, R.L
      • Emerson S.S
      Therapy of hepatitis C: meta-analysis of interferon alfa-2b trials.
      ,
      • Camma C
      • Giunta M
      • Chemello L
      • Alberti A
      • Toyoda H
      • Trepo C
      • et al.
      Chronic hepatitis C: interferon retreatment of relapsers. A meta-analysis of individual patient data. European concerted action on viral hepatitis (EUROHEP).
      ,
      • Camma C
      • Giunta M
      • Linea C
      • Pagliaro L
      The effect of interferon on the liver in chronic hepatitis C: a quantitative evaluation of histology by meta-analysis.
      ,
      • Schalm S.W
      • Hansen B.E
      • Chemello L
      • Bellobuono A
      • Brouwer J.T
      • Weiland O
      • et al.
      Ribavirin enhances the efficacy but not the adverse effects of interferon in chronic hepatitis C. Meta-analysis of individual patient data from European centers.
      ,
      • Camma C
      • Almasio P
      • Craxi A
      Interferon as treatment for acute hepatitis C. A meta-analysis.
      ,
      • Camma C
      • Giunta M
      • Pinzello G
      • Morabito A
      • Verderio P
      • Pagliaro L
      Chronic hepatitis C and interferon alpha: conventional and cumulative meta-analyses of randomized controlled trials.
      ,
      • Bernard B
      • Lebrec D
      • Mathurin P
      • Opolon P
      • Poynard T
      Beta-adrenergic antagonists in the prevention of gastrointestinal rebleeding in patients with cirrhosis: a meta-analysis.
      ,
      • Papatheodoridis G.V
      • Goulis J
      • Leandro G
      • Patch D
      • Burroughs A.K
      Transjugular intrahepatic portosystemic shunt compared with endoscopic treatment for prevention of variceal rebleeding: a meta-analysis.
      ,
      • Bernard B
      • Lebrec D
      • Mathurin P
      • Opolon P
      • Poynard T
      Propranolol and sclerotherapy in the prevention of gastrointestinal rebleeding in patients with cirrhosis: a meta-analysis.
      ,
      • Goulis J
      • Burroughs A.K
      Role of vasoactive drugs in the treatment of bleeding oesophageal varices.
      ,
      • Luca A
      • D'Amico G
      • La Galla R
      • Midiri M
      • Morabito A
      • Pagliaro L
      TIPS for prevention of recurrent bleeding in patients with cirrhosis: meta-analysis of randomized clinical trials.
      ,
      • Bernard B
      • Grange J.D
      • Khac E.N
      • Amiot X
      • Opolon P
      • Poynard T
      Antibiotic prophylaxis for the prevention of bacterial infections in cirrhotic patients with gastrointestinal bleeding: a meta-analysis.
      ,
      • Bernard B
      • Grange J.D
      • Khac E.N
      • Amiot X
      • Opolon P
      • Poynard T
      Antibiotic prophylaxis for the prevention of bacterial infections in cirrhotic patients with ascites: a meta-analysis.
      ,
      • Goulis J
      • Leandro G
      • Burroughs A.K
      Randomized controlled trials of ursodeoxycholic-acid therapy for primary biliary cirrhosis: a meta-analysis.
      ] were identified in a Medline search. Thirteen were based on summarized trial results (‘classical’ meta-analyses) and two were based on individual patient data [
      • Camma C
      • Giunta M
      • Chemello L
      • Alberti A
      • Toyoda H
      • Trepo C
      • et al.
      Chronic hepatitis C: interferon retreatment of relapsers. A meta-analysis of individual patient data. European concerted action on viral hepatitis (EUROHEP).
      ,
      • Schalm S.W
      • Hansen B.E
      • Chemello L
      • Bellobuono A
      • Brouwer J.T
      • Weiland O
      • et al.
      Ribavirin enhances the efficacy but not the adverse effects of interferon in chronic hepatitis C. Meta-analysis of individual patient data from European centers.
      ]. Ten were published in major journals (Journal of Hepatology, Hepatology and The Lancet) [
      • Poynard T
      • Leroy V
      • Cohard M
      • Thevenot T
      • Mathurin P
      • Opolon P
      • Zarski J.P
      Meta-analysis of interferon randomized trials in the treatment of viral hepatitis C: effects of dose and duration.
      ,
      • Carithers Jr, R.L
      • Emerson S.S
      Therapy of hepatitis C: meta-analysis of interferon alfa-2b trials.
      ,
      • Camma C
      • Giunta M
      • Chemello L
      • Alberti A
      • Toyoda H
      • Trepo C
      • et al.
      Chronic hepatitis C: interferon retreatment of relapsers. A meta-analysis of individual patient data. European concerted action on viral hepatitis (EUROHEP).
      ,
      • Camma C
      • Giunta M
      • Linea C
      • Pagliaro L
      The effect of interferon on the liver in chronic hepatitis C: a quantitative evaluation of histology by meta-analysis.
      ,
      • Schalm S.W
      • Hansen B.E
      • Chemello L
      • Bellobuono A
      • Brouwer J.T
      • Weiland O
      • et al.
      Ribavirin enhances the efficacy but not the adverse effects of interferon in chronic hepatitis C. Meta-analysis of individual patient data from European centers.
      ,
      • Bernard B
      • Lebrec D
      • Mathurin P
      • Opolon P
      • Poynard T
      Beta-adrenergic antagonists in the prevention of gastrointestinal rebleeding in patients with cirrhosis: a meta-analysis.
      ,
      • Papatheodoridis G.V
      • Goulis J
      • Leandro G
      • Patch D
      • Burroughs A.K
      Transjugular intrahepatic portosystemic shunt compared with endoscopic treatment for prevention of variceal rebleeding: a meta-analysis.
      ,
      • Bernard B
      • Lebrec D
      • Mathurin P
      • Opolon P
      • Poynard T
      Propranolol and sclerotherapy in the prevention of gastrointestinal rebleeding in patients with cirrhosis: a meta-analysis.
      ,
      • Bernard B
      • Grange J.D
      • Khac E.N
      • Amiot X
      • Opolon P
      • Poynard T
      Antibiotic prophylaxis for the prevention of bacterial infections in cirrhotic patients with gastrointestinal bleeding: a meta-analysis.
      ,
      • Goulis J
      • Leandro G
      • Burroughs A.K
      Randomized controlled trials of ursodeoxycholic-acid therapy for primary biliary cirrhosis: a meta-analysis.
      ]. Three were published in supplement issues [
      • Carithers Jr, R.L
      • Emerson S.S
      Therapy of hepatitis C: meta-analysis of interferon alfa-2b trials.
      ,
      • Goulis J
      • Burroughs A.K
      Role of vasoactive drugs in the treatment of bleeding oesophageal varices.
      ,
      • Bernard B
      • Grange J.D
      • Khac E.N
      • Amiot X
      • Opolon P
      • Poynard T
      Antibiotic prophylaxis for the prevention of bacterial infections in cirrhotic patients with ascites: a meta-analysis.
      ]. Seven dealt with therapies for hepatitis C [
      • Poynard T
      • Leroy V
      • Cohard M
      • Thevenot T
      • Mathurin P
      • Opolon P
      • Zarski J.P
      Meta-analysis of interferon randomized trials in the treatment of viral hepatitis C: effects of dose and duration.
      ,
      • Carithers Jr, R.L
      • Emerson S.S
      Therapy of hepatitis C: meta-analysis of interferon alfa-2b trials.
      ,
      • Camma C
      • Giunta M
      • Chemello L
      • Alberti A
      • Toyoda H
      • Trepo C
      • et al.
      Chronic hepatitis C: interferon retreatment of relapsers. A meta-analysis of individual patient data. European concerted action on viral hepatitis (EUROHEP).
      ,
      • Camma C
      • Giunta M
      • Linea C
      • Pagliaro L
      The effect of interferon on the liver in chronic hepatitis C: a quantitative evaluation of histology by meta-analysis.
      ,
      • Schalm S.W
      • Hansen B.E
      • Chemello L
      • Bellobuono A
      • Brouwer J.T
      • Weiland O
      • et al.
      Ribavirin enhances the efficacy but not the adverse effects of interferon in chronic hepatitis C. Meta-analysis of individual patient data from European centers.
      ,
      • Camma C
      • Almasio P
      • Craxi A
      Interferon as treatment for acute hepatitis C. A meta-analysis.
      ,
      • Camma C
      • Giunta M
      • Pinzello G
      • Morabito A
      • Verderio P
      • Pagliaro L
      Chronic hepatitis C and interferon alpha: conventional and cumulative meta-analyses of randomized controlled trials.
      ], five with therapies for portal hypertension [
      • Bernard B
      • Lebrec D
      • Mathurin P
      • Opolon P
      • Poynard T
      Beta-adrenergic antagonists in the prevention of gastrointestinal rebleeding in patients with cirrhosis: a meta-analysis.
      ,
      • Papatheodoridis G.V
      • Goulis J
      • Leandro G
      • Patch D
      • Burroughs A.K
      Transjugular intrahepatic portosystemic shunt compared with endoscopic treatment for prevention of variceal rebleeding: a meta-analysis.
      ,
      • Bernard B
      • Lebrec D
      • Mathurin P
      • Opolon P
      • Poynard T
      Propranolol and sclerotherapy in the prevention of gastrointestinal rebleeding in patients with cirrhosis: a meta-analysis.
      ,
      • Goulis J
      • Burroughs A.K
      Role of vasoactive drugs in the treatment of bleeding oesophageal varices.
      ,
      • Luca A
      • D'Amico G
      • La Galla R
      • Midiri M
      • Morabito A
      • Pagliaro L
      TIPS for prevention of recurrent bleeding in patients with cirrhosis: meta-analysis of randomized clinical trials.
      ], two with antibiotic prevention of bacterial infection [
      • Bernard B
      • Grange J.D
      • Khac E.N
      • Amiot X
      • Opolon P
      • Poynard T
      Antibiotic prophylaxis for the prevention of bacterial infections in cirrhotic patients with gastrointestinal bleeding: a meta-analysis.
      ,
      • Bernard B
      • Grange J.D
      • Khac E.N
      • Amiot X
      • Opolon P
      • Poynard T
      Antibiotic prophylaxis for the prevention of bacterial infections in cirrhotic patients with ascites: a meta-analysis.
      ] and one with therapy for cholestatic liver disease [
      • Goulis J
      • Leandro G
      • Burroughs A.K
      Randomized controlled trials of ursodeoxycholic-acid therapy for primary biliary cirrhosis: a meta-analysis.
      ]. The centres of origin were Palermo [
      • Camma C
      • Giunta M
      • Linea C
      • Pagliaro L
      The effect of interferon on the liver in chronic hepatitis C: a quantitative evaluation of histology by meta-analysis.
      ,
      • Camma C
      • Almasio P
      • Craxi A
      Interferon as treatment for acute hepatitis C. A meta-analysis.
      ,
      • Camma C
      • Giunta M
      • Pinzello G
      • Morabito A
      • Verderio P
      • Pagliaro L
      Chronic hepatitis C and interferon alpha: conventional and cumulative meta-analyses of randomized controlled trials.
      ,
      • Luca A
      • D'Amico G
      • La Galla R
      • Midiri M
      • Morabito A
      • Pagliaro L
      TIPS for prevention of recurrent bleeding in patients with cirrhosis: meta-analysis of randomized clinical trials.
      ], Paris [
      • Poynard T
      • Leroy V
      • Cohard M
      • Thevenot T
      • Mathurin P
      • Opolon P
      • Zarski J.P
      Meta-analysis of interferon randomized trials in the treatment of viral hepatitis C: effects of dose and duration.
      ,
      • Bernard B
      • Lebrec D
      • Mathurin P
      • Opolon P
      • Poynard T
      Beta-adrenergic antagonists in the prevention of gastrointestinal rebleeding in patients with cirrhosis: a meta-analysis.
      ,
      • Bernard B
      • Lebrec D
      • Mathurin P
      • Opolon P
      • Poynard T
      Propranolol and sclerotherapy in the prevention of gastrointestinal rebleeding in patients with cirrhosis: a meta-analysis.
      ,
      • Bernard B
      • Grange J.D
      • Khac E.N
      • Amiot X
      • Opolon P
      • Poynard T
      Antibiotic prophylaxis for the prevention of bacterial infections in cirrhotic patients with gastrointestinal bleeding: a meta-analysis.
      ,
      • Bernard B
      • Grange J.D
      • Khac E.N
      • Amiot X
      • Opolon P
      • Poynard T
      Antibiotic prophylaxis for the prevention of bacterial infections in cirrhotic patients with ascites: a meta-analysis.
      ], London [
      • Papatheodoridis G.V
      • Goulis J
      • Leandro G
      • Patch D
      • Burroughs A.K
      Transjugular intrahepatic portosystemic shunt compared with endoscopic treatment for prevention of variceal rebleeding: a meta-analysis.
      ,
      • Goulis J
      • Burroughs A.K
      Role of vasoactive drugs in the treatment of bleeding oesophageal varices.
      ,
      • Goulis J
      • Leandro G
      • Burroughs A.K
      Randomized controlled trials of ursodeoxycholic-acid therapy for primary biliary cirrhosis: a meta-analysis.
      ] and Seattle [
      • Carithers Jr, R.L
      • Emerson S.S
      Therapy of hepatitis C: meta-analysis of interferon alfa-2b trials.
      ] Two were European co-operative studies [
      • Camma C
      • Giunta M
      • Chemello L
      • Alberti A
      • Toyoda H
      • Trepo C
      • et al.
      Chronic hepatitis C: interferon retreatment of relapsers. A meta-analysis of individual patient data. European concerted action on viral hepatitis (EUROHEP).
      ,
      • Schalm S.W
      • Hansen B.E
      • Chemello L
      • Bellobuono A
      • Brouwer J.T
      • Weiland O
      • et al.
      Ribavirin enhances the efficacy but not the adverse effects of interferon in chronic hepatitis C. Meta-analysis of individual patient data from European centers.
      ].
      The 15 meta-analyses were assessed according to the 18 QUOROM points. Since in some of these points there are a number of subpoints, some judgement had to be made in order to decide if the point could be considered fulfilled or not. The result of this crude pilot assessment is shown in Table 1 for the total sample and for a few subsamples.
      Table 1Percent of hepatologic meta-analyses fulfilling the 18 QUOROM main points
      QUOROM main pointSubsample
      TotalA
      Treatments of hepatitis C (n=7) [4–10].
      B
      Treatments of portal hypertension (n=5) [11–15].
      C
      Classical meta-analyses published in regular issues of major journals (n=7) [4,7,11–13,16,18].
      1. Title9310080100
      Abstract
       2. Structured format40576057
       3. Objectives737180100
       4. Data sources801006086
       5. Review methods20142029
       6. Results73868086
       7. Conclusion808610086
      8. lntroduction100100100100
      Methods
       9. Searching8710080100
       10. Selection737180100
       11. Validity assessment67578086
       12. Data abstraction8710080100
       13. Study characteristics878680100
       14. Quantitative data synthesis100100100100
      Results
       15. Trial flow47574071
       16. Study characteristics878680100
       17. Quantitative data synthesis57100100100
       18. Discussion9310080100
      a Treatments of hepatitis C (n=7)
      • Poynard T
      • Leroy V
      • Cohard M
      • Thevenot T
      • Mathurin P
      • Opolon P
      • Zarski J.P
      Meta-analysis of interferon randomized trials in the treatment of viral hepatitis C: effects of dose and duration.
      ,
      • Carithers Jr, R.L
      • Emerson S.S
      Therapy of hepatitis C: meta-analysis of interferon alfa-2b trials.
      ,
      • Camma C
      • Giunta M
      • Chemello L
      • Alberti A
      • Toyoda H
      • Trepo C
      • et al.
      Chronic hepatitis C: interferon retreatment of relapsers. A meta-analysis of individual patient data. European concerted action on viral hepatitis (EUROHEP).
      ,
      • Camma C
      • Giunta M
      • Linea C
      • Pagliaro L
      The effect of interferon on the liver in chronic hepatitis C: a quantitative evaluation of histology by meta-analysis.
      ,
      • Schalm S.W
      • Hansen B.E
      • Chemello L
      • Bellobuono A
      • Brouwer J.T
      • Weiland O
      • et al.
      Ribavirin enhances the efficacy but not the adverse effects of interferon in chronic hepatitis C. Meta-analysis of individual patient data from European centers.
      ,
      • Camma C
      • Almasio P
      • Craxi A
      Interferon as treatment for acute hepatitis C. A meta-analysis.
      ,
      • Camma C
      • Giunta M
      • Pinzello G
      • Morabito A
      • Verderio P
      • Pagliaro L
      Chronic hepatitis C and interferon alpha: conventional and cumulative meta-analyses of randomized controlled trials.
      .
      b Treatments of portal hypertension (n=5)
      • Bernard B
      • Lebrec D
      • Mathurin P
      • Opolon P
      • Poynard T
      Beta-adrenergic antagonists in the prevention of gastrointestinal rebleeding in patients with cirrhosis: a meta-analysis.
      ,
      • Papatheodoridis G.V
      • Goulis J
      • Leandro G
      • Patch D
      • Burroughs A.K
      Transjugular intrahepatic portosystemic shunt compared with endoscopic treatment for prevention of variceal rebleeding: a meta-analysis.
      ,
      • Bernard B
      • Lebrec D
      • Mathurin P
      • Opolon P
      • Poynard T
      Propranolol and sclerotherapy in the prevention of gastrointestinal rebleeding in patients with cirrhosis: a meta-analysis.
      ,
      • Goulis J
      • Burroughs A.K
      Role of vasoactive drugs in the treatment of bleeding oesophageal varices.
      ,
      • Luca A
      • D'Amico G
      • La Galla R
      • Midiri M
      • Morabito A
      • Pagliaro L
      TIPS for prevention of recurrent bleeding in patients with cirrhosis: meta-analysis of randomized clinical trials.
      .
      c Classical meta-analyses published in regular issues of major journals (n=7)
      • Poynard T
      • Leroy V
      • Cohard M
      • Thevenot T
      • Mathurin P
      • Opolon P
      • Zarski J.P
      Meta-analysis of interferon randomized trials in the treatment of viral hepatitis C: effects of dose and duration.
      ,
      • Camma C
      • Giunta M
      • Linea C
      • Pagliaro L
      The effect of interferon on the liver in chronic hepatitis C: a quantitative evaluation of histology by meta-analysis.
      ,
      • Bernard B
      • Lebrec D
      • Mathurin P
      • Opolon P
      • Poynard T
      Beta-adrenergic antagonists in the prevention of gastrointestinal rebleeding in patients with cirrhosis: a meta-analysis.
      ,
      • Papatheodoridis G.V
      • Goulis J
      • Leandro G
      • Patch D
      • Burroughs A.K
      Transjugular intrahepatic portosystemic shunt compared with endoscopic treatment for prevention of variceal rebleeding: a meta-analysis.
      ,
      • Bernard B
      • Lebrec D
      • Mathurin P
      • Opolon P
      • Poynard T
      Propranolol and sclerotherapy in the prevention of gastrointestinal rebleeding in patients with cirrhosis: a meta-analysis.
      ,
      • Bernard B
      • Grange J.D
      • Khac E.N
      • Amiot X
      • Opolon P
      • Poynard T
      Antibiotic prophylaxis for the prevention of bacterial infections in cirrhotic patients with gastrointestinal bleeding: a meta-analysis.
      ,
      • Goulis J
      • Leandro G
      • Burroughs A.K
      Randomized controlled trials of ursodeoxycholic-acid therapy for primary biliary cirrhosis: a meta-analysis.
      .
      In the total sample the fulfilment was not complete for many of the points. There were no obvious systematic differences between meta-analyses dealing with treatments for hepatitis C and those dealing with treatments for portal hypertension (Table 1). However, meta-analyses of the ‘classical’ type published in regular issues of major journals were complying to a high degree with the QUOROM points (Table 1).
      Some of the points which scored low concern the abstract. Part of the explanation is probably space restrictions imposed by the journals. The specified details are therefore being referred to the paper proper. Some journals (e.g. Hepatology) do not use a structured abstract format.
      Among the points concerning the methods, especially the validity assessment of the individual trials (point 11) scored rather low. This point is important, especially the degree of allocation concealment throughout the trial (see below). In a substantial number of meta-analyses, the flow of trials (point 15, see above for definition) was not adequately reported according to the QUOROM statement.
      The meta-analyses based on individual data [
      • Camma C
      • Giunta M
      • Chemello L
      • Alberti A
      • Toyoda H
      • Trepo C
      • et al.
      Chronic hepatitis C: interferon retreatment of relapsers. A meta-analysis of individual patient data. European concerted action on viral hepatitis (EUROHEP).
      ,
      • Schalm S.W
      • Hansen B.E
      • Chemello L
      • Bellobuono A
      • Brouwer J.T
      • Weiland O
      • et al.
      Ribavirin enhances the efficacy but not the adverse effects of interferon in chronic hepatitis C. Meta-analysis of individual patient data from European centers.
      ] scored particularly low on selection (point 10) and on trial flow (point 15). Since these analyses are dependent on the authors of the identified RCTs giving access to the individual data, some kind of selection bias cannot be ruled out. Furthermore, unbalanced designs [
      • Schalm S.W
      • Hansen B.E
      • Chemello L
      • Bellobuono A
      • Brouwer J.T
      • Weiland O
      • et al.
      Ribavirin enhances the efficacy but not the adverse effects of interferon in chronic hepatitis C. Meta-analysis of individual patient data from European centers.
      ] relying heavily on adjustment procedures using complex statistical methods may be another difficulty to deal with. Even when using individual data and powerful statistical methods, the inclusion of only randomized studies should still be an essential requirement.
      The fact that the fulfilment of the QUOROM points seems to be associated with the journal of publication and the type of issue (regular or supplement) is unfortunate. Any publication should adhere to the highest criteria of reporting and the QUOROM statement should be a reminder to editors, reviewers and authors in this respect.
      Improvement of the quality of reporting of meta-analyses will be a step forward, but to improve the reliability of their information, the quality of the individual RCTs, which form the basis of the meta-analyses, needs improvement. This is the main task.
      A survey of RCTs in two major hepatologic journals revealed considerable weaknesses in a large proportion of the published RCTs [
      • Gluud C
      • Nikolova D
      Quality assessment of reports on clinical trials in the Journal of Hepatology.
      ,
      • Kjaergard L.L
      • Nikolova D
      • Gluud C
      Randomized clinical trials in hepatology: predictors of quality.
      ] including: inadequate reporting of allocation sequence, inadequate allocation concealment, inadequate blinding, lack of intention-to-treat analysis, missing sample-size calculations, and a small number (<20) of patients per intervention arm.
      Allocation concealment and its maintenance throughout the study is of major importance for the reliability of the RCT results. Inadequate reporting of allocation concealment is associated with an overestimation of the intervention effect of up to 30% [
      • Schulz K.F
      • Chalmers I
      • Hayes R.J
      • Altman D.G
      Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials.
      ]. Because of its marked influence, insufficient allocation concealment needs to be identified and adjusted for in some way in the meta-analyses.
      Publication bias due to a favoured publication of the positive trials is another very important source of bias in meta-analyses. A thorough search of the literature is mandatory but identification of unpublished RCTs may be extremely difficult and the necessary extra effort may not be successful. A recent survey including 252 RCTs within a given domain shows that the percentage of positive RCTs varies with the country of origin e.g. Austria 89%, Italy 89%, France 83%, Germany 63%, UK 60%, Sweden 59%, USA 53%, Finland 50%, Denmark 50%, the Netherlands 33%, Canada 27%, a probably explanation being varying degrees of publication bias [
      • Vickers A
      • Goyal N
      • Harland R
      • Rees R
      Do certain countries produce only positive results? A systematic review of controlled trials.
      ]. One example of likely publication bias in hepatology concerns the effect of interferon in chronic hepatitis B where a large meta-analysis based on published and unpublished RCTs [
      • Krogsgaard K
      • Bindslev N
      • Christensen E
      • Craxi A
      • Schlichting P
      • Schalm S
      • et al.
      European concerted action on viralhepatitis (Eurohep). The treatment effect of alpha interferon in chronic hepatitis B is independent of pre-treatment variables. Results based on individual patient data from 10 clinical controlled trials.
      ] demonstrated a significantly smaller effect than in meta-analysis based on published data only [
      • Tine E
      • Liberati A
      • Craxi A
      • Almasio P
      • Pagliaro L
      Interferon treatment in patients with chronic hepatitis B: a meta-analysis of the published literature.
      ].
      In another field it has been demonstrated that the type of affiliation or sponsoring of the investigator was strongly associated with the conclusion of a review article [
      • Barnes D.E
      • Bero L.A
      Why review articles on the health effects of passive smoking reach different conclusions.
      ]. Some of the hepatologic meta-analyses reviewed mentioned specifically that the investigator was not supported financially by any pharmaceutical company, government agency or other grants. This point is important.
      One aspect which is not included in the QUOROM statement is the comparability between the intervention groups in the original RCTs. Imbalance in respect to variables associated with the measured outcome may be a significant problem in the smaller trials [
      • Lachin J.M
      Statistical elements of the randomized clinical trial.
      ] of which there are many in the field of hepatology. This emphasizes the importance of presenting a thorough description (including all variables which may be associated with the outcome) of each intervention group in the original RCTs. If an imbalance favours the new intervention, the trial may be positive without the intervention being effective. If the imbalance disfavours the new intervention, the trial may be negative without the intervention being ineffective. RCTs with the former type of imbalance will be published more often than RCTs with the latter imbalance type. Therefore publication bias will tend to be more pronounced if a significant number of the RCTs have a small sample size. A decrease of the intervention effect with the sample size (e.g. in a so-called ‘funnel plot’) is indicative of publication bias and such an assessment should always be performed in a meta-analysis. If publication bias is detected, the overall intervention effect can be considerably overestimated and some kind of effect-adjustment would be necessary. This implies that even more weight should be given to the largest RCTs at the expense of the small RCTs, and exclusion of the most positive, smallest RCTs from the analysis could be justified.
      It may to some degree be possible to adjust for some imbalance in important prognostic variables if these are reported in the RCTs [
      • Christensen E
      • Gluud C
      Glucocorticorticoids are ineffective in alcoholic hepatitis: a meta-analysis adjusting for confounding variables.
      ]. If considerable heterogeneity exists between the RCTs, more special methods may be applied [
      • Thompson S.G
      • Sharp S.J
      Explaining heterogeneity in meta-analysis: a comparison of methods.
      ]. However, there is a limit to how much information can be extracted from reported summarized RCT data. More elaborate results may be obtained by combining RCT databases and performing meta-analyses using the individual data from the RCTs [
      • Camma C
      • Giunta M
      • Chemello L
      • Alberti A
      • Toyoda H
      • Trepo C
      • et al.
      Chronic hepatitis C: interferon retreatment of relapsers. A meta-analysis of individual patient data. European concerted action on viral hepatitis (EUROHEP).
      ,
      • Schalm S.W
      • Hansen B.E
      • Chemello L
      • Bellobuono A
      • Brouwer J.T
      • Weiland O
      • et al.
      Ribavirin enhances the efficacy but not the adverse effects of interferon in chronic hepatitis C. Meta-analysis of individual patient data from European centers.
      ,
      • Krogsgaard K
      • Bindslev N
      • Christensen E
      • Craxi A
      • Schlichting P
      • Schalm S
      • et al.
      European concerted action on viralhepatitis (Eurohep). The treatment effect of alpha interferon in chronic hepatitis B is independent of pre-treatment variables. Results based on individual patient data from 10 clinical controlled trials.
      ]. This will make possible a more comprehensive analysis with adjustment for any confounding variables, and a study of the intervention effect in special subgroups may be performed more reliably. However, the superior approach is to perform primary RCTs having the necessary large sample size to make such analyses possible within the same RCT. Therefore, large multicenter – or even better – large multinational RCTs will provide the best chances for progress in this field. Such studies will tend to have a higher quality and the chance of publication will be high even if the results are negative. We may hope that such co-operative studies will be more common in the future.

      References

        • Sacks H.S
        • Berrier J
        • Reitman D
        • Ancona-Berk V.A
        • Chalmers T.C
        Meta-analyses of randomized controlled trials.
        N Engl J Med. 1987; 316: 450-455
        • Begg C
        • Cho M
        • Eastwood S
        • Horton R
        • Moher D
        • Olkin I
        • et al.
        Improving the quality of reporting of randomized controlled trials. The CONSORT statement.
        J Am Med Assoc. 1996; 276: 637-639
        • Moher D
        • Cook D.J
        • Eastwood S
        • Olkin I
        • Rennie D
        • Stroup D.F
        Improving the quality of reports of meta-analyses of randomized controlled trials: the QUOROM statement.
        Lancet. 1999; 354: 1896-1900
        • Poynard T
        • Leroy V
        • Cohard M
        • Thevenot T
        • Mathurin P
        • Opolon P
        • Zarski J.P
        Meta-analysis of interferon randomized trials in the treatment of viral hepatitis C: effects of dose and duration.
        Hepatology. 1996; 24: 778-789
        • Carithers Jr, R.L
        • Emerson S.S
        Therapy of hepatitis C: meta-analysis of interferon alfa-2b trials.
        Hepatology. 1997; 26: 835-885
        • Camma C
        • Giunta M
        • Chemello L
        • Alberti A
        • Toyoda H
        • Trepo C
        • et al.
        Chronic hepatitis C: interferon retreatment of relapsers. A meta-analysis of individual patient data. European concerted action on viral hepatitis (EUROHEP).
        Hepatology. 1999; 30: 801-807
        • Camma C
        • Giunta M
        • Linea C
        • Pagliaro L
        The effect of interferon on the liver in chronic hepatitis C: a quantitative evaluation of histology by meta-analysis.
        J Hepatol. 1997; 26: 1187-1199
        • Schalm S.W
        • Hansen B.E
        • Chemello L
        • Bellobuono A
        • Brouwer J.T
        • Weiland O
        • et al.
        Ribavirin enhances the efficacy but not the adverse effects of interferon in chronic hepatitis C. Meta-analysis of individual patient data from European centers.
        J Hepatol. 1997; 26: 961-966
        • Camma C
        • Almasio P
        • Craxi A
        Interferon as treatment for acute hepatitis C. A meta-analysis.
        Dig Dis Sci. 1996; 41: 1248-1255
        • Camma C
        • Giunta M
        • Pinzello G
        • Morabito A
        • Verderio P
        • Pagliaro L
        Chronic hepatitis C and interferon alpha: conventional and cumulative meta-analyses of randomized controlled trials.
        Am J Gastroenterol. 1999; 94: 581-595
        • Bernard B
        • Lebrec D
        • Mathurin P
        • Opolon P
        • Poynard T
        Beta-adrenergic antagonists in the prevention of gastrointestinal rebleeding in patients with cirrhosis: a meta-analysis.
        Hepatology. 1997; 25: 63-70
        • Papatheodoridis G.V
        • Goulis J
        • Leandro G
        • Patch D
        • Burroughs A.K
        Transjugular intrahepatic portosystemic shunt compared with endoscopic treatment for prevention of variceal rebleeding: a meta-analysis.
        Hepatology. 1999; 30: 612-622
        • Bernard B
        • Lebrec D
        • Mathurin P
        • Opolon P
        • Poynard T
        Propranolol and sclerotherapy in the prevention of gastrointestinal rebleeding in patients with cirrhosis: a meta-analysis.
        J Hepatol. 1997; 26: 312-324
        • Goulis J
        • Burroughs A.K
        Role of vasoactive drugs in the treatment of bleeding oesophageal varices.
        Digestion. 1999; 60: 25-34
        • Luca A
        • D'Amico G
        • La Galla R
        • Midiri M
        • Morabito A
        • Pagliaro L
        TIPS for prevention of recurrent bleeding in patients with cirrhosis: meta-analysis of randomized clinical trials.
        Radiology. 1999; 212: 411-421
        • Bernard B
        • Grange J.D
        • Khac E.N
        • Amiot X
        • Opolon P
        • Poynard T
        Antibiotic prophylaxis for the prevention of bacterial infections in cirrhotic patients with gastrointestinal bleeding: a meta-analysis.
        Hepatology. 1999; 29: 1655-1661
        • Bernard B
        • Grange J.D
        • Khac E.N
        • Amiot X
        • Opolon P
        • Poynard T
        Antibiotic prophylaxis for the prevention of bacterial infections in cirrhotic patients with ascites: a meta-analysis.
        Digestion (Suppl 59). 1998; 2: 54-57
        • Goulis J
        • Leandro G
        • Burroughs A.K
        Randomized controlled trials of ursodeoxycholic-acid therapy for primary biliary cirrhosis: a meta-analysis.
        Lancet. 1999; 354: 1053-1060
        • Gluud C
        • Nikolova D
        Quality assessment of reports on clinical trials in the Journal of Hepatology.
        J Hepatol. 1988; 29: 321-327
        • Kjaergard L.L
        • Nikolova D
        • Gluud C
        Randomized clinical trials in hepatology: predictors of quality.
        Hepatology. 1999; 30: 1134-1138
        • Schulz K.F
        • Chalmers I
        • Hayes R.J
        • Altman D.G
        Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials.
        J Am Med Assoc. 1995; 273: 408-412
        • Vickers A
        • Goyal N
        • Harland R
        • Rees R
        Do certain countries produce only positive results? A systematic review of controlled trials.
        Control Clin Trials. 1998; 19: 159-166
        • Krogsgaard K
        • Bindslev N
        • Christensen E
        • Craxi A
        • Schlichting P
        • Schalm S
        • et al.
        European concerted action on viralhepatitis (Eurohep). The treatment effect of alpha interferon in chronic hepatitis B is independent of pre-treatment variables. Results based on individual patient data from 10 clinical controlled trials.
        J Hepatol. 1994; 21: 646-655
        • Tine E
        • Liberati A
        • Craxi A
        • Almasio P
        • Pagliaro L
        Interferon treatment in patients with chronic hepatitis B: a meta-analysis of the published literature.
        J Hepatol. 1993; 18: 154-162
        • Barnes D.E
        • Bero L.A
        Why review articles on the health effects of passive smoking reach different conclusions.
        J Am Med Assoc. 1998; 20: 1566-1570
        • Lachin J.M
        Statistical elements of the randomized clinical trial.
        in: Tygstrup N Lachin J.M Juhl E The randomized clinical trials and therapeutic decisions. Marcel Dekker, New York1982: 77-103
        • Christensen E
        • Gluud C
        Glucocorticorticoids are ineffective in alcoholic hepatitis: a meta-analysis adjusting for confounding variables.
        Gut. 1995; 37: 113-118
        • Thompson S.G
        • Sharp S.J
        Explaining heterogeneity in meta-analysis: a comparison of methods.
        Stat Med. 1999; 18: 2693-2708