Appraisal of a study

 

• Are the aims stated?
• Literature reviewed?
• Relevant to GP, to my patients?
• Does study simply replicate others or offer something new?

Method

• Is study design appropriate?
  • qualitative/quantitative
  • observational/experimental
  • retrospective/prospective
  • longitudinal/cross-sectional

Instruments

• Are they appropriate to concepts?
• Have they been validated in practice?

Subjects

• Recruitment
  • Are study and control groups equally matched?
  • Age/sex/class
  • Were they recruited in the same way?
  • Sensible inclusions, clear criteria?
  • Do exclusions influence applicability?
  • Subjects studied in “real life” circumstances?
• Design
  • Are the intervention and comparison clear?
  • Are outcome criteria defined?
  • What was actually measured or done? (Beware surrogate markers)
  • Is sample size sufficient? (To indicate clinical and statistical significance) was “power” calculated?
    b-error = no effect seen when one exists -due to small sample size
    a-error = significant effect due to sampling error
  • Is randomsation/allocation described? Is it concealed?
  • Was assessment blind?
  • Remember biases
    • selection/inclusion
    • performance
    • exclusion
    • detection
  • Follow up – were completeness and duration OK?

Results

• Consider drop outs/exclusions (analysis by intention to treat?)
• Are tables and graphs honest?
• Are statistics appropriate? – CI’s, p-values.
• Is clinical significance achieved?
• Were all subjects accounted for at the conclusion of the study?

Discussion

• Aware of methodological/design limitations?
• Conclusions justified?
• Compare with previous work?
• Are speculations appropriate and realistic?

Application of findings

• Are the patients comparable to your own?
• Is the setting comparable?
• Are the results clinically significant?
• Is the intervention feasible in your own practice setting?
• Is it affordable, available and sensible?