Pediatric Residents and Interns in an Italian Hospital Perform Improved Bibliographic Searches when Assisted by a Biomedical Librarian
A Review of:
Gardois, P., Calabrese, R., Colombi, N., Lingua, C., Longo, F., Villanacci, M., Miniero, R., & Piga, A. (2011). Effectiveness of bibliographic searches performed by paediatric residents and interns assisted by librarian. A randomised controlled trial. Health Information and Libraries Journal, 28(4), 273-284. doi: 10.1111/j.1471-1842.2011.00957.x
Bradford Teaching Hospitals NHS Foundation Trust
Bradford, United Kingdom
Received: 27 Nov. 2012 Accepted: 22 Jan. 2013
2013 Stone. This is an Open Access article distributed under the terms of the Creative Commons‐Attribution‐Noncommercial‐Share Alike License 2.5 Canada (http://creativecommons.org/licenses/by‐nc‐sa/2.5/ca/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly attributed, not used for commercial purposes, and, if transformed, the resulting work is redistributed under the same or similar license to this one.
Objective – To establish whether the assistance of an
experienced biomedical librarian delivers an improvement in the searching of
bibliographic databases as performed by medical residents and interns.
Design – Randomized controlled trial.
Setting – The pediatrics
department of a large Italian teaching hospital.
Subjects – 18 pediatric
residents and interns.
Methods – 23 residents and interns from the pediatrics department of a large Italian teaching hospital
were invited to participate in this study, of which 18 agreed. Subjects were
then randomized into two groups and asked to spend between 30 and 90 minutes
searching bibliographic databases for evidence to answer a real-life clinical
question which was randomly allocated to them. Each member of the intervention
group was provided with an experienced biomedical librarian to provide
assistance throughout the search session. The control group received no
assistance. The outcome of the search was then measured using an assessment
tool adapted for the purpose of this study from the Fresno test of competence
in evidence based medicine. This adapted assessment tool rated the “global
success” of the search and included criteria such as appropriate question
formulation, number of PICO terms translated into search terms, use of Boolean
logic, use of subject headings, use of filters, use of limits, and the
percentage of citations retrieved that matched a gold standard set of citations
found in a prior search by two librarians (who were not involved in assisting
the subjects) together with an expert clinician.
Main Results – The intervention group scored a median
average of 73.6 points out of a possible 100, compared with the control group
which scored 50.4. The difference of 23.2 points in favour of the librarian
assisted group was a statistically significant result (p value = 0.013) with a
95% confidence interval of between 4.8 and 33.2.
Conclusion – This study presents credible evidence that
assistance provided by an experienced biomedical librarian improves the quality
of the bibliographic database searches performed by residents and interns using
real-life clinical scenarios.
Searching for evidence is a core activity in the practice of evidence based medicine and hence there is an obvious opportunity for librarians, as expert searchers, to play a major supporting role. Although previous studies have already examined a range of library based interventions aimed at improving search performance for health evidence, the lack of reliable research for one-to-one librarian support for healthcare staff is the justification for this study. Indeed the authors voice a more general discontent with the quality of research in this area by highlighting the lack of experimental study designs employed and the subjective manner in which outcomes have often been measured.
tested in this study is one tailored more for one-off evidence based medicine
projects such as devising local guidelines or commissioning new services,
rather than as an adjunct to routine patient care where clinicians will
presumably struggle to find the time to search bibliographic databases (with or
without the support of a librarian) between patient appointments.
The use of an
assessment tool (albeit one that appears to have not been validated) to measure
the quality of the search process is a main strength of this study. This
combined with the random allocation of subjects to intervention/control groups
(which reduces the impact of confounding variables) elevates the study to a
very high level of evidence. The use of an objective assessment tool minimizes
the effects of observer bias, a factor which has potentially distorted studies
of similar interventions where subjects have been asked to assess their own
The sample size of
this study was small (only 18 participants), but it was adequately powered to
produce a statistically significant result due to the large difference between
the outcomes in the two groups. However, while we can be confident that
librarian assistance causes an improved search performance, we can be less sure
about the magnitude of this improvement. The wide 95% confidence interval
signifies that we can only be confident that the extent of improvement lies
somewhere between 4.8 and 33.2 points. This means librarians will have to ask
themselves if they can justify providing this kind of one-to-one support if the
true impact of this assistance is at the lower end of this confidence interval.
A larger sample size would have produced a narrower confidence interval and
therefore allowed us to assess the effect size of this intervention with
The generalizability of these results to the wider population of medical residents and interns is strengthened by the high participation rate in this study. 78% of invitees were recruited in to the study and so the possibility of a non-response bias (where only confident searchers agree to participate) resulting in an unreflective sample is minimized. However, the degree to which we can generalize from interns and residents to a more general clinical population is uncertain. For instance, one might expect interns and residents who are not long out of medical school to be comparatively better searchers of bibliographic databases than their more experienced clinical colleagues and consequently less likely to benefit from the intervention. But despite the possibility of a selection bias, the effect of which would most probably be to moderate the positive result of this intervention, this randomized controlled trial provides reliable evidence that bibliographic databases searches are improved when clinicians are supported by a librarian.
Librarians can take a couple of key points from this study, one quite specific and the other more general. Sub-category analysis reveals that the particular component of the intervention that was most beneficial to clinicians was the help translating PICO terms into search terms, so librarians would be well advised to focus their efforts in this direction. More generally, librarians now have compelling evidence showing how they can improve the evidence based medicine process by helping the clinician to search the literature more effectively. Librarians can therefore push confidently (if they are not doing so already) for greater involvement in evidence based medicine projects within their organizations.
Evidence Based Library and Information Practice (EBLIP) | EBLIP on Twitter