Measuring Impact: A Summary of an MLA’17 Special Session on Research Assessment Services

July 21, 2017 at 3:48 pm | Posted in Awards and Recognition, Professional Development, Uncategorized | Leave a comment

Sarah Carnes was awarded a scholarship to attend the MLA Annual Meeting. Congratulations on winning a NAHSL Professional Development Award!


As other attendees can attest, MLA ‘17 consisted of a truly vast array of sessions. From the plenary speeches to the lightning rounds, from the special sessions to the vendor demonstrations, MLA 17 provided many opportunities for professional development, process improvement, and inspiration. For me, the conference certainly lived up to the theme of “Dream, Dare, Do.”

I especially valued a special session entitled “From Dreaming to Doing: Implementing Research Assessment Services.” This panel presentation and discussion occurred on Tuesday, May 30th and was sponsored by the Medical Informatics Section, Leadership and Management Section, and the Translational Science Collaboration SIG. The presenters, health science librarians from government and academia, shared a variety of lessons learned and best practices from academic and health science libraries that provide impact assessment services. Specifically, the goals of the session were:

  • Discuss types of research impact assessment services
  • Share resources and tools for assessment based activities
  • Describe experiences and lessons learned
  • Examine how research impacts services

As a clinical librarian supporting both clinicians and researchers, I’m very interested in ways that I can improve how we accurately and effectively assess the impact of individuals, groups, organizations, and programs. I could not have hoped for a better panel of experts:

  • US National Institutes of Health Library
  • Northwestern University Feinberg School of Medicine, Galter Library
  • Weill Cornell Medicine, Samuel J. Wood Library
  • University of Alberta Libraries (attending virtually)
  • Washington University in St. Louis, Becker Medical Library
  • University of Minnesota, Health Sciences Library
  • UMass Center for Clinical & Translational Science, UMass Medical School

Terri Wheeler and Michael Bales of the Samuel J. Wood library at Weill Cornell Medicine described the Citation Impact Tool they developed with Paul Albert, Prakash Adekkanattu, and Sarbajit Dutta. The tool “measures the percentile rank of the number of times papers have been cited, at the level of the individual paper, measured against a baseline of other articles in the same field, of the same type (research or review), and published the same year.” Wheeler and Bales explained that they organize this data by period of time, type, discipline, and percentile rankings rather than just by first and last author. This makes the data more useful and meaningful, allowing them to focus on the types of impact needed to inform leadership, support T32 grants, or fulfill an information need of the individual researcher. They did note that they have an advantage that has allowed them to create such a bespoke tool: Weill Cornell’s medical library is organized under the CIO and they have robust IT support to make automated tools. Thankfully, their tool is available on GitHub at

Karen Gutzman and Kristi Holmes of the Galter Health Sciences Library at Northeastern have used Clarivate and other applications (Web of Science, SCOPUS, etc.) and have found that with any application you can derive a lot of useful information. They have developed some refined initiatives (rather than an ad hoc response) that has led to a home grown customized citation tool and a more comprehensive bibliometric program that effectively measures the impact of research. They report this data not only to researchers but also to the University as the leadership wants to know about research impact.

Ya-Ling Lu and Chris Belter represented the NIH staff library. They explained that they started up their program from scratch. In fact, Lu was hired because library noticed they were receiving an increasing number of requests for impact evaluation. The library started a more formal program that measures not only the impact of individual papers but of entire grant portfolios.

Sally Gore, of the UMass Center for Clinical & Translational Science, originally conducted “traditional, informal” work evaluating impact to support tenure and promotion applications and for individual’s interests. Then she observed the evaluation of research impact at Washington University and established a program to better capture research impact at UMass that has become extremely popular.

Several panelists commented that this service does pose some obstacles. For one, it requires a lot of time. Kristi Holmes explained that this service is in high demand, so they are always seeking ways to balance research assessment with providing reference services. Sally Gore said that UMass is conducting an assessment of how much time each assessment “job” requires. The librarians from Weill Cornell mitigate this time burden by designating five librarians who share the responsibilities so that it doesn’t overwhelm one individual librarian.

While most of the panelists provide this as a fee-based service, Scott Library and NIH do charge and others did comment that they may move to such a system as a way of controlling the quantity of requests. The NIH representatives explained that a clear delineation of the services and fees keeps customers’ requests within the true scope of need.

The panelists agreed that assessing impact is complex and requires consideration of many questions, including time period and whether to include publications that occurred prior to position at the current organization or school. A better understanding of the customers’ needs and interests is very important. This includes understanding how the impact assessment is being used by the individual and the organization. This process may require letting the customers know what options are available, such as report format and data visualization, especially if the bibliometric service is unfamiliar to the customer. This can prevent inundating customers with unnecessary information while reducing the time burden for librarians. Another consideration is how the information will be delivered for its purpose and intended impact. Delivery methods can include reports, resumes, or even social media.

The panelists reminded the audience that measuring the number of citations can be more indicative of distribution rather than impact. Events such as public health departments adopting research recommendations and employing them in community health initiatives with measurable outcomes can be a more accurate reflection of impact. As one panelist explained, in the last several years, research linking tanning beds near college campuses to both higher risks and high rates of cancer amongst people 20-29 years old led to public health outreach campaigns and the passage of laws, including those raising the age limit, across the country. Clearly, this is a more meaningful impact than only measuring the number of citations.

More information about this session, including some of the resources mentioned by panelists, is available at



MLA’17 – What I Learned: Let me count the ways

July 11, 2017 at 11:13 am | Posted in Awards and Recognition, Professional Development, Uncategorized | Leave a comment

Jennifer Miglus, AHIP, MLS

Jennifer Miglus was awarded a scholarship to attend the MLA Annual Meeting. Congratulations on winning a NAHSL Professional Development Award!


I like sessions that spark ideas that could be practically implemented at my institution. Currently I am interested in angles for effective outreach.  Here are some presentations that resonated.  Some were lighting rounds, some were full hour sessions:

Dare to Invite Yourself to the Table”, Alissa Fial of McGoogan Library of Medicine, Omaha, NB Alissa showed how efforts at her library align with the institutional mission.  An emphasis on interprofessional teamwork has allowed them to become involved with medical education curriculum development.  Librarians have focused on mentorship, educational scholarship and have helped develop learning objects.  They have been honest about their strengths – and weaknesses.  They also participate in new faculty orientations.  To promote the mission of “healing” they have created a reflection room in the library where people can come for quiet and meditation.

Negotiating for Yourself and Your Library”, Kathel Dunn, Associate Fellowship Program Director, NLM; Kristi Holmes, Library Director, Galter Health Sciences Library, Northwestern; Cynthia Henderson, Executive Director, Howard University.  Kathel was motivational about being assertive, using silence and timing as negotiating tools.  Kristi talked about challenges she faced as a new library director and the ways she has approached new initiatives and projects.  She is an advocate of teamwork and recommended:

    • Bennett, L. M., & Gadlin, H. (2012). Collaboration and team science. Journal of Investigative Medicine, 60(5), 768-775.
    • Cynthia emphasized that it’s important to set your goals in advance, to prioritize, and be persistent. Institutional anniversaries are often opportunities, and telling an engaging and compelling story can draw people in. Be data driven; know your peer libraries.

Adventures in Scholarly Publishing”, Rebecca Welzenbach, Director of Strategic Integration and Partnerships, Michigan Publishing, University of Michigan. Rebecca described the creation of a student-run, peer-reviewed medical journal the Michigan Journal of Medicine.  This journal was developed at the request of students and was the product of a for-credit elective course offered to 4th year medical students.  Course objectives included awareness of all aspects of scholarly publishing: writing, data display, selection and editing of content and ethical issues.  Further objectives were to help students improve their professional profiles and raise awareness of bibliometrics.

Health Sciences Libraries to the Rescue: Bolstering the Library’s Role with Repository & Publishing Services”, Dave Stout, BePress; Anne Linton, Director, Himmelfarb Library, George Washington University, Washington DC; Dan Kipnis, Thomas Jefferson University, Philadelphia, PA.  Dave gave an overview of products offered by BePress.  This includes the institutional respository Digital Commons, Impact Dashboards for analytics and reporting and a new product called Expert Gallery Suite.  The latter is a new product, designed to act as a showcase for an institution’s expert researchers and faculty. BePress is content agnostic and is a great place to put grey literature.

  • In her case study, Anne Linton said that GWU uses Digital Commons for research day posters, MPH capstone projects, and for publishing a student-led medical humanities journal Fusion. She gave examples of grey literature found in IRs: legal briefs, congressional testimonies, reports from non-profits and governmental agencies. If items have DOIs and your institution subscribes to an altmetrics service, you can get data on downloads.
  • In his case study, Dan Kipnis expanded on the types of formats that may be uploaded to Digital Commons. These include medical grand rounds, oral histories, abstracts of capstone projects, teaching videos, and transcripts of archival letters, along with scans of the original documents. He also mentioned the platform’s publishing capabilities: JHN Journal is published by neuroscience residents, and The Medicine Forum is published by internal medicine residents.

Online Research Data Management Training Modules for Health Sciences Librarians”, Alisa Surkis Head Data Services and Translational Science Librarian, NYU School of Medicine.  NYU has developed an 8-module online tutorial to support the management of research data.

Visualizing Success: Development of a Data Visualization Service in an Academic Medical Library”, Fred LaPolla, Knowledge Management Librarian, NYU Health Sciences Library.  Fred was engaging as he talked about how he educated himself about data visualization and how his first class was the students’ least favorite section.  After a colleague suggested he teach Prism GraphPad (which is free for students) his session rose to being tied for second “most valuable”.  Students liked the concrete skills they learned and the fact that the program required no heavy coding.

My vote for the coolest application that I saw presented goes to:

What Type of Review Are You?” An Interactive Game to Teach Users about Review Literature Typology, Nha Huynh, Education and Instruction Librarian at Texas Medical Center Library, Houston TX.  This was a clever, fun and practical way to educate people about the different types of reviews and help them find the best one to suit their needs.

Dream – – Dare – – Do. I am dreaming of increased engagement with our medical curriculum and improved teamwork in our department.  I will dare to propose the creation of a student-run journal hosted by our BePress account.  And I do plan to get more comfortable with data visualization tools and work to support data management in our research community.

Thanks MLA. And super thanks to NAHSL for helping me to attend!

Taxation Without Information

June 30, 2017 at 8:23 am | Posted in Advocacy and Gov't Relations, Uncategorized | Leave a comment

As this article from The Scholarly Kitchen points out, we as taxpayers are being denied access to information that we’ve paid for and continue to pay for in some instances. As the author cleverly notes, this “taxation without representation” isn’t just an academic argument. It has real consequences as the people of Flint Michigan know all too well.

So as we gather to celebrate the 4th of July and remember the cry of “No taxation without representation!” let’s also stop and think about the consequences of “no taxation without information” for ourselves and our nation.

Happy 4th of July!

(Credit to M.J. Tooey from the University of Baltimore for posting a message about this article on the AAHSL listserv.)

[Submitted by Gary Atwood, Chair, NAHSL Govt. Relations Comm.]

Reaction to the Senate’s Healthcare Bill

June 27, 2017 at 1:26 pm | Posted in Advocacy and Gov't Relations, Uncategorized | Leave a comment

The Senate released the details of its healthcare bill this week and it did not take long for several key healthcare associations to issue statements critical of the proposal. See the links below for specific statements:

[Submitted by Gary Atwood, Chair, NAHSL Govt. Relations Comm.]

NIH Funding Roundup

June 6, 2017 at 1:26 pm | Posted in Advocacy and Gov't Relations, Uncategorized | Leave a comment

There have been a huge number of stories related to President Trump’s proposed budget and the impact that it would have on medical research in general and the National Institute of Health’s budget in particular. Here are some of the stories that you might want to check out:

I believe that everyone should be able to access all of the stories linked above, but please let me know (gatwood at uvm dot edu) if any are blocked for some reason.

[Submitted by Gary Atwood, Chair, NAHSL Govt. Relations Comm.]

Next Page »

Blog at
Entries and comments feeds.