Measuring Impact: A Summary of an MLA’17 Special Session on Research Assessment Services

July 21, 2017 at 3:48 pm | Posted in Awards and Recognition, Professional Development, Uncategorized | Leave a comment

Sarah Carnes was awarded a scholarship to attend the MLA Annual Meeting. Congratulations on winning a NAHSL Professional Development Award!


As other attendees can attest, MLA ‘17 consisted of a truly vast array of sessions. From the plenary speeches to the lightning rounds, from the special sessions to the vendor demonstrations, MLA 17 provided many opportunities for professional development, process improvement, and inspiration. For me, the conference certainly lived up to the theme of “Dream, Dare, Do.”

I especially valued a special session entitled “From Dreaming to Doing: Implementing Research Assessment Services.” This panel presentation and discussion occurred on Tuesday, May 30th and was sponsored by the Medical Informatics Section, Leadership and Management Section, and the Translational Science Collaboration SIG. The presenters, health science librarians from government and academia, shared a variety of lessons learned and best practices from academic and health science libraries that provide impact assessment services. Specifically, the goals of the session were:

  • Discuss types of research impact assessment services
  • Share resources and tools for assessment based activities
  • Describe experiences and lessons learned
  • Examine how research impacts services

As a clinical librarian supporting both clinicians and researchers, I’m very interested in ways that I can improve how we accurately and effectively assess the impact of individuals, groups, organizations, and programs. I could not have hoped for a better panel of experts:

  • US National Institutes of Health Library
  • Northwestern University Feinberg School of Medicine, Galter Library
  • Weill Cornell Medicine, Samuel J. Wood Library
  • University of Alberta Libraries (attending virtually)
  • Washington University in St. Louis, Becker Medical Library
  • University of Minnesota, Health Sciences Library
  • UMass Center for Clinical & Translational Science, UMass Medical School

Terri Wheeler and Michael Bales of the Samuel J. Wood library at Weill Cornell Medicine described the Citation Impact Tool they developed with Paul Albert, Prakash Adekkanattu, and Sarbajit Dutta. The tool “measures the percentile rank of the number of times papers have been cited, at the level of the individual paper, measured against a baseline of other articles in the same field, of the same type (research or review), and published the same year.” Wheeler and Bales explained that they organize this data by period of time, type, discipline, and percentile rankings rather than just by first and last author. This makes the data more useful and meaningful, allowing them to focus on the types of impact needed to inform leadership, support T32 grants, or fulfill an information need of the individual researcher. They did note that they have an advantage that has allowed them to create such a bespoke tool: Weill Cornell’s medical library is organized under the CIO and they have robust IT support to make automated tools. Thankfully, their tool is available on GitHub at

Karen Gutzman and Kristi Holmes of the Galter Health Sciences Library at Northeastern have used Clarivate and other applications (Web of Science, SCOPUS, etc.) and have found that with any application you can derive a lot of useful information. They have developed some refined initiatives (rather than an ad hoc response) that has led to a home grown customized citation tool and a more comprehensive bibliometric program that effectively measures the impact of research. They report this data not only to researchers but also to the University as the leadership wants to know about research impact.

Ya-Ling Lu and Chris Belter represented the NIH staff library. They explained that they started up their program from scratch. In fact, Lu was hired because library noticed they were receiving an increasing number of requests for impact evaluation. The library started a more formal program that measures not only the impact of individual papers but of entire grant portfolios.

Sally Gore, of the UMass Center for Clinical & Translational Science, originally conducted “traditional, informal” work evaluating impact to support tenure and promotion applications and for individual’s interests. Then she observed the evaluation of research impact at Washington University and established a program to better capture research impact at UMass that has become extremely popular.

Several panelists commented that this service does pose some obstacles. For one, it requires a lot of time. Kristi Holmes explained that this service is in high demand, so they are always seeking ways to balance research assessment with providing reference services. Sally Gore said that UMass is conducting an assessment of how much time each assessment “job” requires. The librarians from Weill Cornell mitigate this time burden by designating five librarians who share the responsibilities so that it doesn’t overwhelm one individual librarian.

While most of the panelists provide this as a fee-based service, Scott Library and NIH do charge and others did comment that they may move to such a system as a way of controlling the quantity of requests. The NIH representatives explained that a clear delineation of the services and fees keeps customers’ requests within the true scope of need.

The panelists agreed that assessing impact is complex and requires consideration of many questions, including time period and whether to include publications that occurred prior to position at the current organization or school. A better understanding of the customers’ needs and interests is very important. This includes understanding how the impact assessment is being used by the individual and the organization. This process may require letting the customers know what options are available, such as report format and data visualization, especially if the bibliometric service is unfamiliar to the customer. This can prevent inundating customers with unnecessary information while reducing the time burden for librarians. Another consideration is how the information will be delivered for its purpose and intended impact. Delivery methods can include reports, resumes, or even social media.

The panelists reminded the audience that measuring the number of citations can be more indicative of distribution rather than impact. Events such as public health departments adopting research recommendations and employing them in community health initiatives with measurable outcomes can be a more accurate reflection of impact. As one panelist explained, in the last several years, research linking tanning beds near college campuses to both higher risks and high rates of cancer amongst people 20-29 years old led to public health outreach campaigns and the passage of laws, including those raising the age limit, across the country. Clearly, this is a more meaningful impact than only measuring the number of citations.

More information about this session, including some of the resources mentioned by panelists, is available at




Sally Goes to Seattle

July 18, 2017 at 11:22 am | Posted in Awards and Recognition, Professional Development | Leave a comment

Sally Gore was awarded a scholarship to attend the MLA Annual Meeting. Congratulations on winning a NAHSL Professional Development Award!


I want to thank the members of NAHSL’s Professional Development Committee, as well as all of the membership of NAHSL, for the award that I received to cover the cost of registration to attend the annual meeting of the Medical Library Association that took place in Seattle, WA last month. As always, it was a wonderful opportunity to reconnect with old friends and colleagues, meet lots of new people, network with those who do similar work, learn some nuggets of gold to put into practice back home, and of course, to see and to share the work that we all do as medical and health sciences librarians – or in my case, a Research Evaluation Analyst, aka a “Librarian by Any Other Name.”

I experienced a few highlights of the meeting including participating on a panel discussion with colleagues from several other institutions to talk about our work measuring and tracking the impact of research. I also really enjoyed Julie Sollenberger’s Janet Doe lecture. I have the honor of knowing Julie a little bit, so her choice of topic – kindness and how practicing it shapes not only our work, but our very beings – was spot on. She is a wonderfully kind person – one who lives what she preaches. It was a special plenary.

For this post, though, I want to reflect on another plenary session, i.e. the McGovern Lecture, given this year by Julie Angus, a scientist, bestselling author, and winner of the National Geographic Adventurer of the Year Award for her accomplishment as the first woman to row across the Atlantic Ocean, mainland to mainland. When I first saw that she was one of the keynote speakers this year and I saw the title of her talk, “Rowing Across the Atlantic: Strategies to Reach Your Goals,” I admit that I said to myself, “What in the world does this have to do with medical librarians and/or libraries?” And in truth, much of her talk came across as canned, meaning it struck me as likely the same talk that she gives to any audience. Yes, she threw in a few library references, but it was pretty general in nature – your typical motivational talk by a motivating person. Don’t read this as negative criticism. I enjoyed the talk and I’ll likely borrow a copy of her book from my local library and read it. It’s a great story – and she told it well. And I did take away two important things from it – two points that made it into my notes:

First, was the importance Angus placed on baby steps. In her talk she said, “When people ask, ‘What’s the most important thing that you’ve brought out of this journey?’, undoubtedly, it’s the importance of baby steps. Eventually all of those baby steps added up and as a result, we achieved our dream.” This point reminded me of the words of another adventurer, the climber, Joe Simpson. (If you like adventure and you’ve never seen the dramatized documentary, “Touching the Void,” seek it out!) Stuck in a life-or-death situation during a climb, he says, “You have to make decisions. You have to keep making decisions, even if they’re wrong decisions. If you don’t make decisions, you’re stuffed.” (He’s British, in case you wonder the “stuffed” reference.) Baby steps and continuous decision-making are crucial not only in big adventures, but in day-to-day life. It’s how we get from Point A to Point B, how we continue to be relevant in a changing work environment, how we continue to find fulfilment in our work. Complacency, apathy, dullness in the daily work is a career killer – not only for an individual, but for an entire organization. THAT is a message I found quite relevant – a great take-away from the talk.

The second thing I noted was the discussion that I followed during the Lecture; the back-channel, if you will. Following the #MLANET17 Twitter stream, I discovered that while I was thinking Angus’ talk was a bit predictable, LOTS of my colleagues did not. She would say something and someone would immediately apply the thought or the message to something in his/her career or workplace. It was wonderful to see all of the connections people were making between the Lecture and their lives. I love following the Twitter stream throughout the conference, generally to see what others are learning in concurrent sessions that I’m not attending, but it was also a real treat to see a motivational speaker truly motivating an awful lot of the audience with her talk. It’s hard to argue that this isn’t just what the planning committee wanted when they signed her up and I credit them for the good pick.

I’m off to the Special Library Association’s annual meeting in Phoenix this coming week. I had a paper selected for presentation. It’s going to be interesting as I don’t know many people in SLA. It will be like going to my first MLA meeting many years ago now. I’m a little nervous, but the one thing that that eases my nerves is that I do know that it’s pretty hard to find a librarian conference that doesn’t leave me wanting for good stuff. I look forward to that – and again thank NAHSL for helping me get to Seattle. It’s a terrific benefit of belonging to and serving the organization.



Barbara Ingrassia Presents Poster on Human Trafficking at NEGEA

July 13, 2017 at 11:15 am | Posted in Awards and Recognition, Professional Development | Leave a comment

Barbara Ingrassia was awarded a scholarship to attend the NEGEA Annual Meeting. Congratulations on winning a NAHSL Professional Development Award!


Thank you to the NAHSL Professional Development Committee for enabling me to attend the annual conference of the AAMC’s Northeast Group on Educational Affairs (NEGEA) on May 4, 2017 at the University of Rochester Medical Center. My intent was to display my accepted/approved poster HUMAN TRAFFICKING: HEALTH CARE PROVIDER PERSPECTIVES and receive feedback.

IngrassiaThe poster describes the one-week elective I developed for the Flexible Clinical Experiences program (FCE) for 3rd year medical students at UMMS.

The University of Massachusetts Medical School (UMMS) third year curriculum now requires students to participate in four weeklong electives of their choice. These Flexible Clinical Experiences allow exposure to evolving public health concerns and the development of skills pertinent to the practice of medicine outside of the core clinical experiences.

The FCE platform provided me the opportunity to offer an introduction to human trafficking– a public health epidemic that often goes unrecognized in busy health care settings.   A component of the course is research; I am able to promote EBM research skills.  Beyond that, I connect them with community resources; they meet with appropriate faculty and clinicians, law enforcement, the district attorney’s office, advocates and other service providers for a variety of perspectives.

In the pre-course questionnaire, students have indicated that they had very little or no knowledge about human trafficking and that it had not been mentioned in their formal course work. During the final debriefing session, students reported that these experiences have been “eye-opening.” I serve as the facilitator; I have no expertise in the topic, but I bring a passionate concern. I pull pieces together to enable an “experience” for the students; I learn so much from them and eagerly await the debriefing session.

The poster sessions provide an opportunity to learn about developments in a profession and share ideas. Several attendees stopped to review my poster and offer helpful suggestions. I had a good discussion with a physician from Brown University/Rhode Island Hospital whose poster described a curriculum on acute sexual assault on adolescents and adults for Residents.

I even met some others from UMass Med School!

Part 2

I was able to attend the opening plenary The learning Environment as the Context for and Target of Change presented by Larry Gruppen, PhD  (Professor of Learning Health Sciences at the University of Michigan Medical School). He noted that interest in the medical learning environment (LE) moves in and out of the lime light, but is once again a “hot topic” since the inclusion of a question about “mistreatment” on the AAMC’s Graduation Questionnaire. [1]

The challenge is the lack of one generally accepted definition of LE (especially in medical education) and no underlying theory. Gruppen asserted that defining LE today is challenging because there are so many simultaneous and varying factors.[2] Learning takes place at many levels of scale: individual, group, organization, region, state, entire nation and beyond. It takes place continuously and everywhere, by various and multiple methods. Ever changing and evolving.

He called for better research to determine the impact of various factors: specific topic, content, context, official vs. hidden curriculum, intended vs. unintended learning, reality vs. perception vs. experience, physical facilities…

In this very interactive session, I suggested several LE issues/challenges faced by medical libraries over the years:

  • Providing group study space in traditionally-designed libraries—as well as quiet space– to accommodate various learning styles, evolving instructional techniques and electronic resources.
  • Housing computers and workstations, etc.
  • Permitting food in libraries (An issue faced by medical libraries a number of years ago –and more recently by other types of libraries)

… all of this while losing square footage.

It was a thought-provoking session.

My time at NEGEA was too brief, but it was a wonderful experience. I had the chance to wear “multiple hats.”

Again, thank you for making this opportunity possible.

(And YES—I did have the chance to catch up with our former NER colleague Donna Berryman. She is now Director of the Miner Libraries. Can you believe she has been there for TEN YEARS? How time flies!)

Barbara c. Ingrassia, MLS, AHIP, CCM

Certified Copyright Manager, Speaker and Trainer

[1] The 2017 survey is available at

(SEE especially pp. 39-49.) Accessed May 25, 2017.

For aggregate data 2012 – 2016:

AAMC. All Schools Summary Report of the 2016 Medical School Graduation Questionnaire.

(SEE espcecially pp. 35-40.)

Available at  Accessed May 25, 2017.

[2] Department of Learning Health Sciences, University of Michigan Medical School.

Available at    Accessed May 25, 2017.

MLA’17 – What I Learned: Let me count the ways

July 11, 2017 at 11:13 am | Posted in Awards and Recognition, Professional Development, Uncategorized | Leave a comment

Jennifer Miglus, AHIP, MLS

Jennifer Miglus was awarded a scholarship to attend the MLA Annual Meeting. Congratulations on winning a NAHSL Professional Development Award!


I like sessions that spark ideas that could be practically implemented at my institution. Currently I am interested in angles for effective outreach.  Here are some presentations that resonated.  Some were lighting rounds, some were full hour sessions:

Dare to Invite Yourself to the Table”, Alissa Fial of McGoogan Library of Medicine, Omaha, NB Alissa showed how efforts at her library align with the institutional mission.  An emphasis on interprofessional teamwork has allowed them to become involved with medical education curriculum development.  Librarians have focused on mentorship, educational scholarship and have helped develop learning objects.  They have been honest about their strengths – and weaknesses.  They also participate in new faculty orientations.  To promote the mission of “healing” they have created a reflection room in the library where people can come for quiet and meditation.

Negotiating for Yourself and Your Library”, Kathel Dunn, Associate Fellowship Program Director, NLM; Kristi Holmes, Library Director, Galter Health Sciences Library, Northwestern; Cynthia Henderson, Executive Director, Howard University.  Kathel was motivational about being assertive, using silence and timing as negotiating tools.  Kristi talked about challenges she faced as a new library director and the ways she has approached new initiatives and projects.  She is an advocate of teamwork and recommended:

    • Bennett, L. M., & Gadlin, H. (2012). Collaboration and team science. Journal of Investigative Medicine, 60(5), 768-775.
    • Cynthia emphasized that it’s important to set your goals in advance, to prioritize, and be persistent. Institutional anniversaries are often opportunities, and telling an engaging and compelling story can draw people in. Be data driven; know your peer libraries.

Adventures in Scholarly Publishing”, Rebecca Welzenbach, Director of Strategic Integration and Partnerships, Michigan Publishing, University of Michigan. Rebecca described the creation of a student-run, peer-reviewed medical journal the Michigan Journal of Medicine.  This journal was developed at the request of students and was the product of a for-credit elective course offered to 4th year medical students.  Course objectives included awareness of all aspects of scholarly publishing: writing, data display, selection and editing of content and ethical issues.  Further objectives were to help students improve their professional profiles and raise awareness of bibliometrics.

Health Sciences Libraries to the Rescue: Bolstering the Library’s Role with Repository & Publishing Services”, Dave Stout, BePress; Anne Linton, Director, Himmelfarb Library, George Washington University, Washington DC; Dan Kipnis, Thomas Jefferson University, Philadelphia, PA.  Dave gave an overview of products offered by BePress.  This includes the institutional respository Digital Commons, Impact Dashboards for analytics and reporting and a new product called Expert Gallery Suite.  The latter is a new product, designed to act as a showcase for an institution’s expert researchers and faculty. BePress is content agnostic and is a great place to put grey literature.

  • In her case study, Anne Linton said that GWU uses Digital Commons for research day posters, MPH capstone projects, and for publishing a student-led medical humanities journal Fusion. She gave examples of grey literature found in IRs: legal briefs, congressional testimonies, reports from non-profits and governmental agencies. If items have DOIs and your institution subscribes to an altmetrics service, you can get data on downloads.
  • In his case study, Dan Kipnis expanded on the types of formats that may be uploaded to Digital Commons. These include medical grand rounds, oral histories, abstracts of capstone projects, teaching videos, and transcripts of archival letters, along with scans of the original documents. He also mentioned the platform’s publishing capabilities: JHN Journal is published by neuroscience residents, and The Medicine Forum is published by internal medicine residents.

Online Research Data Management Training Modules for Health Sciences Librarians”, Alisa Surkis Head Data Services and Translational Science Librarian, NYU School of Medicine.  NYU has developed an 8-module online tutorial to support the management of research data.

Visualizing Success: Development of a Data Visualization Service in an Academic Medical Library”, Fred LaPolla, Knowledge Management Librarian, NYU Health Sciences Library.  Fred was engaging as he talked about how he educated himself about data visualization and how his first class was the students’ least favorite section.  After a colleague suggested he teach Prism GraphPad (which is free for students) his session rose to being tied for second “most valuable”.  Students liked the concrete skills they learned and the fact that the program required no heavy coding.

My vote for the coolest application that I saw presented goes to:

What Type of Review Are You?” An Interactive Game to Teach Users about Review Literature Typology, Nha Huynh, Education and Instruction Librarian at Texas Medical Center Library, Houston TX.  This was a clever, fun and practical way to educate people about the different types of reviews and help them find the best one to suit their needs.

Dream – – Dare – – Do. I am dreaming of increased engagement with our medical curriculum and improved teamwork in our department.  I will dare to propose the creation of a student-run journal hosted by our BePress account.  And I do plan to get more comfortable with data visualization tools and work to support data management in our research community.

Thanks MLA. And super thanks to NAHSL for helping me to attend!

Many Moving Parts…

December 22, 2016 at 10:08 am | Posted in Awards and Recognition, Professional Development | Leave a comment

Our last NAHSL 2016 meeting blog post is by Nathan Norris, Senior Information Specialist at Beth Israel Deaconess Medical Center.  Nathan is another winner of NAHSL Professional Development award.  Thank you, Nathan and congratulations on winning the PDC award!


I want to thank the NAHSL Professional Development Award Committee for the award to attend the NAHSL annual meeting in New Haven, CT!

It enabled me to participate in the poster session and enjoy excellent speakers, not to mention spending time with the fine company to be found among our membership!  Also, many thanks belong to our CPC, as the circus-themed program and entertainment were just great!

I enjoyed all 3 of our keynote speakers!:

William Powers – Having listened to his talk and spoken to others NAHSL members, I will read his book!  In general, I am a firm believer that we are well-served by spending significant, scheduled “unplugged” time from the Internet and what can be the “Siren Song” of social media.

Susan Keane Baker – I not only enjoyed her talk, but thought having a copy of her book is a really nice gesture! It is chock full of great, practical tips to promote kindness – This is something that is so important in our hurried world today, and Susan reminds us that it can be done in minimal time!  After listening to Susan’s points about making a difference, I know that I am lucky to be working with colleagues who already practice what she is preaching (As for myself, I will strive to do better….New Years’ Resolution perhaps?!)

Jack HughesWhile Professor Hughes’ topic was a bit drier than the other two speakers, it is likely to be the most salient over the next few years – at least to all of us working in healthcare. I would like to spend a little bit of time discussing this! hughes

It is likely that the Accountable Care Act (ACA) will be “under the microscope” soon with our next administration.  Professor Hughes did an excellent job of outlining the current healthcare climate driving high costs and mediocre results (comparatively speaking) in the United States.  From Professor Hughes’ presentation, here is a quick synopsis. Our Healthcare environment is characterized by:

  • More intensive treatment near death
  • Fear of malpractice
  • Administrative complexity
  • Higher income for health professionals
  • High prices

Underlying causes include:

  • Lack of concentrated purchasing power, price controls, volume controls, technology availability and control over hospital budgets – Many of these are exercised by other countries.
  • Overutilization
  • Waste, duplication and fraud
  • Costs associated with technology, treatment and drugs
  • High Expectations / Attitudes surrounding perceived “need”
  • Overall Economics – insurance/ tax exemptions for healthcare / traditional fee for service environment

The discussion we need to have centers around the cost savings not yet realized through the ACA to date.  It won’t be easy, but it will be fascinating. There are many moving parts  – due to social and demographic factors, insurance company participation, pharmaceutical costs, the roles of Medicare & Medicaid, and of course – politics!

Stay tuned for further discussion! – Maybe we can reconvene and see how we have done over the past year at NAHSL 2017?!

I hope for everyone to have a happy & healthy holiday season and beyond!

Many thanks,

Nathan Norris, Senior Information Specialist

IS / Knowledge Services, Beth Israel Deaconess Medical Center


Next Page »

Create a free website or blog at
Entries and comments feeds.