Beyond the Metrics: Applying AI to Sentiment Analysis

Rob: For years, medical communication professionals have relied on traditional metrics like downloads, citations, and maybe some attention scores to measure success. But today, with advancements in artificial intelligence and data analysis, we're uncovering new ways to assess not just what people engage with, but how they feel about it.

This is In Plain Cite, a podcast exploring the biggest questions and trends facing medical publication and communication professionals. I'm your host Rob Matheis, president and CEO of ISMPP. 

Today's episode is generously sponsored by Digital Science. 

For this episode, we have a guest host taking over my duties. I'm excited to introduce Dr. Carrie Brubaker. Dr. Brubaker is an independent consultant specializing in medical affairs and communication strategy, as well as a member of the International Society for Medical Publication Professionals. She's joined by two incredible guests, Julia Mutygullina, product solutions manager at Digital Science, and Dr. Carlos Areia, senior data scientist at Altmetric, also part of Digital Science. Together, they'll explore how sentiment analysis is transforming the way medical communication professionals assess the performance and the impact of their work, and how artificial intelligence is elevating these insights.

Now over to you, Dr. Brubaker.

—--—

Carrie: Greetings, everyone, and welcome to this podcast discussion. So I'd like to lay the groundwork for our conversation by asking you a key definition. Could you please share with our listeners, what is sentiment? How would you define it? 

Carlos: So what is sentiment analysis? Sentiment analysis, I think it's been one of the key topics over, well, at least a decade, but more so since the AI coming to play into the new world. So sentiment analysis usually brings a bit of the qualitative world by quantifying discussions and quantifying whether they are positive, they are neutral, they are negative. And now with the AI, one can go a step further saying whether people are criticizing or basically, uh, both qualifying and quantifying opinions and discussions on social media and other platforms. So, um, yes. So for me, I think that's the definition of sentiment. 

Carrie: That's great. Thanks so much. And Julia, now that we've heard from Carlos about the definitions, could you elaborate for us why medical communications professionals should be thinking about sentiment in the work that they do and in the publication strategies that they're designing and executing? 

Julia: So, first of all, I want to say that sentiment analysis empowers medical affairs teams to go beyond basic metrics. Uh, and by basic metrics, we understand the article views, downloads, just attention volume or general reach. Uh, it really allows them to evaluate not just the quantity, but the quality and emotional tone of stakeholder interaction. And, um, it ensures that their strategies are not just effective in a sense of general reach, but also aligned with the values, uh, and expectations of their audience. And ultimately, these can support the experts at medical affairs in fostering real trust, transparency, meaningful scientific exchange with their stakeholders.

We all know that experts at medical affairs, they interact with diverse stakeholders, including healthcare professionals, patient advocacy groups, regulators… And sentiment analysis helps them in understanding how these stakeholders perceive their research. And, um, to identify whether the feedback that 

they get and their research is getting is positive, uh, neutral, or even sometimes negative.

And apart from that as a bigger picture, sentiment analysis also helps in monitoring the overall brand scientific credibility and trustworthiness, and highlight the shifts in perception of individual products or overall the company. So basically by adapting, based on sentiment analysis insights, medical affairs can ensure that their materials really resonate with their audience. They can effectively communicate with, um, communicate complex scientific information, refine their engagement strategies, and in the end create a stronger relationship with our stakeholders. Sentiment analysis can also obviously help and support in evaluation of how patients and advocacy groups, uh, feel about clinical trial results or specific research results or educational initiatives.

Carrie: Thank you very much, Julia, for this comprehensive assessment of sharing why medical communications and medical affairs professionals should be thinking about sentiment. What can you share with us about your industry partners experience of deploying these analytics, of performing sentiment analysis, as part of publication performance assessment?

Julia: That's a great question, Carrie. And well, first of all, at Digital Science, we're proud to partner on this topic with leading pharmaceutical companies that spearhead the research aimed to improve patient's lives and patient's outcomes, and we are excited to contribute to their effort. 

Um, our partners, they quite often feel the strong need for a better way in measuring the real world impact of their research. Although they publish really high quality results, but they struggle with gauging how their research is being received and utilized by key stakeholders. Um, healthcare professionals, policy makers, patient advocacy groups. So traditional metrics just are not enough to capture real time engagement or broader 

social impact and they make it difficult to find communication strategies or sometimes even justify investments.

Using our data, uh, our partners gained access to a comprehensive suite of tools that allowed them to understand and measure engagements of healthcare professionals. Organizations with their research, understand how their findings are being perceived, track their research and how it was resonating most strongly and where.

By now we have multiple teams who integrated our data into their day to day activities, strategic decisions. They monitor mentions across various digital channels and also analyze citation patterns in academic literature. So this is happening in parallel. Uh, the one process doesn't replace another. We never aim to do that, but the platform just allows them to get additional insights and also visualize wider and broader engagement metrics. And actually like using the sentiment to identify sometimes quite influential voices that are amplifying their work or again, it is not always all positive. Sometimes they can easily and timely track negative attention. So we helped our partners to achieve actually remarkable results in identifying which of their most recent studies were frequently referenced or discussed by top tier researchers, specific news outlets on a global and regional level. Uh, clinical guidelines mentions, uh, and this all is given, uh, insights how their research was being shared in different regions, shaping their strategy in communication, supporting local educational initiatives and overall global strategy.

And real time monitoring is helping actually to generate insights within days sometimes, since our data is updated daily, rather than wait for weeks or months to maybe get some reports from somewhere else. 

Carrie: And the way that I'm interpreting that or that I would tend to summarize what you've just described is that there is 

now availability of an even wider range of not just performance or operational metrics, but also impact metrics, which are indeed becoming increasingly important to demonstrating the value of a given publication strategy and the publications themselves.

Julia: That's correct, yes. 

Carrie: I'd like to return to Carlos for a more in depth question, uh, for example, assessing how does sentiment analysis help these audiences to uncover data streams or insights that might not be available through other analysis and analytic types. 

Carlos: So the way that we're using sentiment analysis is a bit different. So normal sentiment analysis usually evaluates the content of, for example, if Julia posts on X or Bluesky saying, I disagree with what you're saying, Carlos, and here's the reason why and text the publication that kind of helps her prove what she is saying. So in a normal sentiment analysis, that would be perceived as negative because she is criticizing me in a reply, However, in our view and for the sentiment that we have built, because it's using the research output, the publication, to prove that I am wrong and to prove to help a claim what she's saying to me, replying to me in Bluesky or a Facebook or X, it's a positive thing for us. So we've built this bespoke sentiment analysis that's tailored to the use of research. And when I say research, this can include anything from clinical trials to publications. 

By having this data available at the post level, for example, at Trulia's post saying that I am wrong and the reason why, we can segment this in multiple ways. We can now say, for example, that a particular product has more negative sentiment than another, for example, or a partner or collaborator's research has been really well received by a particular demographic. A good example that I usually like to use is, and we’re going really into the 

rabbit hole, um, we can, for example, say that a publication on rare diseases has been shared by patients on, of that rare disease in a positive, neutral or negative manner. And these are just some examples that I'm giving now, but this can basically open another door to use our creativity to slice and dice this data as, as we see fit and to provide, like, insights that might be hidden or signals that might be hidden some somewhere else. It does increase the, this like adds numbers to the equation saying whether a post has been positive or negative, but I do believe that it also adds a bit of qualitative insights into it. It's not just about, okay, this publication was shared like 500 times. Now we know, this was shared 500, 500 times by 30 clinicians that are talking mostly positively about, about this research. So I think that kind of opens the door for a lot of things. 

Carrie: So I think that to wrap up our discussion today, uh, I will return to you, Carlos, with, with a final, with the final question, what's coming next? What is coming next for medical publication professionals to get more value out of sentiment analysis out of processes that are already available to them?

Carlos: I think we've kind of moved away from, like, static impact, impact metrics. And medical affairs professionals and most in general are becoming more open to dynamic metrics and things that may mean some impact today might not be the same as in a one or one, two or three years time. Um, so what's coming next?

Well, we have built the first sentiment analysis model a couple of years ago, so we are improving that with artificial intelligence. We're making it faster and better. But besides this all, this more technical aspect, we have been focusing a lot on social media. So at the moment we are tracking, we are doing sentiment analysis on Facebook, X, more recently Bluesky that we just added to Altmetric. But at some point we want to 

expand this to give a, again, a bigger picture of like different ways of impact, for example, sentiment analysis on news or blogs, and maybe at the latest stage, even policy and clinical guidelines and so on. So the opportunities are endless and we're doing quite a bit of work on it to give a broader picture of sentiment on different types of impact and attention.

So yeah, so sentiment is also coming available to the whole of Altmetric Explorer. So that is going to be coming next year so most people with access to Altmetric will be able to see it. Uh, this is after we've done some bespoke work with, with our partners in medical affairs. We thought, okay, why don't we standardize sentiment and then make it available to everyone?

Uh, and yeah, and I think we're playing with some other stuff like conference tracking and sentiment of conference data and so on and so forth. But I think we'll have more to talk about next year on that. 

Carrie: That's great. Thank you for sharing these, essentially the latest news on, on the capabilities for analysis.

This does bring us to the end of our discussion today. So I would like to extend thanks to you and to Julia for these valuable exchanges. And together we extend our thanks to you listeners and invite you to visit the website altmetric.com to learn more about the topics we've reviewed together today.

Rob: Well, that's us for today. Thank you all for listening. Please take a minute to subscribe to In Plain Cite on your favorite podcast app. Share with your colleagues and rate our show highly if you like what you heard today.

In Plain Cite is a production of ISMPP, the International Society for Medical Publication Professionals. This episode is generously sponsored by Digital Science.

Our production partner is CitizenRacecar. Our producer and editor is Hajar Eldass. Post production by Alex Brouwer. Publication and promotion by Candice Chantalou.

To join ISMPP today, go to ismpp.org. Becoming a member means you can participate in value packed webinars and receive instant access to exclusive tools and resources. If you're interested, just go to ismpp.org, that's I S M P P dot org, to learn more.