CMPP: Behind the Exam Curtain

Rob: For publication and medical communication professionals, earning the CMPP credential is an important career milestone. But for many candidates, the certification exam itself can feel like something of a mystery. How is it developed? How is it scored? And what does it really take to prepare successfully?

This is In Plain Cite, a podcast exploring the biggest questions and trends facing medical publication and communication professionals. I'm your host, Rob Matheis, President and CEO of ISMPP.

Today I'm joined by Claudia Piano and Danita Sutton, who oversee and support the CMPP certification program. Together, we'll pull back the curtain on how the exam is developed, why it's designed the way it is, how scoring works, and the resources and strategies that can help candidates prepare with confidence.

Let's get into it.

[End of Intro]

Rob: Why don't we start by just getting a little background on the certification program itself, why it was put in place, um, and what it's all about? And, and maybe we'll have Claudia start us with that.

Claudia: So if we go back to the late nineteen nineties, medical publishing was under intense scrutiny from external stakeholders, and they were citing specific practices which included cherry-picking data, uh, which refers to focusing on positive clinical results in publications while minimizing or even not even reporting less favorable results, lack of transparency, uh, standards around reporting financial disclosures and conflicts of interest were not widespread or consistently practiced.

Uh, publication teams at agencies often took direction from marketing departments in pharma who played key roles in reviewing and influencing data, uh, draft manuscripts. And then we also had guest and which was prevalent. Medical writers were rarely acknowledged in publications, and guest authorship was widespread.

So in response to this scrutiny, and to ensure the highest levels of scientific and professional integrity in the field of medical publications, ISMPP established the CMPP credential, and the first credentials were awarded in two thousand and nine. Today, the CMPP certification is the globally recognized credential for medical publication professionals.

Rob: I'm glad you brought that up, Claudia, because we have so many new entrants coming to the field, and many of those folks that are coming into the field these days may not remember the early days of guest authorship and ghostwriting and things of that nature. So really reminding our listeners that there was a time when things weren't as good as, as they are today is really, really im-important.

And the fact that ISMPP has a credential to distinguish those people who understand that background and can speak to it is really, really important. So if we think about all the things you just said, Claudia, and we think about what the credential stands for, what would somebody who gets CMPP certified expect, uh, to have in terms of their, their credential?

Like, what does it mean to an employer? What does it mean to the person who gets that credential?

Claudia: Yeah, the CMPP credential really certifies that, the person has expertise as a medical publication professional, they have proficiency in good publication practices, they are committed to ethical and transparent data dissemination standards, and they really demonstrate leadership at upholding and fostering integrity and excellence in medical publications and communications. This credential is really something that a lot of companies are looking for when they hire people, to work in the profession.

Danita: Yes. In fact, we are seeing in a lot of job postings in industry as well as agencies that the CMPP credential is, quote, "preferred." So we are seeing uptake by the industry.

Rob: So before we go any further, why don't we talk a little bit about what it means to have a certification versus a certificate program? Are they the same thing? I, I think it's really good to remind our listeners difference between those two.

Danita: Um, well, cert-certification program is a more intense type of program. In order to even become certified, you need to have certain educational requirements, there's certain experiential requirements, as well as you have to pass an exam. And a certification also is only, um, valid for so many years.

It varies by the certification. For example, the CNPP is valid for five years, then you need to recertify to maintain your credential. A certificate program is a program in which, uh, an individual learns something. They may go through a-- watch a webinar, go through a series of, knowledge-based materials and then take an exam, a, a b-brief test, and are awarded a certificate, and that certificate never expires.

So there's really not as much, in terms of eligibility requirements, and it's just considered more you teach someone something, they prove they've learned it, they get, a certificate. Whereas a certification is very broad-based and has many components to it and is a much more intense program for an association to even develop and maintain.

Rob: So would you say that someone who has a certification has a, a better seal of approval, if you will, than someone who just holds a certificate?

Danita: Yes.

Rob: Okay. Okay. Well, while we're talking about that, let's talk about then what makes our certification program so special. Um, we have a very, rigorously developed exam.

I know a lot of associations can claim to have rigorous exams, but I think it's time now we kinda peel back the layers here and look at how the exam is actually developed. Um, I'm sure we didn't just sit down and write items off, off the bat like that. Let's tell our listeners how we actually put the exam together.

Danita: Yeah, sure. This is fun for me to talk about this. So many of our listeners will know that the CMPP exam is a one hundred and fifty question multiple choice three-hour exam that tests knowledge across the three competency domains for the credential, and those are develop a publication plan, implement a publication plan, and foster ethical and compliant behavior in publications because it's a certification exam, all the questions are developed to address these key competency areas, and they're very application based.

So the exam is not based on memorization, it's based on, applying the knowledge you need in these three domains. So in terms of development, the exam is actually developed by a group of subject matter experts who are CMPPs with, substantial experience in medical publications and communications, and they convene once a year to develop new i- items, or what we call questions in the exam development world.

There's a bank of questions, but every year we develop thirty to forty new questions. And at a very high level, each year the three things we consider are, one, are there any new updated guidelines in the area or any new tools that are being used in workflows that a CMPP should be aware of with the minimum eligibility?

Is this information expected knowledge of a person with the minimum eligibility, which is two years experience if you hold a college degree? And finally, will this question be easily understood and answered by a candidate for whom English is not their native language?

So then these subject matter experts, after initial meeting, they go away for about a month, and they write three to five new questions. But that's just the easy part, almost the beginning of the process, and Claudia can speak a little bit about this as well. After they develop those questions, they reconvene for several multi-hour sessions and review these questions.

And then after the question's accepted, what I think many candidates don't realize, it then goes on the exam as what's considered a pilot question. And we look at how those questions are statistically performing on the exam. Are high performers on the exam getting this question incorrect, and why? Is there another option they're choosing? And is the other option they're choosing that we're saying is not correct, could it be correct? Everything's looked at from every angle, and once they believe statistically these pilot questions are performing, only then do they go on to the exam and are actually scored for candidates.

Rob: I guess a-as we talk through the process, it sounds to me there could be a possibility I'm an exam taker and I'm taking an item and I realize that I don't think it's a good item or I think it's, it's lousy written, it's actually possible that it may be a test item and not one that's actually contributing to my score. Is that something that you say is accurate?

Danita: Yes, that's true. It is a 150 question exam, but usually between five to 10 of those questions, and you won't know which ones, they're spread throughout randomly, are pilot questions, and we're looking at how they perform. They are not scored, so actually your exam is only scored on 140 questions.

So correct. That's correct.

Claudia: Yeah.

We are very conscious of the fact we've all taken the test, all of us. We've all taken the exam, so we know that it can be really difficult. We don't wanna make a question that's impossible to get correct, but we also don't wanna make a question that's so simple that anybody could do it.

I just wanna let listeners know, if, if you're thinking that, "Where do they come up with these questions?" You know, or, "Why are these questions so hard?" We're, we're just trying to make sure that the person has to know the material in order to get it right.

We don't wanna, we don't want it to come across as tricky or too difficult.

Danita: And that's another good point, this is all facilitated by our test vendor, a company called PTI, and we intentionally never use questions where the response is all of the above, none of the above, A and B, or A, B and C, or just A, because we don't want it to be a logic question. We want to make sure that the person taking the exam knows the material and that they don't have to go through a process of logic to determine the correct answer because we don't think that's very fair and it's not good testing practice.

Claudia: Absolutely.

Rob: I'm really glad to hear these comments about the exam development and, and how rigorous it is because I think it's important to know that the exam is not something that our committee members just develop or that our members of the board just develop. We actually hire an outside professional vendor that writes exams for major, major associations and, major test takers and so on.

It's really something where there's a huge investment and not something that we take, take lightly. And I think it helps our listeners to know how it unfolds.

Danita:

Rob: So now that we got through exam and item development, let's talk a little bit about how the exam is scored. I think that's also a pretty big mystery. Um, tell our listeners, is there a set score that, that you have to reach? You know, how, how do we get to that pass, fail?

Danita: So the exam is scored by psychometricians at our testing partner via a complex process. Yes, there is a raw score. That is the number of questions you answer correct out of the number of questions to be scored. However, different raw scores may be different passing scores based on the exam form, its difficulty, and other factors on the exam, which is why that raw score is translated psychometrically to a scaled score, where six fifty is the passing score. Now, this six fifty scaled score takes into account the difficulty of the exam because different exam forms vary in difficulty as well as other factors. And this is done so that comparable results are reported even when exams form differs year to year, as well as raw passing scores differ over time.

So this practice represents the best practice in the industry. Now, post-exam analyses are conducted at both the exam level and the individual item, level. So if any anomalies are observed within an individual item on the exam, we resolve them before final scores are determined and released.

If you achieve a six fifty scaled score or above, you've passed the exam, and the passing point is a threshold. It's a pass-fail exam. Which is why if you pass, we don't, distribute your actual scaled score because we don't want people to think that a higher scaled score means they know more.

Uh, it's a threshold. You either pass or fail. Now, those who unfortunately do not pass the exam, they do receive their scaled score. Maybe they had a five eighty or a, a five sixty, so they have a good sense of how far away from the passing point they were. They also receive their scaled score by domain, so that will help in their study efforts if they choose to retake the exam.

Rob: Yeah, I'll tell you, it's been a while since I've talked through and listened to the psychometrics and the process behind this, and I really believe it's, it's such a rigorous examination in terms of the way it's built, and someone with this certification, you really, you can stand behind or they can stand behind the fact that they know what they're doing, they have experience, and, uh, not only can respond to these questions that are most likely very ethical, and competent publication professionals.

So really, really good to hear.

Now that we know how the exam is built and how it's scored, let's talk about how we can prepare for it. So what materials, if I'm thinking about sitting for the exam, should I prioritize, in terms of preparing for the actual exam itself?

Claudia: We actually make it pretty simple for candidates. Right on the website, on the ISMPP website, we have a section on, the certification and the materials that you're going to need to study and prepare. I'm a big fan of the candidate handbook.

That handbook basically tells you what the eligibility criteria are, all about the testing options, the venues, the application process, fees, scheduling process, preparation, et cetera.

So the candidate handbook is the perfect place to start, and I, I tell everyone who's interested, uh, even if they're not ready to take the test, you know, download that PDF and start reading it. Um, so that's sort of number one. Then on the website, we have a study resources section, which has a critical reference list and a supplemental reference list.

And the critical reference list are the ones that we actually use as references for the key questions in the exam. So you really need to know these critical references in order to pass the exam, in, in my opinion. Supplemental references are good if you feel you need a little extra preparation for a particular topic.

Look through those. We also have a list of relevant websites and a list of commonly used acronyms. And then also, we also recommend take a look at recent ISMPP new webinars and presentations. Like, for example, when GPP gets updated, there's always a webinar about it, or whenever there's a new, a change or a new trend in the industry, there's a webinar about it.

So we recommend that people look at those. We've also got a wonderful mentor program. These are volunteers, CMPPs who actually support exam candidates. And, we have 100 of them in the US, UK, Europe, and APAC and we try to match the mentee with a mentor who is in your time zone, region, similar industry, et cetera, and if possible, same native language.

And they will help you identify what your sort of gaps are in knowledge. Everyone that I've talked to who has opted to use a mentor has said that it's a fantastic resource. And we recommend that maybe you reach out to the mentor eight weeks out from the exam and start talking to them if you can.

Um, Danita, do you want to talk about the question bank?

Danita: Sure. So we rolled out a new tool that has had great uptake. We call it the CMPP question bank. It was launched in February right before our March exam, and this is a subscription-based tool that provides unlimited access to a hundred and fifty retired exam questions. These are actual authentic questions that appeared on exams in the past.

However, we felt at this point they've been overused a bit and we retired them, and they are a great source as a practice tool that will provide candidates with exposure to the style, the structure, and even the rigor of exam questions. Once you subscribe, there are flexible practice options.

You can take a full one hundred and fifty question exam, or you can take short targeted domain-specific quizzes. There are timed and untimed options, and it also provides progress reports with color-coded analytics, which are great for candidates. It keeps track of your last five practice test results for you, and it also provides access to over a hundred digital flashcards.

And these flashcards cover topic areas such as clinical terminology, publication standards and guidelines, principles of good clinical practice, and key organizations, and what those organization functions are. It helps you know if you're ready to take the exam, but it also helps you prepare to take the exam if you've already applied.

Rob: And I know at the time of this recording, we're also in the midst of putting together a publication preparation tool, Pub's Proficiency. Is that something that might be useful in terms of preparing for the exam?

Danita: Yes, I believe so. The PUPS Proficiency is a comprehensive online training program that is also subscription-based. It was developed by ISMPP in collaboration with Woven Health Collective. And the training program is based on the three competency domains of the CNPP exam, and it takes the learner through various scenarios, and each area of learning within each competency domain is based on a key duty that's been identified as part of that competency domain.

So, I believe that if someone could do well throughout the PUPS Proficiency training program, they would be in a strong position to take the CNPP exam for sure.

Rob: Yeah, it seems like there's more preparation materials than we've ever had before to make sure the candidates are ready, so really, really good to hear. Now, for someone who's thinking about sitting for the exam, they've gone through the handbook, they've looked through the resources, if you could give them one piece of advice, one thing that they should do to make sure that they're ready for the exam and prepared, um, what would you have that be?

Claudia: I, I have two pieces of advice, but I'll give you one. I would say that the most important thing is setting aside time to study. I know a lot of folks who, because they've been publication professionals for a long time and they really know their stuff, they assume they don't need to study.

They say, "Oh, I'm familiar with the resources. I'm familiar with the references. I've read them all in the past." This is, as Danita said, it is a certification program. It's not just a certificate. It's not a simple test. It's difficult for a reason. Um, and you need to take some time to actually make sure you know it all and you know it all really well.

The people that I know who have succeeded, who've passed on the first go or, you know, who've passed period, usually say about two months prior to scheduling the exam they started setting aside some study time every week for, you know, like about eight weeks in the run-up to the test.

That would be my number one piece of advice.

Rob: Excellent. Excellent. How about you, Danita?

Danita: I think I would say really consider the eligibility criteria. Just because someone has two years in the industry might not mean they're ready. I've had several mentors come back to me here and there saying, "Gee, my candidate, I know that on paper they meet the eligibility criteria, but Denita, they've never developed a publication plan."

They're implementing every day. They understand the guidelines and how to be compliant and ethical, but they don't... They're not doing well in this domain, and I don't think they're ready. And so I would really seriously, if I were a candidate, think about just because you meet the two years, do you really feel ready and comfortable?

Because if you're not, it doesn't hurt to wait an additional year because, a, it's an investment in your time, in your company's time. Financially, it's an investment. So I would make sure you feel ready in all the areas of the exam.

Rob: Excellent. Good advice from both of you. I think it's, it's really, really important to remember that the exam is rigorous for a reason. It's not impossible. It's, um, it's sufficiently difficult so that we can differentiate, but certainly passable, with the right amount of study time.

And the other piece that's been brought up on this podcast a few times is it's really not a memorization examination. It's one where experience counts. A lot of what we do in the publication profession is working in that gray zone, trying to figure things out, handle scenarios, and there's nothing that exper-- nothing can replace that except for experience and time in the profession. So really, really important.

Rob: That's all for today. Thank you all for listening. Please take a minute to subscribe to In Plain Cite on your favorite podcast app. Share with your colleagues and rate our show highly if you liked what you heard today.

(Credits)

In Plain Cite is a production of ISMPP, the International Society for Medical Publication Professionals. 

Our production partner is CitizenRacecar. Our producer and editor is Hajar Eldaas. Post production by Alex Brouwer. Publication and promotion by Candice Chantalou.

To join ISMPP today, go to ismpp.org. Becoming a member means you can participate in value packed webinars and receive instant access to exclusive tools and resources. If you're interested, just go to ismpp.org, that's ISMPP.org, to learn more.