Rethinking How We Prompt GenAI

Rob: More and more, we're using AI tools like ChatGPT in our profession. But still, many of us believe we need to speak to them like machines, mastering arcane prompts or learning a whole new language just to get a good response. Our guest today challenges that notion.

This is In Plain Cite, a podcast exploring the biggest questions and trends facing medical publication and communication professionals. I'm your host, Rob Matheis, president and CEO of ISMPP. 

Today's episode is generously sponsored by Avalere Health. 

Joining us today is Conor Grennan, Chief AI Architect at NYU Stern and CEO of AI Mindset, a consulting firm that helps organizations rethink how they interact with generative AI.

In this conversation, we'll explore why the most effective way to work with AI isn't technical at all, it's deeply human. And we'll look at what makes human-centered approaches so powerful and why connection, curiosity, and plain language often beat complex prompts. 

Rob: It's funny, I was gonna ask you, you know, if you had some advice on how to program ChatGPT or a large language model, but I think you'd probably, uh, steer me in a different direction because I used the word program, but I'll let you take that. 

Conor: I like that. It’s funny, it's, it's a common thought. How do we prompt this? How do we program it? But in fact, the best thing we can possibly do is just talk to it like a person. That's hard because, you know, for our brains, because it doesn't look like a person. But ultimately we all have all the skills we need. I think that we're just so used to, when we deal with technology, having a, a pattern or a formula. If you're using, you know, Excel or whatever, you just, you have to learn something before you use technology. Always. That's always been the history of technology. And this is different. It turned out there was nothing to learn. If you just talk to it like a, like a person, you do great. This is a software that behaves like a human, it doesn't behave like a software. 

And so that's why I, I always thought early on the prompt libraries that we had were just not helpful because it was, it was almost as if you said, hey, here's a list of things that you can say to another person. Because if you look at those prompt libraries, that's all they were. And I get it. I get the idea that people want to have these kind of patterns where you say, okay, create a persona and then you know, additional context or whatever it is, and there's an acronym for it, but if you forget about AI for a second and you just think, how would you do it with a person? Here's how you would do it, right? You'd sort of give that new colleague a task, right? “Hey, here's what I want you to do.” And then you add context to that. It's like, Oh, here's what the client likes. Here's what our mandate is, here's the values of the company, et cetera, et cetera. The third part is you refine it back and forth with that colleague until they get it right and that's it. There's nothing else. There's nothing special you have to say to a colleague or something like that, but we do that so instinctively that's what you do with ChatGPT too. You give it a task, you add context, you refine it, and then give it the next thing. But there's no daylight between being a good manager of people and being a good quote unquote prompter, because it's just like talking to a person.

Rob: Yeah. And I imagine it'll take some time for a complete behavior change to happen. We're so used to doing things a certain way that it just, it takes people time to break their patterns. You know, I wanna go back on or to, um, use the word context before, and I'm trying to get a sense for our listeners why context might be so important when we're, we're trying to work with these models.

I. 

Conor: Yeah, I mean, context is everything. You know, if you're talking to a person, like, not somebody you know, but imagine a brand new colleague comes in and they're just, they're brilliant and you know, they're gonna be a star and all that sort of stuff. And they're like, okay, so how can I help you? And what if they've never worked in this field before?

But they're brilliant. They're brilliant. Know they're gonna pick it up very well. But I. You have to give them context around like, okay, so here's who we are, here's how we tend to work. Like when you write a memo, well, I've never written a memo before. Okay, let me show you some examples of what a memo looks like.

And they're like, so should I write like this? No, no. You're talking to it as if you're talking to like a person. No, you wanna, you know, this is a client. Like you have to be really careful. And then, and what about like this? No, no, no. This client isn't finance. They don't talk like that. This is how they talk.

So you give them some examples of a fi. Like, you'd have to give that person tons of context in order for them to be allowed to do their job. Like they could never walk in. You could just say like, Hey, write me a five page memo on the importance of, you know, drug discovery to whatever I, whatever it would be, and make it sort of like our client is whatever it, whatever it would be.

Point being that if you just said that to a person that had no idea, you get a bad answer. But with this, there's a sense that, well, if this knows everything, then it should know how to do that. But remember that it doesn't know who you are or who you're writing for or anything like that. So the context is really, really important.

Just like it's important to give context to a human. Yeah. I mean, I'm, I'm thinking 

Rob: about one particular use case for our people who are medical communication professionals and they draft publications and. If they review literature and synthesize it, and I can imagine, you know, perhaps sending some literature over and into the system to basically see if it could synthesize it and come up with some key takeaways.

But I imagine that would be kind of hard without context as to like, why. Why the model's doing it and what it's gonna be used for, or if it's a synthesis towards a publication or a synthesis that's gonna go up to senior leadership. So, um, any thoughts on, on that kind of thing for that, that use case? 

Conor: So, you know, number one, and one of these demos that I do a lot is just illustrating how good a, a model like Chacha, bt, or any large language model you use, how good it is at imitating human behavior.

And I say that because in communications, which is what we all do, right? I. Still we're flawed at communicating. And the reason we're flawed is because we can't possibly get into the head of the person we're communicating with. And so no matter how good a communicator we are, and I'm guessing a lot of people listening are actually excellent communicators, but even then, you don't necessarily know exactly what your audience would want to hear.

So if you think about, you know, a teacher teaching year after year to a group of 25 students or something like that. That teacher just sort of has found the lowest common denominator, the best analogies that tend to resonate with the most people. But if that teacher was just tutoring a student that he or she knew really, really well over the years, they'd be like, okay, and remember that time where you went into the woods and blah, blah, blah.

So if you think about how physics works, you know, they could give it really a ton of. Extra context so that person would understand. So I say all that to say one of the things that this does really, really well is like, hey, I'm writing this, uh, this paper and I'm trying to convince this person and this person's a, you know, a banker and they've been in finance 10 years, but before that they came from this part of the world.

And also they're Argentinian, which means that they're sort of like a little more bent toward what you start to get a sense of like, oh, this is how I'm going to write this thing and it will do it for you. Instantly. So I always say, you know, write your first draft, but then say, actually the audience is this, this person's a banker and this person's a teacher, and this person's a patient who's looking for new ways of doing this.

And this person is a, a doctor who's overworked, and this is an epidemiologist who, uh, super skeptical. And it will write it for all those different audiences. And I think that's really critical because every time we think, well, you know, like, let's just write an email. It's not about that. It's about how do we tailor what we do.

F and, and have sort of a standin proxy for that audience member. You know, in ChatGPT and the ChatGPT can reach that sort of person in that kind of way. In a way that's very hard for our brain to do. We can do it for a few different kinds of audiences that we know very well. What this does is sort of like democratizes our ability to communicate with a broad set of demographics.

Rob: No, it's super interesting, right? To get the best output and the best results. It, you know, it takes, takes a certain amount of talent to get that done. Now use the words I was waiting for you to use and I was hoping to bring the conversation over, you said, um, first draft. Um, the burning question our audience wants to know is, can we use these models to write the first draft of a medical publication?

Conor: Yeah, no, it's such a good question. So a lot of this, by the way, you know, we should always remember that it's not like there's rules here. Do you know what I mean? Like there aren't any hard and fast rules. So large language models, everything I'm talking about, this is just my viewpoint, having worked on this a ton.

So when we think about first drafts. For sure there is something to be said for, oh, this will help you write your first draft. So I'm actually kind of against that. And the reason I'm against it, but let me clarify, is that if you really have writer's block, I've been there before. I'm, I'm a writer by background.

When you have something like this, can I write your first draft? Yes. But if you just say, Hey, write a two page paper on whatever, you'll get something. I guarantee that sounds like it's been written by ai. Now, if you're not that familiar with AI writing. To you, it'll look fantastic. But to other folks who look at AI writing a lot, it'll be glaringly obvious because it'll ask a lot of rhetorical.

It'll be like, it's not just this, it's this. Or, so what happens now? This happens now. Or We're gonna delve into this, or we're gonna harness the power of ai. Like there's certain phrases that AI just tends to use, and it's very obvious to other people. So this is why I always say. I'm not saying you have to write, like let's say you have to write a five page paper on something.

I'm not saying you have to write five pages out. What I am saying is. I make sure that I write at least one paragraph in my tone of voice, you know? And much better if you can actually take things that you've written before, which are really in your voice and tone, and just dump it in there and say like, Hey, this is the voice and tone and format I like to use is the humor or style.

So that's part of the rough draft. But the other thing is what I do. I go into like, ChatGPT on my phone, or even you can do it on your desktop too, but you just hit the, there's a little microphone button within the, um, the sort of like the text bar there, and you just hit that. And when you do that, it starts to just record you as if you're leaving a voice text.

And I do that for, I. Two or three minutes. So I just like pick up my phone, click on that and be like, Hey, so I gotta write this thing about how AI is gonna impact labor force. And one of the things I'm really thinking is that, you know, Microsoft came out with this index, but I actually don't really think so because I think that it's gonna be a lot slower.

But on the junior recruiting side, that's where I think it's really gonna be impactful. And I'll talk for like two minutes just on my thoughts and it'll capture all of that. And then I'll say, okay, use that as your first draft. So it doesn't have to be a first draft in the traditional sense, it has to just be your thoughts.

So it can play off something, otherwise it's just gonna grab the most generic AI is really gonna impact work because it can augment or, you know what I mean? So I don't mean write a full first draft, I just mean get something in there so it knows what to kind of base it off. 

Rob: It's a really good suggestion too, because now you're not just talking to a search bar, you're actually talking to a phone, and it's a little bit closer to just conversational, which I think, think goes a long way.

A hundred percent. You know, I, I find that our people or our professionals tend to always think about just writing drafts as the only use case, but within our profession, we do so many different things that I often don't wonder. People don't realize that they can use ChatGPT and other models for these purposes.

You know, one of the things that we do regularly is we have to make decisions about like authorship and you know, who, who actually meets byline authorship, who doesn't? Um, and we have I-C-M-J-E criteria that we use to help to establish that. I wonder if a use case for AI could be to help to do some of that type of decision making.

Conor: Yeah. So let's say you had to do that and I'm, I'm not in your world, but let's say you had to define authorship of, so I am in the academia world, so I do know a little bit about that. You know, a lot of times we'll be looking for like, well, how would we do that? How would we use a large language model to do that?

And it's interesting, and I get this question by the way, a lot I. With, you know, investors and things like, Hey, how do we use a large language model to better surface opportunities? All that sort of stuff. And you can multiply that across every industry and everything else. And you can imagine the kind of questions that fly my way and, and what the ones that are in people's brains.

So, but taking that one for a second, I always come back to the same advice on it. So I say. Don't think that there's like a hack to doing this with large language models instead, and this is kind of the beauty of it, I think. How do you do it now? Do you know what I mean? Like, and, and we're gonna augment that process.

So if, for example, you know, to take an example, I know maybe a little bit better, I was working with a, somebody who was running a, like a. Private investment bank, basically. And they were saying, well, we're trying to figure out whether we should open an office in, you know, Denver or something like that. Can Chad BT help with that?

And I was like, well, it can't really help with that. And he was like, well, that's kind of like what we really need. I was like, but, but, but hold on for a second. Like, talk to me about how do you figure that out now? Like, forget AI for a second. Like what's the process like? Well, we sort of like do research on this and then we figure out this and then we do this, and then we run it through a bunch of people and then we figure, I'm like, okay.

So this is how your large language models is gonna speed up all that. Like literally write down those steps. What are the steps? So step one is, well, we investigate this and we read this and we model this after about, okay, so your large language model can help you with that because just say, Hey, here's what I'm trying to do, Chad, GBT, I'm looking at this and I'm trying to determine this and you're not trying to hit the end result.

So let's say it's a five step process. Take it step by step. It's using ChatGPT for every step along the way, rather than going from sort of like point A to point Z by entering a special prompt or something like that. 

Rob: And it's, it's a really good suggestion. It's interesting, we had our, our pre-call not so long ago, and what I took away from our discussion was this idea that I could just be conversational with the model over the course of the day.

Like, you know, well the first thing I'm gonna do today is I'm gonna go to a meeting about X, Y, Z, and, and just, and then it came back with things that it could help me do that I never thought of before. So it's amazing. I found that really, it's really neat. And I think that's kind of what you're saying here is that.

You know, maybe I don't say how do I select authors, but I say, this is what I'm doing now to try to do this task. And all of a sudden there's things that that ChatGPT says 

Conor: that it can do to help me. I, I think that all the time, you know, and Chet BT is actually, and sometimes this can be annoying for people, but it tends to ask follow-up questions, right?

So I'll be like, you know, if I'm trying to eat healthier, I'll be like, Hey, this is what I'm eating right now. What do you think? It's like, oh, well actually this is pretty good because blah, blah, blah, blah, blah. Do you want me to give you a couple suggestions about ways that you could tweak this to make it more interesting so that I'm like, actually, yeah, sure.

Go on. It's like, also, do you want me to add something in case? 'cause I know you have kids. Do you want them to be doing the, I was like. Sh. Yeah, why not? So it's, it's one of these things which is unbelievably helpful because you couldn't possibly think to ask those questions because we're not used to it.

We're not used to having something at our fingertips where you can just say, Hey, yeah, gimme that and I'll explain this to me, but in a better analogy, and use this thing that I'm like really passionate about to help explain to me and dumb this down, and now make this and now write it out for somebody, and now make a drawing and now make a flow chart about.

We're just not used to being able to do that. 'cause you don't have somebody to command around. And even if you did, they would take hours and hours and hours to do it. So it's the creativity around figuring out like what it can help you do. But the great thing is, and you hit the nail on the head, is it can tell you how it can help you.

That's the other thing. It's, it's, you just don't need anything. You don't, and I don't wanna say, you don't have to think anymore, you really do. This really favors the critical thinkers. But, but it is an amazing tool of just helping advance your thinking on things. 

Rob: It kind of tells you things that you didn't know you really needed.

Right. So, and I like that. I like that about it. So. Alright, so some of our listeners, um, that are listening to this and they're thinking, you know what, it's not worth it because all those hallucinations and Yeah. You know, I guess in brief, like what would you say to those, those people about the hallucinations?

I. 

Conor: Yeah, I get it. I get the, I get the worry because why wouldn't that be a worry? I mean, you know, if Google doesn't really hallucinate, right? So why would, so couple things. First of all, if you haven't used ChatGBT and things like that in a while, you'll find hallucination rates are way, way down. In the early days it would hallucinate all the time.

That's not what this thing does anymore. The hallucination rates are just, they're just very, very low. And so I think that's important to note. The second thing though, is. It's not gonna hallucinate in terms of brainstorming process. Where I'm always careful is if I say, Hey, who's speaking at the I-S-M-P-B conference next month?

I would not trust it there. Even if it's searching the web for that, it might just grab the wrong webpage. Something like that. Like for that I just go double check. 

Rob: Yeah. I feel like I've heard you say something about this before, but you know, even humans make mistakes, so you would not hire an employee and you wouldn't use the system.

So it's, it's, it's 

Conor: fair. I think that's actually, I think that's really critical too, because a lot of times we're comparing it to a software that doesn't make any mistakes. So like a calculator won't make mistakes. However, a calculator is pretty limited in what it can do. Do you know what I mean? So instead, you want to kind of think about this as like, does this outperform the best available human that you have?

And, and not always, right? I mean, there's sometimes where I'll, I'll need advice on something and you know, my wife is just gonna. Tell me better than Chacha D can tell me, although it is incredible giving, kind of empathetic advice. But remember that it's, if it does make a mistake or doesn't give you the answer you want, it's not a one-to-one thing.

Like, again, it's not a calculator because it's not limited. The whole point of this thing is that it's more like a car than a train. It can go anywhere, but the risks are probably, you know, bigger with a, with a car than a train. But you're not gonna take a train just to where a train can take you like you want that car. 

Rob: Well, that's us for today. Thank you all for listening. Please take a minute to subscribe to In Plain Cite on your favorite podcast app. Share with your colleagues and rate our show highly if you like what you heard today.

In Plain Cite is a production of ISMPP, the International Society for Medical Publication Professionals. This episode is generously sponsored by Avalere Health.

Our production partner is CitizenRacecar. Our producer and editor is Hajar Eldaas. Post production by Alex Brouwer. Publication and promotion by Candice Chantalou.

To join ISMPP today, go to ismpp.org. Becoming a member means you can participate in value packed webinars and receive instant access to exclusive tools and resources. If you're interested, just go to ismpp.org, that's I S M P P dot org, to learn more.