Technology in Clinical Trials #6 – From Lab Coats to Chat Bots: AI Prompts are the New 'Lab Assistants' in Clinical Trials

Welcome back to my monthly newsletter on ‘technology in clinical trials’.

I hope everyone had a good vacation season and is excited about the latest developments in the clinical research field. Based on my observations and what I have heard from acquaintances, it seems that the implementation of artificial intelligence (AI)-based tools is gaining traction. Hence, I believe this trend of increased AI integration warrants discussion on how to best harness the power of AI.

For this reason, I have dedicated this month’s issue to a topic that many people out there may find useful and applicable to their everyday activities – namely, AI prompting.

🖥️ The Basics of Prompting

Before we delve into specific use cases of AI prompting in the clinical research field, it is important that all readers are on the same page when it comes to understanding what prompts are. As defined in Effective Prompts for AI: The Essentials, prompts are “What and how you tell something to the AI for it to respond in a way that generates useful responses for you.” In other words, prompts are like the instructions or questions you feed an AI system to retrieve specific information or have the AI system perform a specified function.

Writing a prompt may seem like a simple task at first, but receiving an accurate answer or having an AI perform a specific function may warrant refinement of your prompts. As described in How to Write Prompts for ChatGPT, “If you enter a basic prompt, you’ll get a bare-bones, encyclopedic-like answer... Do that enough times and you’ll convince yourself that this is just a toy and you can get better results from an internet search engine.”

Obtaining an answer or end result that you are looking for may take several reiterations of your prompt. For example, if you ask ChatGPT to “generate a summary of the attached study protocol” you might end up with a general and extensive summary that does not fit the purpose of what you are going to use the summary for. To obtain a more accurate and satisfactory response from the AI tool, you should consider the below points during prompt iteration:

1.      What persona or role should the AI assistant embody?

2.      In what context, time frame, or scenario do you want the AI's response to be situated?

3.      Are you asking the AI the actual question you want answered, or is there an underlying query you're trying to address?

4.      Does your input provide enough information and context for the AI to generate a comprehensive, detailed, and nuanced response?

5.     Is your prompt clear and specific, or is it ambiguous and unfocused? The AI's response will likely reflect the nature of your input.

Let’s say you’re a medical writer tasked with developing an intermediate clinical study report (CSR) and would like the AI tool to summarize the overall study design based on the source protocol. This is how you should answer the above questions:

1.      What persona or role should the AI assistant embody? -> Medical writer in a biotech/pharmaceutical/CRO company

2.      In what context, time frame, or scenario do you want the AI's response to be situated? -> Appropriate the study design to an intermediate CSR

3.      Are you asking the AI the actual question you want answered, or is there an underlying query you're trying to address? -> You want the AI to summarize which endpoints were evaluated up until the intermediate data cut-off (DCO) date

4.      Does your input provide enough information and context for the AI to generate a comprehensive, detailed, and nuanced response? -> Stating “intermediate CSR” may not be sufficient. To obtain a more detailed response, include information on study aspects you would like the study design summary to encompass – eg, study population, main inclusion and exclusion criteria, study intervention groups, endpoints analyzed at the intermediate DCO date

5.     Is your prompt clear and specific, or is it ambiguous and unfocused? The AI's response will likely reflect the nature of your input. -> In short, are you asking the AI a simple question or did you provide it with sufficient details to obtain a specific answer or end results?

If we take our original prompt of “generate a summary of the attached study protocol” and apply the above-stated principles, a refined prompt would look something like this:

“I am a medical writer who works for a pharmaceutical company that specializes in cardiovascular diseases. I have been tasked with writing an intermediate CSR based on study results received up to date. Could you help me develop an overall summary to include in the CSR introduction section according to the attached study protocol?

The overall summary should:

·        Provide an overall description of the conducted study

·        Outline all endpoints evaluated up until the intermediate DCO date

·        Describe the study population including the main inclusion and exclusion criteria

·        Define the study intervention groups and treatment periods

·        Sum up the main statistical methods used

In addition, the summary should not exceed 600 words.”

Since I referred to the medical writing role in the above example, let’s understand the importance of integrating the use of AI in the medical writing profession.

📝 Prompting for Medical Writers

[Industry Report] Celegence and the Regulatory Affairs Professionals Society (RAPS) conducted a Regulatory Readiness & Resources Survey in 2024. There was a total of 698 survey respondents from the regulatory affairs field from 139 pharma companies headquartered chiefly in the US, Europe, or the Asia-Pacific regions. According to the survey, there is an urgent need for support in medical writing due to mounting time pressures in the delivery and submission of clinical and regulatory documentation. The survey revealed that 57% of participating companies intend to allocate resources towards medical writing technology improvements in the coming year. This investment priority nearly matches that of eCTD v4.0, with these two areas emerging as the primary focuses for immediate regulatory IT expenditure.

The survey highlighted that regulatory professionals identified clinical study protocol and report writing and the creation of regulatory documents as the most critical areas requiring additional assistance in medical writing. Over half of the respondents recognized the potential of AI in enhancing their workflows. Specifically, 56% saw a need for AI in data extraction, while 53% identified its potential in information summarization. However, current AI adoption in these areas remains low, with only 9-10% of respondents currently utilizing AI for these purposes in medical writing. Additionally, 12% of participants reported that they are in the process of integrating AI for automated report generation from diverse sources, which represents the pinnacle of opportunities in this field.

Despite these forward-looking aspirations, the survey uncovered a significant capability gap. More than half (53%) of the pharmaceutical industry respondents admitted that their organizations lacked the necessary internal expertise to implement AI technologies independently. This finding underscores the need for external support or partnerships to fully leverage AI's potential in medical writing within the pharmaceutical sector.

Although there is a clear potential for the integration of AI tools in the medical and regulatory writing professions, the decision to implement such tools is normally made by an organization’s senior stakeholders. Whether the organization has implemented AI tools or has yet to implement them, medical writers can get a head start with the use of such tools by understanding how to use AI prompting effectively.

[Blog post] According to Michelle Guillemard from Health Writer Hub, below are some tips on how to create better AI prompts:

1.      Precise Instructions: Craft clear, detailed prompts to guide the AI. Specify the topic, required information, and desired content type to ensure aligned outputs.

2.      Iterative Refinement: If initial results fall short, refine your prompts. Experiment with variations and provide additional context to improve accuracy.

3.      Fact Verification: Always cross-check AI-generated information with reputable sources to maintain accuracy and reliability.

4.      Contextual Enrichment: Enhance prompts with relevant details such as patient demographics or specific research questions to obtain more tailored responses.

5.      Brevity Emphasis: Request concise responses in your prompts to receive focused, easy-to-digest information.

6.      Interactive Approach: Engage in a dialogue with the AI, asking follow-up questions to refine and improve content quality.

7.      Human Touch: Always review, edit, and refine AI-generated content. Adapt the tone and style to suit your audience and writing standards.

8.      Ethical Considerations: Uphold ethical standards by avoiding plagiarism (Mati’s personal tip: avoid plagiarism by providing complete references to external sources), respecting patient privacy, and adhering to relevant regulations when using AI-generated content.

By following the above guidelines, medical writers as well as any other clinical or regulatory processionals can effectively leverage AI as a powerful tool while maintaining the integrity and quality of their work.

I hope you found the explanation and examples of prompts in this issue of the newsletter helpful. Since I like to encourage readers to share their thoughts and experience – what are your best tips on refining prompts? And what aspects of prompt iteration do you find the most challenging?

❗ Disclosure statement ❗

Lastly, I would like to point out that I am NOT associated with and am NOT sponsored by any of the organizations or individuals I refer to in this edition of the newsletter.