Exploration of Localized Large Language Models to Identify Safety Reports in the Adverse Event Reporting System

Description

*Applications will be reviewed on a rolling-basis.

CDER Office and Location: A research opportunity is available within the Food and Drug Administration (FDA) in The Center for Drug Evaluation and Research (CDER), located at Silver Spring, Maryland.

The Center for Drug Evaluation and Research (CDER) performs an essential public health task by making sure that safe and effective drugs are available to improve the health of people in the United States. As part of the U.S. Food and Drug Administration (FDA), CDER regulates over-the-counter and prescription drugs, including biological therapeutics and generic drugs. This work covers more than just medicines.

Research Project: This project is located in the Office of Surveillance and Epidemiology (OSE). The project will explore the application of localized large language models (LLMs) to identify safety-related clinical concepts from individual case safety reports (ICSRs) in the FDA Adverse Event Reporting System (FAERS). Research aims include:

• 1) To evaluate the performance of LLMs to identify safety-related clinical concepts from ICSRs.
• 2) To compare the performance of LLMs to a currently implemented rule-based natural language processing system.
• 3) To understand the utility, strengths, and limitations of applying LLMs to ICSRs in FAERS. Understanding of safety-related clinical concepts described in ICSRs can facilitate and increase the efficiency of the review of ICSRs to support signal detection and evaluation processes for pharmacovigilance.

Learning Objectives: Under the guidance of the mentor, the participant will gain insight into best practices in drug and biological product post market safety surveillance. The participant will be provided hands-on training to retrieve individual case safety reports from FAERS, an FDA database, as well as to review the report to understand the contents related to safety-related clinical concepts relevant to pharmacovigilance. As a result of the appointment, the participant will be able to:

• 1) Identify a dataset of safety reports to be included in the training and test set for LLMs
• 2) Collect, review, compare, and analyze safety-related clinical concepts output from the LLM to that of the rule-based natural language processing system
• 3) Interpret findings to understand the usefulness, strengths, and limitations of LLMs in this context
• 4) To prepare a presentation and write a manuscript suitable for publication on the findings from this research

Appointment Length: The appointment will initially be for one year, but may be renewed upon recommendation of FDA and is contingent on the availability of funds.

Level of Participation: The appointment is full time.

Citizenship Requirements: This opportunity is available to U.S. citizens, Lawful Permanent Residents (LPR), and foreign nationals. Non-U.S. citizen applicants should refer to the Guidelines for Non-U.S. Citizens Details page of the program website for information about the valid immigration statuses that are acceptable for program participation.

This program, administered by ORAU through its contract with the U.S. Department of Energy to manage the Oak Ridge Institute for Science and Education, was established through an interagency agreement between DOE and FDA. The participant will receive a monthly stipend commensurate with educational level and experience. Proof of health insurance is required for participation in this program. Participants do not become employees of FDA, DOE or the program administrator, and there are no employment-related benefits.

Completion of a successful background investigation by the Office of Personnel Management is required for an applicant to be on-boarded at FDA.  OPM can complete a background investigation only for individuals, including non-US Citizens, who have resided in the US for a total of three of the past five years.

FDA Ethics Requirements

If an ORISE Fellow, to include their spouse and minor children, reports what is identified as a Significantly Regulated Organization (SRO) or prohibited investment fund financial interest in any amount, or a relationship with an SRO, except for spousal employment with an SRO, and the individual will not voluntarily divest the financial interest or terminate the relationship, then the individual is not placed at FDA.   For additional requirements, see FDA Ethics for Nonemployee Scientists. 

FDA requires ORISE participants to read and sign their FDA Education and Training Agreement within 30 days of his/her start date, setting forth the conditions and expectations for his/her educational appointment at the agency.  This agreement covers such topics as the following:

• Non-employee nature of the ORISE appointment;
• Prohibition on ORISE Fellows performing inherently governmental functions;
• Obligation of ORISE Fellows to convey all necessary rights to the FDA regarding intellectual property conceived or first reduced to practice during their fellowship;
• The fact that research materials and laboratory notebooks are the property of the FDA;
• ORISE fellow’s obligation to protect and not to further disclose or use non-public information.

Qualifications

Applicants must be currently pursuing or have received a Master or Doctorate degree in medicine, pharmacy, epidemiology, public health, bioinformatics, pharmaceutical science, or related field appropriate for the opportunity. 

Eligibility Requirements

• Degree: Master's Degree or Doctoral Degree received within the last 60 months or anticipated to be received by 8/30/2024 12:00:00 AM.