RAG-HPO

• Retrieval-Augmented Generation to support HPO text mining
• Method developed by the Posey Lab, Baylor University
• Adapted from the code for HPO-RAG, 2025, Genome Med

To run the notebook, install the following packages

pip install hpo-toolkit
pip install fastembed
pip install sentence_transformers
pip install faiss-cpu

import hpotk
from pprint import pp
from sentence_transformers import SentenceTransformer
import re
import json
import numpy as np
import pandas as pd
from typing import Dict, List, Any
from fastembed import TextEmbedding
from sentence_transformers import SentenceTransformer

/Users/peterrobinson/GIT/bio-ontollm-hm/.venv/lib/python3.12/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

Initialize the HPO

We need the full HPO ontology in this notebook to check the labels of the HPO ids that are returned from the LLM, not to perform the RAG itself.

store = hpotk.configure_ontology_store()
hpo = store.load_hpo()

FAISS Package

Faiss is a library for efficient similarity search and clustering of dense vectors. See the documentation for details on how to install. We are install cpu only version, it is also possible to install a GPU version on many architectures. It was necessary to set the number of threads to one because sentence-transformers/torch initializes its own bundled OpenMP runtime and faiss (also OpenMP-parallelized internally) initializes a second, separate one. Running these in parallel (more than one thread) lead to a segfault on the Macbook this notebook was developed on. It may not be required on other architectures.

#! pip install faiss-cpu
import faiss
faiss.omp_set_num_threads(1)

Initialization

The following functions are required to load the vector databases and metainformation that we created in the HPO_Vectorization notebook.

get_embedding_model

We are retrieving the same embedding model used in the HPO_Vectorization script so that we can work with these embeddings.

def get_embedding_model(
    sbert_model: str = 'pritamdeka/SapBERT-mnli-snli-scinli-scitail-mednli-stsb',
):
    return SentenceTransformer(sbert_model)

create_faiss_index

FAISS (Facebook AI Similarity Search) is a library for storing a large set of vectors and quickly finding the ones most similar to a given query vector. We build an index from the HPO vectors, meaning we can compare the vectors using the fast FAISS code rather than one-by-one in vanilla Python.

def create_faiss_index(emb_matrix: np.ndarray, metric: str = 'cosine'):
    # Build FAISS index for embeddings
    dim = emb_matrix.shape[1]
    if metric == 'cosine':
        faiss.normalize_L2(emb_matrix)
        index = faiss.IndexFlatIP(dim)
    else:
        index = faiss.IndexFlatL2(dim)
    index.add(emb_matrix)
    return index

Loading metadata and embeddings

This function loads the files hpo_meta.json and hpo_embedded.npz that were created in the HPO_Vectorization.ipynb notebook.

def load_vector_db(meta_path: str = 'hpo_meta.json',
                   vec_path:  str = 'hpo_embedded.npz'):
    if not os.path.exists(meta_path) or not os.path.exists(vec_path):
        raise FileNotFoundError(f"DB files not found: {meta_path}, {vec_path}")
    try:
        with open(meta_path, 'r', encoding='utf-8') as f:
            combined = json.load(f)
            constants = combined.get('constants', {})
            entries  = combined.get('entries', [])
    except Exception as e:
        raise RuntimeError(f"[FATAL] Could not load metadata JSON: {e}")
    try:
        arr = np.load(vec_path)
        emb_matrix = arr['emb'].astype(np.float32)
    except Exception as e:
        raise RuntimeError(f"[FATAL] Could not load embedding npz: {e}")

    if len(entries) != emb_matrix.shape[0]:
        print("[WARN] Metadata entries count and embedding rows mismatch "
              f"({len(entries)} vs {emb_matrix.shape[0]})")
    # ─── Reconstruct docs list in the original output format ───
    docs = []
    for entry, vec in zip(entries, emb_matrix):
        hp_id = entry.get('hp_id')
        const = constants.get(hp_id, {})
        doc = {
            'hp_id':          hp_id,
            'info':           entry.get('info'),
            'lineage':        const.get('lineage'),
            'organ_system':   const.get('organ_system'),
            'direction':      entry.get('direction'),
            # preserve these keys even if absent in the new JSON:
            'depth':          const.get('depth'),
            'parent_count':   const.get('parent_count'),
            'child_count':    const.get('child_count'),
            'descendant_count': const.get('descendant_count'),
            'embedding':      vec
        }
        docs.append(doc)
    return docs, emb_matrix

embed_query

Creates an embedded vector represent the query that we wish to compare to the HPO vector database.

def embed_query(text: str, model, metric: str = 'cosine'):
    if hasattr(model, 'encode'):
        vec = model.encode(text, convert_to_numpy=True)
    else:
        vec = np.array(list(model.embed([text]))[0], dtype=np.float32)
    if vec.ndim == 1:
        vec = vec.reshape(1, -1)
    if metric == 'cosine':
        faiss.normalize_L2(vec)
    return vec

_collect_metadata_best

Purpose: Given a phrase (and its precomputed embedding), retrieve a shortlist of candidate HPO terms from the vector index — a hybrid of semantic similarity search and lexical (token) overlap — to be passed downstream (e.g. to system_message_II’s LLM-based term selection) as the candidate list.

• Run a FAISS approximate nearest-neighbor search for the top top_k (default 500) most similar embeddings to query_vec.
• Tokenize the input phrase into a lowercase word set for overlap checking.
• Iterate candidates in descending similarity order, and for each one accepts it into the results if any of three conditions hold:
  • its info text shares at least one word with the query phrase (lexical overlap), or
  • its cosine similarity is at or above similarity_threshold (default 0.35), or fewer than min_unique (default 15) results have been accepted so far (a guaranteed-fill fallback, so weakly-matching phrases still return a usable minimum candidate set).
• Deduplicate by hp_id (a term already added is skipped if seen again), and stops once max_unique (default 20) results are collected.
• Return a list of dicts with hp_id, phrase, definition, organ_system, and similarity for each accepted candidate.

def _collect_metadata_best(
    phrase: str,
    query_vec: np.ndarray,
    index: faiss.Index,
    docs: List[Dict[str, Any]],
    top_k: int = 500,
    similarity_threshold: float = 0.35,
    min_unique: int = 15,
    max_unique: int = 20
) -> List[Dict[str, Any]]:
    """
    Single‐pass hybrid retrieval: token overlap, threshold, fill to min_unique.
    """
    clean_tokens = set(re.findall(r'\w+', phrase.lower()))
    dists, idxs = index.search(query_vec, top_k)
    sims, indices = dists[0], idxs[0]

    seen_hp = set()
    results = []

    for sim, idx in sorted(zip(sims, indices), key=lambda x: x[0], reverse=True):
        if len(results) >= max_unique:
            break
        doc = docs[idx]
        hp = doc.get('hp_id')
        if not hp or hp in seen_hp:
            continue

        info = doc.get('info', '') or ''
        token_overlap = bool(clean_tokens & set(re.findall(r'\w+', info.lower())))

        # accept if token overlap, or above similarity threshold, or to reach min_unique
        if token_overlap or sim >= similarity_threshold or len(results) < min_unique:
            seen_hp.add(hp)
            results.append({
                'hp_id': hp,
                'phrase': info,
                'definition': doc.get('definition'),
                'organ_system': doc.get('organ_system'),
                'similarity': float(sim)
            })
    return results

Parsing JSON response

We have the LLM return its response(the findings) in JSON format. The following functions parse the JSON

def clean_and_parse(s: str):
    # Extracts and parses JSON from string
    try:
        m = re.search(r'\{.*\}', s, flags=re.S)
        js_str = m.group(0) if m else s.strip()
        return json.loads(js_str)
    except Exception:
        return None

def extract_findings(response: str) -> list:
    # Extracts findings from LLM response
    if not response:
        return []
    parsed = clean_and_parse(response)
    if not isinstance(parsed, dict):
        return []
    return parsed.get("phenotypes", [])

process_findings

Purpose: Takes the list of phenotype findings extracted by the first LLM pass (system_message_I) and, for each finding, enriches it with candidate HPO metadata and the original clinical sentence it came from — producing a DataFrame ready for the next LLM stage (system_message_II) to pick the best HPO match from.

• Sentence splitting: splits clinical_note into sentences on ., used later for context matching. (Naive — see caveat below.)
• Embedding: for each finding’s phrase, computes its embedding vector via embed_query. — Candidate retrieval: calls _collect_metadata_best (the function from your previous message) to get up to keep_top candidate HPO terms — note min_unique and max_unique are both set to keep_top, which pins the result to exactly keep_top candidates rather than “up to” that many, removing the variable-size flexibility that function otherwise supports. — Best sentence match: finds which original sentence in the note shares the most word-overlap with the phrase, using simple bag-of-words intersection count — this becomes original_sentence, giving the downstream LLM the full-sentence context (as system_message_II’s prompt format expects).
• Row assembly: builds one output row per finding, containing the phrase, its category, its candidate metadata list, the best-matching sentence, and a patient_id.

Output: a DataFrame with one row per extracted finding, columns phrase, category, unique_metadata (list of candidate dicts), original_sentence, patient_id.

def process_findings(findings, clinical_note: str, embeddings_model, index, docs,
                    metric: str = 'cosine',
                    keep_top: int = 15):
    """
    Processes findings and returns DataFrame with phrase, category,
    metadata, sentence, patient_id.
    - keep_top: number of unique metadata entries to retrieve.
    """
    # ─── Split note into sentences for context matching ───
    sentences = [s.strip() for s in clinical_note.split('.') if s.strip()]
    rows = []

    for f in findings:
        phrase = f.get('phrase', '').strip()
        category = f.get('category', '')
        if not phrase:
            continue

        qv = embed_query(phrase, embeddings_model, metric=metric)
        unique_metadata = _collect_metadata_best(
            phrase=phrase,               # literal text for token-phase
            query_vec=qv,                # embedded vector
            index=index,                 # FAISS index
            docs=docs,                   # list of HPO docs
            top_k=500,                   # FAISS retrieval size
            similarity_threshold=0.35,   # max distance for semantic matches
            min_unique=keep_top,         # ensure at least keep_top entries
            max_unique=keep_top          # cap at keep_top entries
        )

        fw = set(re.findall(r'\b\w+\b', phrase.lower()))
        best_sent, best_score = None, 0
        for s in sentences:
            sw = set(re.findall(r'\b\w+\b', s.lower()))
            score = len(fw & sw)
            if score > best_score:
                best_score, best_sent = score, s

        rows.append({
            'phrase':           phrase,
            'category':         category,
            'unique_metadata':  unique_metadata,
            'original_sentence': best_sent,
            'patient_id':       f.get('patient_id')
        })

    return pd.DataFrame(rows)

RAG-HPO Pipeline

With all of the setup done, we can now present a simplified version of the RAG-HPO pipeline

System prompts

Tell the model what we expect of it.

• system_message_I: Extraction and triage. Scans the full clinical note and pulls out every phenotype-related phrase, sorting each into one of five categories (Abnormal, Normal, Family History, Other, Suspected). Only the Abnormal findings are kept for HPO mapping — this stage’s job is to cast a wide net over the note and filter out everything that isn’t an actual observed abnormal finding (history, demographics, suspected-but-unconfirmed diagnoses, etc.) before the more expensive HPO-matching stage runs.
• system_message_II: Constrained selection. Given a single abnormal phrase, its originating sentence, and a fixed list of candidate HPO terms (already retrieved via embedding + token-overlap search), picks the one best-matching term — strictly from the supplied candidates, never inventing a new ID. This stage doesn’t search the ontology itself; it disambiguates among options _collect_metadata_best already narrowed down, using the surrounding sentence for context and returning null when no candidate is a good enough fit.

system_message_I = "You are a highly specialized clinical information extraction system. Your task is to identify *all* phenotypic mentions in a clinical note and categorize them into exactly one of: Abnormal, Normal, Family History, Other, or Suspected.\n\n**Instructions:**\n1. **Scan Every Sentence**: Extract every phrase that describes a phenotype—signs, symptoms, abnormal findings, test results, anatomical observations—or any diagnostic conjecture or history statement.\n2. **Assign One Category**:\n   • **Abnormal**: Any *observed* finding deviating from healthy (disease signs, lab abnormalities, pathological conditions). *These will be sent on for HPO mapping.*\n   • **Normal**: Explicit statements of normalcy.\n   • **Family History**: Phenotypes in relatives.\n   • **Other**: Personal medical history, risk factors, demographics, or any non‐finding statement (e.g., “history of HIV,” “smoker,” “height 5′10″”).\n   • **Suspected**: Any mention of a possible or rule‐out diagnosis (e.g., “possible sciatica,” “rule out PE”).\n3. **Preserve Specificity**: Extract the phrasing exactly as written.\n4. **Output JSON Only**: Return `{ \"phenotypes\": [ { \"phrase\": ..., \"category\": ... }, … ] }`.\n\n**Post‐processing:** Immediately drop anything not labeled Abnormal before HPO mapping."
system_message_II = "You are an expert clinical phenotype mapper.  \nGiven one abnormal phenotype phrase and a list of candidate HPO terms, select exactly one best matching HPO term—and only from those candidates.  Do not invent, hallucinate, or suggest any term not in the provided list.  \n\nWhen you receive input, it will be a JSON object:\n{\n  \"phrase\": \"​…​\",\n  \"category\": \"​…​\",\n  \"original_sentence\": \"​…​\",\n  \"candidates\": [ {\"term\": \"​…​\", \"id\": \"HP:…​\"}, … ]\n}\n\nDecision steps:\n1. Leverage context: Use the original_sentence to capture nuance.  \n2. Match specificity: Pick the candidate whose term description most precisely matches the phenotype.  \n3. If two are equally good, choose the one with the highest semantic similarity score.  \n4. If none fits well (similarity < 80 or no clear match), return null.  \n\nOutput requirement:\nReturn exactly one JSON object, nothing else:\n\n    {\"hpo_id\": \"HP:0001234\"}\n\nor\n\n    {\"hpo_id\": null\"}\n\n—no commentary, no extra keys, no surrounding text."

Initialize the LLM model via ollama

We will use the relatively small gemma3:12b model which can be downloaded from ollama (ca. 8GB) and can be run on many laptops.

from langchain_ollama import ChatOllama

llm_client = ChatOllama(model="gemma3:12b", temperature=0)

The clinical query

This is a description of a patient encounter. Our goal is to extract HPO terms that describe the clinical manifestations of the affected individual. The description here is taken from the training data presented in the RAG-HPO GitHub repository.

clinical_note = "A 43-year-old Caucasian female experienced MCAS symptoms at age 18: specific foods and odours caused flushing, rashes, itching, wheezing, dizziness and nausea. At age 20, she noted problems with postprandial bloating/pain, constipation, evacuating stool and flatus with rotten egg odour. Restless legs syndrome (RLS) gradually developed. At age 23, she developed orthostatic lightheadedness and tachycardia, body pain, generalised weakness and painful dependent leg oedema. Ultimately, she suffered from 45 individual clinically significant symptoms (table 1). For 6 years, she became disabled owing to orthostatic symptoms, fatigue and body pain. Faecal evacuation resulted in syncope with efforts >3 min. Early satiety led to a liquid diet. Pressure-induced hives and angioedema, paraesthesia and nocturnal urination (seven times nightly) interfered with sleep. Over 16 years, 19 physicians failed to diagnose POTS and MCAS which were established at a second location of the Mayo Clinic. Supine pulse of 80 beats/min increased to 160 after standing 10 min. Facial rash and oedema, cold, blue hands, Terry’s fingernails and dermatographism were present. "

Time/Memory tracker

Not necessary, but interesting to see how many resources are consumed in each step

import psutil, os, time
from contextlib import contextmanager

process = psutil.Process(os.getpid())

@contextmanager
def track(step_name: str):
    mem_before = process.memory_info().rss / 1e9
    t0 = time.perf_counter()
    yield
    elapsed = time.perf_counter() - t0
    mem_after = process.memory_info().rss / 1e9
    print(f"[{step_name}] time: {elapsed:.2f}s | "
          f"memory: {mem_before:.2f} GB → {mem_after:.2f} GB "
          f"(Δ {mem_after - mem_before:+.2f} GB)")

with track("Load embedding model"):
    emd_model = get_embedding_model()
with track("Load vector DB"):
    docs, emb_matrix = load_vector_db(meta_path="hpo_meta.json", vec_path="hpo_embedded.npz")
with track("Build FAISS index"):
    index = create_faiss_index(emb_matrix, metric='cosine')

Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
Loading weights: 100%|██████████| 199/199 [00:00<00:00, 47079.16it/s]

[Load embedding model] time: 3.46s | memory: 0.70 GB → 0.72 GB (Δ +0.03 GB)
[Load vector DB] time: 0.28s | memory: 0.72 GB → 1.53 GB (Δ +0.80 GB)
[Build FAISS index] time: 0.06s | memory: 1.53 GB → 1.97 GB (Δ +0.44 GB)

LangChain

• SystemMessage: Sets the behavior, persona, or rules for the LLM (e.g., “You are an expert medical coder…”).
• HumanMessage: Represents the actual question provided by the user (in this case, s clinical_note).
• invoke: Sends the structured list of messages to the LLM (here, our local gemini model)

from langchain_core.messages import SystemMessage, HumanMessage

messages = [
    SystemMessage(content=system_message_I),
    HumanMessage(content=clinical_note),
]

response = llm_client.invoke(messages)
findings_list = extract_findings(response.content) # a list of dictionaries, these are the unstructured medical phrases identified by the LLM
findings_list[:10]

[{'phrase': 'MCAS symptoms', 'category': 'Suspected'},
 {'phrase': 'flushing', 'category': 'Abnormal'},
 {'phrase': 'rashes', 'category': 'Abnormal'},
 {'phrase': 'itching', 'category': 'Abnormal'},
 {'phrase': 'wheezing', 'category': 'Abnormal'},
 {'phrase': 'dizziness', 'category': 'Abnormal'},
 {'phrase': 'nausea', 'category': 'Abnormal'},
 {'phrase': 'postprandial bloating/pain', 'category': 'Abnormal'},
 {'phrase': 'constipation', 'category': 'Abnormal'},
 {'phrase': 'evacuating stool', 'category': 'Abnormal'}]

# restrict to abnormal findings
findings = [f for f in findings_list if isinstance(f, dict) and f.get('category') == 'Abnormal']

required_cols = ['phrase', 'category', 'unique_metadata', 'original_sentence', 'patient_id']

df = process_findings(findings, clinical_note, emd_model, index, docs)
for col in required_cols:
    if col not in df.columns:
        df[col] = np.nan
df.head()

	phrase	category	unique_metadata	original_sentence	patient_id
0	flushing	Abnormal	[{'hp_id': 'HP:0031284', 'phrase': 'flushing',...	A 43-year-old Caucasian female experienced MCA...	None
1	rashes	Abnormal	[{'hp_id': 'HP:0000988', 'phrase': 'rash', 'de...	A 43-year-old Caucasian female experienced MCA...	None
2	itching	Abnormal	[{'hp_id': 'HP:0000989', 'phrase': 'itching', ...	A 43-year-old Caucasian female experienced MCA...	None
3	wheezing	Abnormal	[{'hp_id': 'HP:0030828', 'phrase': 'wheezing',...	A 43-year-old Caucasian female experienced MCA...	None
4	dizziness	Abnormal	[{'hp_id': 'HP:0002321', 'phrase': 'dizziness'...	A 43-year-old Caucasian female experienced MCA...	None

for idx, row in df.iterrows():
    phrase = row['phrase'].strip()
    normalized = phrase.lower().replace('-', ' ').strip()
    category = row['category']
    original = row['original_sentence']
    metadata_list = row['unique_metadata'] or []
    candidates = []
    seen = set()
    for m in metadata_list:
        term = m.get('phrase') or m.get('info') # term is a potential label such as 'exanthem'
        hp = m.get('hp_id')
        if term and hp and hp not in seen:
            candidates.append({'term': term, 'id': hp})
            seen.add(hp)
    candidate_ids = {c['id'] for c in candidates}
    payload = json.dumps({
            'phrase': normalized,
            'category': category,
            'original_sentence': original,
            'candidates': candidates
        })
    
    messages = [
        SystemMessage(content=system_message_II),
        HumanMessage(content=payload),
    ]
    if idx < 1: # print out the system/user messages once
            print("Example prompt")
            for msg in messages:
                msg.pretty_print()
            print("**************************\n\n")

    response = llm_client.invoke(messages) # A response such as {"hpo_id": "HP:0031284"}
    content_str = response.content
    data = json.loads(content_str)
    hpo_id = data["hpo_id"]
    # get the corresponding label
    term: hpotk.Term = hpo.get_term(hpo_id)
    if term:
        term_label = term.name
    else:
        term_label = "n/a"
    print("********")
    print(f"phrase='{phrase} => llm response={hpo_id}'-'{term_label}'")

Example prompt

================================ System Message ================================



You are an expert clinical phenotype mapper.  

Given one abnormal phenotype phrase and a list of candidate HPO terms, select exactly one best matching HPO term—and only from those candidates.  Do not invent, hallucinate, or suggest any term not in the provided list.  



When you receive input, it will be a JSON object:

{

  "phrase": "​…​",

  "category": "​…​",

  "original_sentence": "​…​",

  "candidates": [ {"term": "​…​", "id": "HP:…​"}, … ]

}



Decision steps:

1. Leverage context: Use the original_sentence to capture nuance.  

2. Match specificity: Pick the candidate whose term description most precisely matches the phenotype.  

3. If two are equally good, choose the one with the highest semantic similarity score.  

4. If none fits well (similarity < 80 or no clear match), return null.  



Output requirement:

Return exactly one JSON object, nothing else:



    {"hpo_id": "HP:0001234"}



or



    {"hpo_id": null"}



—no commentary, no extra keys, no surrounding text.

================================ Human Message =================================



{"phrase": "flushing", "category": "Abnormal", "original_sentence": "A 43-year-old Caucasian female experienced MCAS symptoms at age 18: specific foods and odours caused flushing, rashes, itching, wheezing, dizziness and nausea", "candidates": [{"term": "flushing", "id": "HP:0031284"}, {"term": "facial flushing after alcohol intake", "id": "HP:0001033"}, {"term": "seizure flurries", "id": "HP:0033349"}, {"term": "focal autonomic seizure with pallor/flushing", "id": "HP:0032762"}, {"term": "exanthem", "id": "HP:4000054"}, {"term": "blushing", "id": "HP:0001041"}, {"term": "focal aware autonomic seizure with pallor/flushing", "id": "HP:0032761"}, {"term": "hemoptysis", "id": "HP:0002105"}, {"term": "fasciculation", "id": "HP:0002380"}, {"term": "severe influenza", "id": "HP:0034249"}, {"term": "atrial flutter", "id": "HP:0004749"}, {"term": "facial puffiness", "id": "HP:0000282"}, {"term": "quotidian fever", "id": "HP:0033087"}, {"term": "flaking skin", "id": "HP:0040189"}, {"term": "sweating and flushing in the preauricular area in response to mastication", "id": "HP:6000637"}]}

**************************





********

phrase='flushing => llm response=HP:0031284'-'Flushing'

********

phrase='rashes => llm response=HP:0000988'-'Skin rash'

********

phrase='itching => llm response=HP:0000989'-'Pruritus'

********

phrase='wheezing => llm response=HP:0030828'-'Wheezing'

********

phrase='dizziness => llm response=HP:0002321'-'Vertigo'

********

phrase='nausea => llm response=HP:0002018'-'Nausea'

********

phrase='postprandial bloating/pain => llm response=HP:0033843'-'Postprandial fullness'

********

phrase='constipation => llm response=HP:0002019'-'Constipation'

********

phrase='evacuating stool => llm response=HP:0040183'-'Encopresis'

********

phrase='flatus with rotten egg odour => llm response=HP:0032142'-'Fetor hepaticus'

********

phrase='Restless legs syndrome (RLS) => llm response=HP:0012452'-'Restless legs'

********

phrase='orthostatic lightheadedness => llm response=HP:0012670'-'Orthostatic syncope'

********

phrase='tachycardia => llm response=HP:0001649'-'Tachycardia'

********

phrase='body pain => llm response=HP:0033047'-'Body ache'

********

phrase='generalised weakness => llm response=HP:0003324'-'Generalized muscle weakness'

********

phrase='painful dependent leg oedema => llm response=HP:0010741'-'Pedal edema'

********

phrase='fatigue => llm response=HP:0012378'-'Fatigue'

********

phrase='Faecal evacuation resulted in syncope => llm response=HP:0012668'-'Vasovagal syncope'

********

phrase='early satiety => llm response=HP:0033842'-'Early satiety'

********

phrase='Pressure-induced hives => llm response=HP:0410134'-'Physical urticaria'

********

phrase='angioedema => llm response=HP:0100665'-'Angioedema'

********

phrase='paraesthesia => llm response=HP:0003401'-'Paresthesia'

********

phrase='nocturnal urination (seven times nightly) => llm response=HP:0000017'-'Nocturia'

********

phrase='pulse of 80 beats/min increased to 160 after standing 10 min => llm response=HP:0001649'-'Tachycardia'

********

phrase='Facial rash => llm response=HP:0000988'-'Skin rash'

********

phrase='oedema => llm response=HP:0010741'-'Pedal edema'

********

phrase='cold, blue hands => llm response=HP:0001063'-'Acrocyanosis'

********

phrase='Terry’s fingernails => llm response=HP:0001812'-'Hyperconvex fingernails'

********

phrase='dermatographism => llm response=HP:0011971'-'Dermatographic urticaria'

Compare without RAG

system_message_no_RAG = "You are a highly specialized clinical information extraction system. Your task is to identify *all* phenotypic mentions in a clinical note and extract the corresponding Human Phenotype Ontology (HPO). Categorize them into exactly one of: Abnormal, Normal, Family History, Other, or Suspected.\n\n**Instructions:**\n1. **Scan Every Sentence**: Extract every phrase that describes a phenotype—signs, symptoms, abnormal findings, test results, anatomical observations—or any diagnostic conjecture or history statement.\n2. **Assign One Category**:\n   • **Abnormal**: Any *observed* finding deviating from healthy (disease signs, lab abnormalities, pathological conditions). *These will be sent on for HPO mapping.*\n   • **Normal**: Explicit statements of normalcy.\n   • **Family History**: Phenotypes in relatives.\n   • **Other**: Personal medical history, risk factors, demographics, or any non‐finding statement (e.g., “history of HIV,” “smoker,” “height 5′10″”).\n   • **Suspected**: Any mention of a possible or rule‐out diagnosis (e.g., “possible sciatica,” “rule out PE”).\n3. **Preserve Specificity**: Extract the phrasing exactly as written.\n4. **Output JSON Only**: Return `{ \"phenotypes\": [ { \"phrase\": ..., \"category\": ... , \"HPO id\": ..., \"HPO label\": ...,}, … ] }`.\n\n**Post‐processing:** Immediately drop anything not labeled Abnormal before HPO mapping."

messages = [
    SystemMessage(content=system_message_no_RAG),
    HumanMessage(content=clinical_note),
]

response = llm_client.invoke(messages)
print(response.content)

```json
{
  "phenotypes": [
    {
      "phrase": "flushing",
      "category": "Abnormal",
      "HPO id": "HP:0000981",
      "HPO label": "Skin flushing"
    },
    {
      "phrase": "rashes",
      "category": "Abnormal",
      "HPO id": "HP:0000981",
      "HPO label": "Skin rash"
    },
    {
      "phrase": "itching",
      "category": "Abnormal",
      "HPO id": "HP:0000981",
      "HPO label": "Itching"
    },
    {
      "phrase": "wheezing",
      "category": "Abnormal",
      "HPO id": "HP:0002090",
      "HPO label": "Wheezing"
    },
    {
      "phrase": "dizziness",
      "category": "Abnormal",
      "HPO id": "HP:0000403",
      "HPO label": "Dizziness"
    },
    {
      "phrase": "nausea",
      "category": "Abnormal",
      "HPO id": "HP:0000311",
      "HPO label": "Nausea"
    },
    {
      "phrase": "postprandial bloating/pain",
      "category": "Abnormal",
      "HPO id": "HP:0002047",
      "HPO label": "Abdominal bloating"
    },
    {
      "phrase": "constipation",
      "category": "Abnormal",
      "HPO id": "HP:0000705",
      "HPO label": "Constipation"
    },
    {
      "phrase": "flatus with rotten egg odour",
      "category": "Abnormal",
      "HPO id": "HP:0033478",
      "HPO label": "Foul-smelling flatulence"
    },
    {
      "phrase": "Restless legs syndrome (RLS)",
      "category": "Abnormal",
      "HPO id": "HP:0000765",
      "HPO label": "Restless legs syndrome"
    },
    {
      "phrase": "orthostatic lightheadedness",
      "category": "Abnormal",
      "HPO id": "HP:0001742",
      "HPO label": "Orthostatic lightheadedness"
    },
    {
      "phrase": "tachycardia",
      "category": "Abnormal",
      "HPO id": "HP:0001945",
      "HPO label": "Tachycardia"
    },
    {
      "phrase": "body pain",
      "category": "Abnormal",
      "HPO id": "HP:0001924",
      "HPO label": "Body pain"
    },
    {
      "phrase": "generalised weakness",
      "category": "Abnormal",
      "HPO id": "HP:0001324",
      "HPO label": "Generalized weakness"
    },
    {
      "phrase": "painful dependent leg oedema",
      "category": "Abnormal",
      "HPO id": "HP:0000898",
      "HPO label": "Dependent edema"
    },
    {
      "phrase": "syncope with efforts >3 min",
      "category": "Abnormal",
      "HPO id": "HP:0000308",
      "HPO label": "Syncope"
    },
    {
      "phrase": "early satiety",
      "category": "Abnormal",
      "HPO id": "HP:0001999",
      "HPO label": "Early satiety"
    },
    {
      "phrase": "Pressure-induced hives",
      "category": "Abnormal",
      "HPO id": "HP:0000981",
      "HPO label": "Skin rash"
    },
    {
      "phrase": "angioedema",
      "category": "Abnormal",
      "HPO id": "HP:0000891",
      "HPO label": "Angioedema"
    },
    {
      "phrase": "paraesthesia",
      "category": "Abnormal",
      "HPO id": "HP:0001017",
      "HPO label": "Paraesthesia"
    },
    {
      "phrase": "nocturnal urination (seven times nightly)",
      "category": "Abnormal",
      "HPO id": "HP:0001629",
      "HPO label": "Nocturia"
    },
    {
      "phrase": "supine pulse of 80 beats/min increased to 160 after standing 10 min",
      "category": "Abnormal",
      "HPO id": "HP:0001945",
      "HPO label": "Tachycardia"
    },
    {
      "phrase": "facial rash",
      "category": "Abnormal",
      "HPO id": "HP:0000981",
      "HPO label": "Skin rash"
    },
    {
      "phrase": "facial oedema",
      "category": "Abnormal",
      "HPO id": "HP:0000891",
      "HPO label": "Angioedema"
    },
    {
      "phrase": "cold, blue hands",
      "category": "Abnormal",
      "HPO id": "HP:0000973",
      "HPO label": "Cyanosis"
    },
    {
      "phrase": "Terry’s fingernails",
      "category": "Abnormal",
      "HPO id": "HP:0000323",
      "HPO label": "Nail discoloration"
    },
    {
      "phrase": "dermatographism",
      "category": "Abnormal",
      "HPO id": "HP:0000979",
      "HPO label": "Dermatographism"
    }
  ]
}
```