batch-processing-clinical-text

Batch processing clinical text

openmed.processing runs OpenMed over many documents efficiently, with progress tracking, per-item error isolation, and streaming. It runs fully on-device: the corpus, the model, and the output never leave the host. This skill shows a resumable runner — sharded, checkpointed, append-only JSONL — that you can restart without reprocessing.

When to use this skill

For corpora, folders, or datasets — anything beyond a handful of notes. For a single note, just call openmed.analyze_text / deidentify directly (extracting-clinical-entities, deidentifying-clinical-text). For an always-on HTTP service, see serving-openmed-rest-api.

Quick start

from openmed import process_batch

texts = ["Patient has type 2 diabetes.", "No acute distress. BP 120/80."]
result = process_batch(texts, model_name="disease_detection_superclinical")

print(result.summary())          # PHI-safe counts + timing
print(result.successful_items, "/", result.total_items)
for item in result.get_successful_results():
    print(item.id, item.result.to_dict()["entities"])   # spans only; avoid raw text in logs

process_batch(...) is a thin wrapper over BatchProcessor. Real signatures (openmed/processing/batch.py):

• process_batch(texts, model_name="disease_detection_superclinical", ids=None, config=None, progress_callback=None, on_progress=None, **kwargs) -> BatchResult
• BatchProcessor(model_name=..., operation="analyze_text", batch_size=8, continue_on_error=True, **analyze_kwargs) with operation ∈ {"analyze_text", "extract_pii", "deidentify"}.
• BatchItem(id, text, source=None, metadata=None)
• BatchResult — .items, .total_items, .successful_items, .failed_items, .success_rate, .average_processing_time, .summary(), .to_dict(), .get_successful_results(), .get_failed_results().
• BatchItemResult — .id, .result (a PredictionResult/DeidentificationResult), .error, .processing_time, .source, .success, .to_dict().