trulens-dataset-curation

TruLens Dataset Curation

Create evaluation datasets with ground truth to measure your LLM app's performance.

Overview

Ground truth datasets allow you to:

• Compare LLM outputs against expected responses
• Evaluate retrieval quality against expected chunks
• Track performance across app versions
• Share evaluation data across your team

Prerequisites

pip install trulens pandas

Instructions

Step 1: Initialize TruSession

from trulens.core import TruSession

session = TruSession()

Step 2: Create Ground Truth Data

Structure your data as a pandas DataFrame with these columns:

Column	Required	Description
query	Yes	The input query/question
query_id	No	Unique identifier for the query
expected_response	No	The expected/ideal response
expected_chunks	No	Expected retrieved contexts (list or string)

import pandas as pd

data = {
    "query": [
        "What is TruLens?",
        "How do I instrument a LangChain app?",
        "What is the RAG triad?",
    ],
    "query_id": ["q1", "q2", "q3"],
    "expected_response": [
        "TruLens is an open source library for evaluating and tracing AI agents.",
        "Use TruChain to wrap your LangChain app for automatic instrumentation.",
        "The RAG triad consists of context relevance, groundedness, and answer relevance.",
    ],
    "expected_chunks": [
        [
            "TruLens is an open source library for evaluating and tracing AI agents, including RAG systems."
        ],
        [
            "from trulens.apps.langchain import TruChain",
            "tru_recorder = TruChain(chain, app_name='MyApp')",
        ],
        [
            "Context relevance evaluates retrieved chunks",
            "Groundedness checks if response is supported by context",
            "Answer relevance measures if the response answers the question",
        ],
    ],
}

ground_truth_df = pd.DataFrame(data)