qcaleval

Calibration plot evaluation

qcal_evaluate sends a plot to a calibration-evaluation model (NVIDIA Ising-Calibration by default) and asks it the six QCalEval questions, each in its own request. The diagnosis it returns has one entry per question: technical_description, experimental_conclusion (outcome: expected / suboptimal / anomalous / apparatus), experimental_significance, fit_reliability (reliable / unreliable / no_fit), parameter_extraction, and calibration_diagnosis (a status code, a suggested range, notes), plus success (the status is SUCCESS).

Preferred tools

• qcal_evaluate: judge one or more calibration plots. Give context for what the background cannot know: the task name, the target (qubit or coupling), the swept range. Do not put fitted values in it; the model reads the plot. Add question when a specific point matters (for example "is the chevron centred on the current frequency?"); it is added to the significance question.
• Pass task_name whenever you know the task (for example CheckRabi): the host then supplies the experiment background and the task's criteria (the parameters to extract, the status codes), which is more reliable than a summary you write yourself.
• When the conversation holds an expected reference figure followed by the measured result (a task review opens this way), pass max_images: 2 and say in context that the first figure is the reference and the last the measurement. The host then supplies both; the default is the newest figure only.
• Pass questions to ask a subset, for example ["calibration_diagnosis"] when only the status matters.
• Pass model (provider/modelId) only when the user names one of the configured evaluation models; otherwise the configured calibration VLM judges.
• Pass dry_run: true to see the prompts the model would get (per question) without asking it, for example when the user wants to check a task's reference before judging.