data-analysis

Data Analysis

Evidence and provenance contract

Analysis must remain traceable to the data actually observed.

• Separate observed source data, user-provided labels, external research, and estimates or modeled assumptions in both the notebook and final answer.
• For material findings, preserve the source connection/file, table or sheet, query or transformation, coverage window, and relevant row counts. If a query fails or only aggregate data is available, narrow the claim accordingly.
• Never invent missing rows, prompts, campaigns, categories, URLs, fields, citations, model versions, or provenance. Missing data stays missing.
• Never present simulated, modeled, cached, delayed, fallback, or sample data as live production data. Label its mode and freshness where the user can see it.
• Reconcile headline totals against the displayed tables before reporting completion. If they disagree, stop and explain the discrepancy instead of choosing the more convenient number.
• User corrections replace prior assumptions. Re-run affected calculations and update every downstream artifact that depended on the old assumption.
• Honor requested implementation constraints such as Python-only, no JavaScript, or reuse-only in deliverable code. js_exec may still orchestrate platform tools, but it does not justify adding JavaScript to a Python-only deliverable.

Python Environment (DO-backed projects)

For DO-backed projects, data analysis runs in a stateless per-workspace sandbox over the project filesystem — there is no persistent VM to set up. The default data stack is preinstalled, so most analysis needs no environment step at all:

pandas, numpy, polars, duckdb, pyarrow, altair, plotly, matplotlib, seaborn, scipy, scikit-learn, statsmodels, openpyxl, xlsxwriter, pdfplumber, python-docx, python-pptx, sqlalchemy (+ psycopg/pymysql), jupyter/nbconvert.

• Need a package beyond the stack? add_python_dependency({ project, packages: ["<pkg>"] }) — it runs uv add and persists pyproject.toml + uv.lock back to the project. (You can also just edit pyproject.toml; the next run's uv sync reconciles it.)
• Run a notebook: run_notebook (see below) — execution + validation in one call.
• Ad-hoc shell/Python: analysis_exec({ command, project? }). The camelai helper package (see the connections section) is importable here and in run_code too, not just in notebooks.