optimize-python-parallelism

AtomLane Python Advisor

Improve a program only after separating three questions: where time is spent, whether iterations are semantically independent, and whether the proposed executor is likely to beat its overhead. A long runtime is a reason to inspect, not proof that parallel execution is legal or useful.

Keep analysis and execution separate

Use python_parallel_advisor for bounded source analysis. It reads strict project-local UTF-8 Python, builds a conservative same-module call/effect summary, and may return a source-hash-bound rewrite preview. It never imports or executes target code and never changes files.

Read references/python-program-ir.md before interpreting or applying a candidate. It defines classification, proof gates, GIL/spawn constraints, rewrite validity, and the verification certificate.

Call the advisor with:

• an absolute project_path;
• concrete paths when the entrypoint is known, otherwise bounded discovery;
• caller-observed hotspots only when they are real serial measurements;
• the actual execution_context, so an inner pool is not multiplied by an AtomLane or native worker pool;
• an explicit worker ceiling only as a ceiling, never as a safety override.
• target_platform when the optimized program will deploy somewhere other than the analysis host.

Do not run a workload merely to obtain a profile when repeating it may mutate state, incur cost, or affect an external system.

Treat each classification precisely

• reviewable_rewrite is the strongest static result, not a runtime proof. Its patch is still conditional on pickling, import, memory, correctness, and measured-performance checks.
• advisory_only identifies a plausible I/O, network, subprocess, or otherwise conditional opportunity. Explain the missing guarantees; do not apply its outline as an automatic transformation.
• prefer_native means vectorization or a library-owned worker pool should be considered before another Python pool.
• already_parallel requires one coordinated outer/inner resource budget.
• blocked remains serial until every hard blocker is removed by evidence or a semantics-preserving redesign. A confidence score cannot override a blocker.