data-analysis

Data Analysis Skill

Process data files in the data/ directory, perform analysis, and output reports to reports/.

Step-by-Step Workflow

1. Identify the data source — List available files and confirm with the user which to analyze:

ls data/

1. Load and inspect the data — Use Python to read the file and show a summary:

import pandas as pd

df = pd.read_csv("data/sales.csv")  # or read_excel / read_json
print(f"Shape: {df.shape}")
print(f"Columns: {list(df.columns)}")
print(df.dtypes)
print(df.describe())
print(f"Missing values:\n{df.isnull().sum()}")

1. Clean the data — Handle missing values, fix types, remove duplicates:

df = df.drop_duplicates()
df["date"] = pd.to_datetime(df["date"], errors="coerce")
df["amount"] = pd.to_numeric(df["amount"], errors="coerce")
df = df.dropna(subset=["date", "amount"])
print(f"Clean shape: {df.shape}")

1. Analyze — Compute the requested statistics or aggregations:

# Example: monthly revenue trend
monthly = df.groupby(df["date"].dt.to_period("M"))["amount"].sum()
print(monthly)

# Example: correlation matrix
print(df[["amount", "quantity", "discount"]].corr())

1. Visualize — Generate charts and save to reports/:

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt

monthly.plot(kind="bar", title="Monthly Revenue")
plt.tight_layout()
plt.savefig("reports/monthly_revenue.png", dpi=150)
plt.close()
print("Chart saved to reports/monthly_revenue.png")

1. Write the report — Save a Markdown report to reports/:

with open("reports/analysis_report.md", "w") as f:
    f.write("# Analysis Report\n\n")
    f.write("## Summary\n")
    f.write(f"- Total records: {len(df)}\n")
    f.write(f"- Date range: {df['date'].min()} to {df['date'].max()}\n")
    f.write(f"- Total revenue: {df['amount'].sum():,.2f}\n\n")
    f.write("## Charts\n")
    f.write("![Monthly Revenue](monthly_revenue.png)\n")
print("Report saved to reports/analysis_report.md")