pandas

pandas Best Practices

Pandas is the backbone of data analysis in Python. Adhere to these guidelines to write efficient, readable, and scalable code that integrates seamlessly into modern AI/ML pipelines.

1. Code Organization & Structure

1.1 Standard Imports

Always import pandas with its conventional alias for consistency.

❌ BAD

import pandas
from pandas import DataFrame

✅ GOOD

import pandas as pd
import numpy as np # For numerical operations

1.2 Method Chaining for Readability

Chain operations to create clear, sequential data transformations. Use .pipe() for custom functions or when intermediate steps improve clarity.

❌ BAD

df_filtered = df[df['value'] > 10]
df_grouped = df_filtered.groupby('category')
df_result = df_grouped['metric'].mean().reset_index()

✅ GOOD

df_result = (
    df[df['value'] > 10]
    .groupby('category')['metric']
    .mean()
    .reset_index()
    .rename(columns={'metric': 'avg_metric'}) # Add a descriptive rename
)

2. Common Patterns & Anti-patterns

2.1 Avoid Python Loops – Embrace Vectorization

Iterating over DataFrames row-by-row (.iterrows(), .apply(axis=1)) is a major performance bottleneck. Pandas operations are optimized C-level functions.

❌ BAD (Slow for large DataFrames)

# Calculating a new column based on existing ones
for index, row in df.iterrows():
    df.loc[index, 'new_col'] = row['col_a'] + row['col_b']

# Complex conditional logic with .apply(axis=1)
def calculate_status(row):
    if row['score'] > 90 and row['grade'] == 'A':
        return 'Excellent'
    return 'Good'
df['status'] = df.apply(calculate_status, axis=1)

✅ GOOD (Vectorized and performant)

# Calculating a new column
df['new_col'] = df['col_a'] + df['col_b']

# Complex conditional logic using np.select or boolean indexing
conditions = [
    (df['score'] > 90) & (df['grade'] == 'A'),
    (df['score'] > 80) & (df['grade'] == 'B')
]
choices = ['Excellent', 'Very Good']
df['status'] = np.select(conditions, choices, default='Good')