ml-best-practices

ML Best Practices

I want to read a story about the data, not just run code. Ensure every code cell is followed by a markdown cell analyzing the results. End the notebook with a summary comprehensively answering the prompt.

If there is a good match between the user's request and a corresponding example plan, then adapt the example plan to fully answer the user's request:

Clustering:

Identify distinct groups based on their features.

• Understand the schema and field descriptions.
• Visualize features referenced in the prompt (e.g., with histograms, scatterplots).
• Transform dates into timestamps.
• Before applying encoders, check if the dataset already contains pre-encoded features and prefer existing numerical representations.
• Prefer to keep data instead of dropping it when possible.
• Transform ordinal data with an ordinal encoder.
• Transform nominal data with a one hot encoder.
• Standardize numerical features.
• Perform clustering with a range of values, and collect the silhouette score.
• Choose the optimal number of clusters based on the silhouette score.
• Use dimensionality reduction (e.g., PCA) to project the data into two dimensions.
• Scatterplot the samples in two dimensions with cluster labels as the hue.
• Scatterplot the samples in two dimensions with a discrete feature as the hue.
• Describe the clusters in text by feature distributions or typical feature values.
• Conclusion: comprehensively answer the prompt in a final markdown cell.

Time Series Forecasting:

Develop a predictive model to estimate future values based on historical trends. How might different modeling approaches impact the prediction accuracy?

• Understand the schema and field descriptions.
• Visualize the target feature over time at a reasonable granularity.
• Always perform a chronological split on the data to create training, validation, and test sets.
• Are there seasonal trends?
• Test for stationarity.
• Discuss possible modeling approaches. How might different modeling approaches impact the prediction accuracy?
• Train two time series forecasting models to predict the target feature. Use previous seasonality and stationarity information as model hyperparameters.
• Predict the target feature for the training and validation sets.
• Optionally, hypertune models with the validation set.
• Visualize the actual and predicted target feature vs time for each model on the training and validation sets.
• Evaluate the validation performance with error metrics.
• Select a model.
• Retrain the selected model on the test and validation sets.
• Predict the test values with the selected model.
• Visualize the average target feature and the predicted test values.
• Conclusion: comprehensively answer the prompt in a final markdown cell.