How to Build a Machine Learning Pipeline

Step 1: Define the Problem and Success Criteria

Before writing any code, specify three things. What are you predicting? A clear target variable (churn yes/no, revenue next quarter, defect probability) anchors every subsequent decision. What data is available at prediction time? Features that exist in your historical dataset but will not be available for future predictions are useless and dangerous (data leakage). What metric determines success? A spam filter might need 99% precision. A medical screening might need 95% recall. The metric drives model selection, threshold tuning, and the final go/no-go decision.

Document these decisions. They are the contract between the ML system and the business. When stakeholders later ask "why does the model do X," the answer traces back to the problem definition and success criteria.

Step 2: Build the Data Ingestion and Cleaning Stage

The first pipeline stage loads raw data and validates its structure. Check that expected columns exist, data types are correct, and no column has an unexpected percentage of missing values. Fail loudly if the data does not match expectations rather than silently producing corrupt predictions.

Missing value handling should be explicit and consistent. Common strategies: drop rows with missing targets (you cannot train on unlabeled data), impute numerical features with the median (robust to outliers), impute categorical features with the mode or a dedicated "missing" category, and flag missingness itself as a binary feature (sometimes the fact that a value is missing is predictive).

Outlier handling depends on the domain. For financial data, extreme values are often genuine (a $10 million transaction is not an error, it is a whale). For sensor data, extreme values are often measurement errors. Winsorizing (capping values at a percentile) is a pragmatic middle ground that limits outlier influence without discarding data.

In scikit-learn, use ColumnTransformer to apply different cleaning strategies to different column types, and wrap everything in a Pipeline object so cleaning steps are automatically applied to both training and prediction data.

Step 3: Create the Feature Transformation Stage

Feature transformations convert cleaned data into the format the model needs. This stage typically includes: encoding categorical variables (one-hot, ordinal, or target encoding), scaling numerical features (standardization or normalization), creating derived features (ratios, date components, interactions), and optionally reducing dimensionality (PCA or feature selection).

The critical rule: fit transformers on training data only, then apply them to both training and new data. If you fit a StandardScaler on the full dataset before splitting, the test data's statistics leak into the training process. Scikit-learn's Pipeline handles this automatically when you call fit on training data and transform or predict on new data.

For complex feature engineering that depends on the target variable (like target encoding), use cross-validated approaches within the pipeline to prevent leakage. Libraries like category_encoders provide target encoders with built-in cross-validation support.

Step 4: Add Model Training and Evaluation

The model is a single component within the larger pipeline. In scikit-learn, it is the last step in the Pipeline object. When you call pipeline.fit(X_train, y_train), the cleaning, transformation, and model fitting all execute in sequence. When you call pipeline.predict(X_new), the same cleaning and transformation apply to new data before the model makes predictions.

For hyperparameter tuning, wrap the pipeline in GridSearchCV or RandomizedSearchCV. You can tune any parameter in any pipeline step using the step__parameter naming convention. This searches across combinations of, say, PCA components and random forest trees simultaneously.

Evaluate using cross-validation on the full pipeline, not just the model. This ensures that preprocessing is properly separated between folds and the evaluation is honest. Report cross-validated metrics with standard deviations.

Step 5: Deploy and Monitor

Serialize the trained pipeline (using joblib or pickle) so it can be loaded and used for prediction without retraining. The serialized pipeline includes all preprocessing steps and the trained model, so prediction on new data requires only pipeline.predict(new_data).

Serve predictions through a REST API (using Flask, FastAPI, or a managed service like AWS SageMaker). The API receives raw input data, runs it through the pipeline, and returns predictions. Validate input data at the API boundary to catch malformed requests before they reach the model.

Monitor two things in production. Data drift: are the distributions of incoming features changing compared to training data? A fraud model trained on 2024 transaction patterns may see different patterns in 2026. Performance drift: are predictions still accurate? If you can obtain ground truth labels (delayed by days or weeks), compute ongoing metrics and alert when they degrade. Schedule periodic retraining when drift is detected.