Executive Summary
Statistical demand forecasting breaks down at SKU scale because most real product catalogs are dominated by intermittent demand. Irregular purchase patterns, extended zero-sales periods, and promotional spikes sit outside what ARIMA (Autoregressive Integrated Moving Average), the statistical method that has anchored forecasting for decades, was designed to handle. Each SKU is modelled independently, with no cross-catalog learning and no integration of external signals.
This case study documents how an XGBoost-LSTM model hybrid pipeline replaced an ARIMA model across a 12,000-SKU distribution operation. Before training began, the historical record was corrected for stockout bias using an Expectation-Maximization algorithm, removing periods where zero sales reflected stock unavailability rather than true low demand. The pipeline also integrated promotional calendars, weather data, and regional event signals that the statistical baseline could not use.
The result: MAPE (Mean Absolute Percentage Error) fell from 21.4% to 15.0% across the full SKU catalog, a 30% reduction against the ARIMA baseline. Safety stock decreased without increasing stockout frequency. Accuracy gains were largest on intermittent demand SKUs, the category where statistical forecasting performs worst and forecast errors carry the most operational cost.
The Problem
The Operating Context
The organisation managed a regional distribution network with over 12,000 active SKUs across multiple warehouse locations, supplying retail and wholesale channels. Demand forecasting ran on an ARIMA model that had performed adequately when the catalog was smaller and demand patterns more predictable. As the assortment grew, promotional activity increased, and new product launches accelerated, forecast accuracy deteriorated.
The deterioration was not uniform. High-volume stable SKUs, a small fraction of the catalog, still forecast reasonably well. The majority of the catalog, dominated by intermittent demand, was the problem. Forecast errors on intermittent SKUs were driving excess safety stock on slow movers and stockouts on fast-moving items in the same reporting period.
Demand planners spent the majority of their time manually adjusting statistical outputs rather than making strategic decisions. The ARIMA model produced a forecast for every SKU, but planners trusted it for only a fraction of them, overriding the rest based on judgment and experience. The process was not scalable as the catalog continued to grow.
Why Statistical Forecasting Was Failing
ARIMA model projects historical sales patterns forward, an approach that works reliably when demand is smooth and predictable. At the SKU level in a real product catalog, none of those conditions hold. Most SKUs sell in irregular bursts, sit at zero for weeks at a time, and respond to promotions, competitor actions, and external signals in ways statistical models cannot capture.
ARIMA model treats these zero-sales periods as low-demand signals. They are not. They are data from a different distribution.
The problem is not just accuracy. It is architecture. Under a statistical approach, 12,000 SKUs require 12,000 individually tuned models, each operating without knowledge of what the others reveal about demand dynamics across the catalog.
Machine learning changes this, and the M5 Forecasting Competition (2020) established the empirical case. The competition ran on Walmart’s sales data across 42,840 product time series; every top-50 submission used machine learning or hybrid approaches. Not a single pure statistical model placed in the top tier. Full results are available via the M5 accuracy competition paper.
Why This Was Hard
Three structural problems had to be resolved before any model could train on the data. All three are pervasive in real supply chain environments. Left unaddressed, they cause the model to learn the wrong relationship between inputs and demand.
Stockout Bias
When a product is out of stock, the sales record shows zero. Zero sales does not mean zero demand: it means demand existed but could not be filled.
A model trained on uncorrected data learns that demand was low during the stockout period, and every forecast it generates for that product carries that error forward. Across documented supply chain implementations, uncorrected stockout bias inflates apparent MAPE by 8 to 15 percentage points depending on how frequently SKUs go out of stock.
The correction used an Expectation-Maximization (EM) algorithm. During identified stockout periods, EM estimates what demand would have been based on pre-stockout and post-stockout sales velocity and on signals from comparable SKUs in the same category. The corrected signal replaces the zero-sales record before any model training begins.
Cold Start
SKUs with fewer than 12 weeks of sales history do not have enough data for a time-series model to identify patterns. In a growing catalog, this affects a material share of active SKUs at any given time.
The pipeline addressed cold start through cross-SKU transfer features: new products inherited the demand signal of the closest established SKU in their category while accumulating their own history. This gave the model a calibrated starting estimate rather than a flat prior or a forced exclusion from the forecast run.
Data Fragmentation
Historical sales data was distributed across multiple legacy systems. Promotional calendars were maintained separately from the core ERP with no automated link to the forecasting pipeline.
External signal data, including weather, regional events, and public holidays, required third-party sourcing and significant data engineering to align to SKU-level records before training could begin. The messiness of the data, not the model architecture, was the first real technical challenge.
Approach and Methodology
The Data Foundation
The pipeline draws from three categories of input. Transaction-level sales data by SKU, location, and date provides the baseline demand signal: what sold, when, and where, but not why.
Catalog attributes, including price tier, shelf life, and substitution relationships, allow the model to learn demand structure across the catalog. These attributes are particularly important for intermittent SKUs where transaction history alone carries limited signal.
External signals such as promotional calendars, local weather data, and regional events drive the spikes and dips that historical sales alone cannot anticipate. ARIMA uses only the first category. The pipeline uses all three, and the accuracy gap widens with every external signal added.
Model Evaluation and the Iteration Story
Four architectures were evaluated across the same dataset before the production model was selected. The table below summarises what each offered and where it fell short.
The first model in production was not the final model. XGBoost (eXtreme Gradient Boosting, a tree-based machine learning algorithm) handled promotional lift and external signal effects well, outperforming the ARIMA model on spike events, but failed on long-horizon sequential patterns where the ordering of past observations carries predictive information that tree models cannot capture structurally.
Adding LSTM (Long Short-Term Memory, a recurrent neural network) improved seasonal accuracy. Still, it introduced a different weakness: less responsiveness to discrete external event signals, where the non-linear input-demand relationship requires the feature engineering that tree models handle better. Neither architecture resolved both gaps simultaneously.
The Temporal Fusion Transformer (TFT) resolved both gaps, achieving 6.32% MAPE on the validation set, the most accurate single architecture evaluated. It was excluded from production on computational grounds: weekly retraining across 12,000 SKUs exceeded available infrastructure. The XGBoost-LSTM hybrid became the production answer, recovering most of TFT’s accuracy within infrastructure constraints and giving planners interpretable feature importance scores for each forecast.
Technical Deep Dive
Pipeline Architecture
The pipeline runs in five stages. In stage one, raw sales data is ingested and cleaned: duplicate records are removed, date gaps identified and filled, and quality flags raised for review. In stage two, stockout periods are identified from inventory records and corrected using EM; cold start treatment is applied to new SKUs.
In stage three, features are engineered from all three data categories into a set that typically runs to several hundred variables per SKU. In stage four, the XGBoost-LSTM hybrid trains on this feature set, with an ensemble layer weighting each model’s contribution by SKU category and demand pattern type. In stage five, the model outputs point forecasts and confidence intervals across the forecast horizon, ranked by confidence tier.
Feature Engineering
Feature engineering determines how much of the available signal the model can actually use. Lag features at one-week, four-week, and 52-week intervals give the model a simultaneous view of recent trend, medium-term velocity, and year-on-year seasonality. Rolling statistics over four, eight, and 12-week windows capture demand momentum and volatility that point-in-time lags miss.
Promotional features encode the presence, depth, and type of promotion active on each SKU on each forecast date. In most catalogs, they rank as the highest-importance feature group in model output. External signals, including weather, public holiday flags, and regional event calendars, are aligned to the SKU date and location; in weather-sensitive categories, they rank among the top predictors.
Trade-offs Made Explicit
The primary trade-off was accuracy versus compute cost. TFT delivered 6.32% MAPE on a higher-volume stable SKU partition but required infrastructure not available for weekly retraining at catalog scale. The hybrid’s 15.0% full-catalog MAPE reflects a harder evaluation, including all intermittent and cold-start SKUs; on the same partition, the gap between the two was narrow.
The secondary trade-off was interpretability versus model complexity. XGBoost’s tree structure provides feature importance outputs at the SKU level; a pure LSTM or TFT architecture does not. Planner adoption depended significantly on explainability: the ability to show why the model produced a given forecast for a given SKU. That made interpretability an operational requirement, not just a preference.
Results: 30% Lower Forecast Error Across the Full SKU Catalog
The pipeline was evaluated on a held-out test set covering 12 weeks of sales data unseen during training. Three metrics were tracked: MAPE, weighted MAPE penalising errors on high-volume SKUs, and stockout frequency. The ARIMA baseline ran at 21.4% MAPE across the full SKU catalog. The XGBoost-LSTM hybrid delivered 15.0%, a 30% reduction on the same evaluation. All three metrics improved.
This 30% reduction was measured across the full SKU catalog including all intermittent demand categories. Accuracy gains were largest on intermittent demand SKUs, where the ARIMA model performs worst. On those SKUs, forecast errors carry the highest operational cost: excess safety stock on slow movers and stockout events on fast-moving items.
Notably, on high-volume stable SKUs where demand is smooth and seasonal patterns are predictable, the improvement over the ARIMA model was smaller. Statistical methods are adequate when the assumptions they depend on actually hold. The AI pipeline adds the most value precisely where those assumptions break down.
The accuracy improvements translated directly into inventory outcomes: safety stock levels decreased across the catalog as a direct consequence of tighter confidence intervals. When the forecast is more accurate, less buffer inventory is needed to cover uncertainty. The reduction freed working capital without increasing stockout frequency, meeting the core business objective.
Operational Impact
What Changed for Planners and the Replenishment System
The pipeline does not replace the replenishment system or the demand planner. It replaces the statistical baseline that the replenishment system previously used as its input, and it changes what the demand planner spends their time on.
The model handles high-volume stable SKUs automatically: confidence is high, replenishment runs without review, and no planner attention is required. For lower-confidence SKUs, including intermittent, high-variance, and cold-start categories, forecasts are surfaced for human oversight. The planner moves from manually adjusting every ARIMA model output to reviewing only the cases where judgment adds value.
Underpinning that workflow is a weekly retraining cycle. New sales data is ingested, the stockout correction is reapplied, and the model updates across the full SKU catalog. The system adapts to demand shifts, new promotions, and seasonal transitions on a one-week cadence rather than requiring a manual recalibration run.
Scalability and Integration
Integration with existing ERP and replenishment platforms is handled at the data layer. The pipeline reads from existing records and outputs forecasts in a format the replenishment system can consume directly, without requiring changes to downstream logic. This makes the architecture replicable across any operation with transaction-level sales data, inventory records, and a promotional calendar.
The operational model scales as the catalog grows: high-confidence SKUs run automatically, low-confidence cases surface for human review. New SKUs enter through the cold-start treatment and accumulate history without requiring model retraining. Catalog growth does not translate into proportional increases in planner workload.
Lessons Learned
Correct the Data Before You Benchmark
The most consequential early mistake was benchmarking on uncorrected historical data. The gaps between ARIMA model and early ML models appeared smaller than they were, slowing the decision to invest in the full pipeline. Applying the EM stockout correction revealed why: zero-sales records were making the ARIMA model look better than it was against true demand. Correct the data first, then benchmark. The sequence matters.
Feature Engineering Matters More Than Architecture
The team spent significant time comparing model architectures. In retrospect, that effort would have been better directed at signal quality: getting promotional calendars complete, aligning weather data to the right geographies, and validating lag feature construction.
Architecture choice contributed at the margin. Feature engineering contributed fundamentally. The hybrid outperformed standalone LSTM not because of the ensemble design, but because the tree-based component could use the feature set more effectively.
Explainability Is an Operational Requirement
A 30% MAPE improvement is measurable. Planner trust is not. Planners needed to understand why the model produced each forecast before they would rely on it for replenishment decisions.
XGBoost’s feature importance output was the enabling factor: it showed which features drove each forecast. Without it, trust-building would have taken significantly longer. For any deployment where human review is part of the workflow, interpretability is not optional.
Cold Start Is Harder in Practice Than in Theory
Cross-SKU transfer features handle cold start adequately but not perfectly: accuracy depends on how closely the matched SKU reflects the new product’s demand profile. The closer the match, the better the starting estimate.
For new products in genuinely new categories where no close match exists, early forecasts can be biased until the new SKU accumulates its own history. New product launches into novel categories remain the hardest forecasting problem and the area where planner override is most frequently needed.
What’s Next
The immediate roadmap has three priorities. First, TFT deployment when compute infrastructure allows: the accuracy gap is measurable on the most complex SKU categories and worth closing as the operation scales. Based on the architecture evaluation results, the gap on comparable SKU partitions falls within two percentage points.
Second, real-time promotional signal integration: the current pipeline ingests promotional calendars weekly, and moving to a live feed would reduce the lag between a promotion being confirmed and the model incorporating it. The greatest accuracy gains would come from short-notice promotions, which are currently the hardest category to forecast accurately.
Third, extending the forecasting scope to supplier lead time. Demand uncertainty is one half of the inventory problem; lead time variability is the other. Addressing both with the same ML infrastructure, on the same weekly retraining cadence, is the natural next step.
One open problem is worth noting: the model does not anticipate events with no historical precedent. Supply disruptions, sudden competitor exits, or demand spikes driven by external news require human override and fast retraining on newly observed outcomes.
Improving the system’s ability to flag these cases proactively, rather than surfacing them only after the forecast error appears, is the most valuable unsolved problem. It is also the area where human judgment and model output will need to work most closely together.
About the Project
This case study is part of Omdena’s industry research series on AI for supply chain operations, synthesised from published studies, benchmarks, and documented real-world deployments. The pipeline architecture, evaluation approach, and results reflect patterns consistently documented across supply chain AI implementations in the published literature.
Omdena builds AI pipelines for supply chain applications including demand forecasting, supplier risk prediction, and inventory optimisation. If you are evaluating an AI forecasting system for your supply chain, reach out to discuss what an implementation scoped to your catalog and data infrastructure would look like.
Primary research sources:
M5 Accuracy Competition — Results, Findings and Conclusions
Machine Learning in Retail Forecasting — M5 Results
