Marcel ILIE1 , Augustin SEMENESCU2
Abstract. Effective inventory management in multi-echelon supply chains is challenged by stochastic demand and uncertain lead times, which amplify variability and increase operational costs. This study presents a Markov chain framework for modeling, analyzing, and optimizing multi-echelon inventory systems under stochastic lead-time conditions. The framework represents inventory levels and lead-time states as a probabilistic transition system, enabling computation of steady-state distributions, stockout probabilities, and transient recovery times. Analytical results are validated through Monte Carlo simulations, demonstrating high fidelity between theoretical and empirical distributions. Numerical experiments quantify the impact of lead-time variability on inventory performance, revealing nonlinear increases in stockout probability and total system cost as lead-time variance grows. Multi-echelon analyses demonstrate the emergence of the bullwhip effect and highlight the effectiveness of information sharing in mitigating variability propagation across echelons. Comparative benchmarking against deep reinforcement learning (DRL) policies shows that while DRL achieves marginally lower total costs, the Markov-based approach provides superior interpretability, robustness, and computational efficiency. The study offers theoretical contributions by unifying stochastic multi-echelon dynamics and transient analysis within a tractable Markov framework. Managerially, it provides actionable insights on lead-time variance reduction, cross-echelon visibility, and hybrid analytical–learning policy design. The framework establishes a foundation for resilient and cost-effective inventory control in complex, uncertain supply-chain networks.
Keywords: multi-echelon inventory, stochastic lead time, Markov chain, supply chain management, bullwhip effect, reinforcement learning
DOI 10.56082/annalsarscieng.2025.2.94
1 Associate. Prof. Ph.D. Georgia Southern University, 1332 Southern Dr. Statesboro GA 30458, USA, *Corresponding author:, milie@georgiasouthern.edu
2 Prof. National Science and Technology University Politehnica Bucharest, Spl.Independentei 313, Bucharest, Romania, augustin.semenescu@upb.ro
PUBLISHED in Annals of the Academy of Romanian Scientists Series on Engineering, Volume 17, No2