Evaluation Protocol Determines Apparent Skill: Leakage-Aware Machine Learning for Early Prediction of Remedial Hole-Cleaning Events from Real-Time Drilling Data

Machine-learning early-warning systems for drilling operations are usually validated on randomly split data. Using ۱۶,۴۵۱ minutes of quality-controlled surface telemetry and five remedial hole-conditioning events mined from ۱۶۳ daily reports of the deviated hard-rock well ۱۶A(۷۸)-۳۲ (Utah FORGE), five classifier families are evaluated under random cross-validation and under a causal walk-forward protocol. Random splitting indicates near-perfect skill for every model (ROC-AUC ۰.۹۹۵ or higher), whereas walk-forward testing, in which two thirds of test minutes lie outside the training applicability domain, reveals modest skill at best (ROC-AUC ۰.۷۵). Leakage-aware evaluation practices are recommended.

کلیدواژه ها:

Drilling Data Analytics ، Hole Cleaning ، Machine Learning ، Data Leakage ، Applicability Domain