AI & ML
ShapleyPipe: A Game-Theoretic Approach to Data Preparation Pipeline Construction
arXiv cs.DB0 views
arXiv:2510.27168v2 Announce Type: replace Abstract: Data preparation is a critical yet highly time-consuming component of machine learning (ML) workflows and has a substantial impact on downstream model performance. Automated data preparation pipeline construction requires selecting appropriate operators, such as missing-value imputation, feature scaling, and feature transformation, and placing them at suitable pipeline positions. However, existing methods lack a principled and interpretable mechanism for understanding the marginal contribution of each operator at a specific position. To address this limitation, we study data preparation pipeline construction from a cooperative game-theoretic perspective. We formulate the task as a pipeline construction game, in which operators act as players that cooperatively improve downstream model utility. Based on this formulation, we propose the Pipeline Shapley value (PipeShapley), a contribution-attribution method that explicitly captures position-dependent contributions and distinguishes repeated occurrences of the same operator type. Building on PipeShapley, we introduce ShapleyPipe, an interpretable pipeline construction framework that selects and orders operators according to their position-aware contributions. To make PipeShapley practical, we develop Hierarchical ShapleyPipe (H-SP) based on a Category Coherence Assumption to exploit coherent contribution patterns, combining MAB-based representative selection with adaptive early termination. Experiments on 27 classification datasets show that H-SP achieves the highest average accuracy of 0.847 and the best average rank among the evaluated practical methods, while lying on the empirical accuracy--runtime Pareto frontier. On three datasets with up to 581K records, 10K-subsample search and 16-worker parallel execution reduce the average wall-clock time to 891 seconds while retaining the highest average accuracy of 0.705.
Source:https://arxiv.org/abs/2510.27168PreviousLearned Adaptive Indexing with Updatable Learned IndexNextContinuous Prompts: LLM-Augmented Pipeline Processing over Unstructured Streams
Related posts
Reading project health off a download page — faster than star count
0dev.to
Best time trackers for developers, for code and for AI
0dev.to

Serving Qwen3.5-0.8B with a 0.016% weight edit: measure the loops, pin the sampling on the server
0dev.to
Design Tokens for AI: Extract a Website's Real Colours, Fonts, Shadows and Icons So an LLM Builds On Brand
0dev.to
Trying to get our Arms around AI efficiency
0Stack Overflow Blog
Why an Embedded Game Fails on Some Phones, and How to Debug It
0dev.to