The paper introduces TREVIS, a method for learning decision trees that jointly optimizes predictive accuracy and structural sparsity by mapping trees onto the continuous latent space of a Tree Transformer Variational Auto-Encoder. This replaces the usual discrete tree-search space with a continuous one, enabling gradient-based optimization through a differentiable surrogate model rather than combinatorial search. Experiments show TREVIS matches the predictive performance of existing near-optimal decision-tree learning algorithms while producing sparser tree structures.
