Subsequent projects | Special projects

Start of funding 01.07.2024

Local explainable machine learning from tree-ensembles

Prof. Dr. Merle Behr
University of Regensburg
Fakultät für Informatik & Data Science

Prof. Dr. Bin Yu
University of California, Berkeley
Department of Statistics


The research project aims to enhance the interpretability of machine learning (ML) models, specifically tree-ensembles like Random Forests. While ML and AI techniques, including RF, achieve high prediction accuracy, they are often seen as 'black boxes' due to their complexity. This makes it difficult to understand the reasoning behind specific predictions. The project's goal is to develop methods to provide explanations for these predictions, particularly at the local level, where the focus is on individual data points and local feature interactions. This project is particularly relevant for applications in healthcare, where patient-specific explanations can inform personalized treatment strategies.

Final report:

Activities

With support from the BaCaTeC funding for the project „Local explainable machine learning from tree-ensembles“, the scientific collaboration between the research group of Prof. Behr (University of Regensburg) and the research group of Prof. Bin Yu (University of California, Berkeley) was successfully expanded.

In October 2024, Dr. Kata Vuk completed a three-week research stay in Berkeley. The aim of this visit was to exchange ideas on methodological questions in the area of local interpretability of tree-ensemble methods. The focus was in particular on extending the LSSFind method [1] at the local level, with Prof. Yu providing substantial input.

In June 2025, Prof. Yu visited the University of Regensburg. During this stay, she gave a scientific lecture and held intensive discussions with the research group of Prof. Behr. The discussions focused on new methodological perspectives in the area of local explainability. In addition, Prof. Yu gave a lecture for students on Veridical Data Science [4].

In October 2025, two PhD students from the Berkeley group (Austin Zane and Zachary Thomas Rewolinski) visited the University of Regensburg. They presented their current research on local importance measures [2] and actively participated in the scientific exchange, thereby strengthening the cooperation at the junior researcher level as well.


Results and Outlook

The activities carried out through the funding have led to progress in the development of methods for the local interpretability of machine learning models. In particular, the Local LSSFind method [3] was further developed conceptually and methodologically. Furthermore, the lectures by the PhD students from Berkeley led to intensive scientific discussions and provided important impulses for the master’s thesis of Alison Durst, who is now working as a PhD student at the Behr group.

The lectures and discussions also contributed to the development of ideas for further joint projects. A central future focus lies on extending previous random forest-based approaches to other tree-based machine learning methods, such as Bayesian Additive Regression Trees. Furthermore, it is intended to publish the results in joint scientific papers and to maintain the collaboration over the long term.


Publications

[1] Merle Behr, Yu Wang, Xiao Li, and Bin Yu. Provable boolean interaction recovery from tree ensemble obtained via random forests. Proceedings of the National Academy of Sciences, 119(22):e2118636119, 2022.

[2] Zhongyuan Liang, Zachary T Rewolinski, Abhineet Agarwal, Tiffany M Tang, and Bin Yu. Local mdi+: Local feature importances for tree-based models. arXiv preprint arXiv:2506.08928, 2025.

[3] Kata Vuk, Nicolas Alexander Ihlo, and Merle Behr. Provable recovery of locally important signed features and interactions from random forest. arXiv preprint ar- Xiv:2512.11081, 2025.

[4] Bin Yu and Rebecca L Barter. Veridical data science: The practice of responsible data analysis and decision making. MIT Press, 2024.