Type: Bachelor Master UROP
Status: Completed August 2022
Student: Piotr Romashov

Artificially Intelligence (AI) is slowly but surely becoming an integrated part of our daily lives. However, with decisions derived from AI systems ultimately affecting human lives (e.g. medicine and law), there is an emerging need for understanding how such decisions are made by AI methods [11]. Furthermore, the recent General Data Protection Regulation (“GDPR”) [1] tasks the machine-learning (ML) community to enable explainability of the models and their output. More specifically, according to the GDPR, the ML models should offer the possibility to answer/provide explanation such as: “You were denied a loan because your annual income was £30,000. If your income had been £45,000, you would have been offered a loan.” [1]. In the ML domain, this task is referred to as “search for counterfactual explanations”. The idea is that, besides the model’s output, additional counterfactual information should be provided of how the world would have to be different for another (e.g., more desirable) outcome to occur. This new requirement has some researchers argue that a model’s accuracy should be sacrificed, and interpretable models should be preferred over black-box ML models for high-stake decisions [2].

In order to augment existing AI systems with explainability, “Explainable Artificial Intelligence (XAI)” methods are being developed actively both in academia [3-9] and industry (e.g., IBM, Microsoft, Facebook and Google). XAI deals with the creation of machine learning techniques that enable end-users to understand, trust, and possibly manage the emerging generation of so-called “artificially intelligent partners” [10].

The goal of this project is to explore existing XAI methods and possibly develop a new method that would improve existing solutions, with a focus on counterfactual-based XAI.

Specifically, this Master thesis has four main tasks:

1. Surveying existing algorithms for generating counterfactual explanations (e.g., [3-9, 12]);
2. Implementing at least one XAI method and analyzing its performance (benefits and drawbacks) on several datasets (preferably in comparison with at least one related method from related studies).
3. Publicly available XAI service (or python library) to be used with scikit-learn models.

References:

[1] Wachter, S., Brent M., and Chris R. Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harv. JL & Tech. 31 : 841, 2017.

[2] Rudin, C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence 1, no. 5:206-215, 2019.

[3] Rudin, C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence 1, no. 5:206-215, 2019.

[4] Joshi, S., Oluwasanmi K., Warut V., Been K., and Joydeep G. Towards realistic individual recourse and actionable explanations in black-box decision making systems. arXiv preprint arXiv:1907.09615, 2019.

[5] Karimi, A.-H., Gilles B., Borja B., and Isabel V. Model-agnostic counterfactual explanations for consequential decisions. arXiv preprint arXiv:1905.11190, 2019.

[6] Wexler, J., Pushkarna, M., Bolukbasi, T., Wattenberg, M., Viégas, F., and Wilson, J. The what-if tool: Interactive probing of machine learning models. IEEE transactions on visualization and computer graphics, 26(1):56–65, 2019.

[7] Tolomei, G., Silvestri, F., Haines, A., and Lalmas, M. Interpretable predictions of tree-based ensembles via actionable feature tweaking. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 465–474. ACM, 2017.

[8] Ustun, B., Spangher, A., and Liu, Y. Actionable recourse in linear classification. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 10–19. ACM,  2019.

[9] Dandl, Susanne, Christoph Molnar, Martin Binder, and Bernd Bischl. “Multi-objective counterfactual explanations.” In International Conference on Parallel Problem Solving from Nature, pp. 448-469. Springer, Cham, 2020.

[10] D.Gunning, Explainableartificialintelligence(xAI),TechnicalReport,DefenseAd- vanced Research Projects Agency (DARPA), 2017.

[11] B.Goodman,S.Flaxman,Europeanunionregulationsonalgorithmicdecision-mak- ing and a “right to explanation”, AI Magazine 38 (3) (2017) 50–57.

[12] Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., … & Herrera, F.,  “Concepts, taxonomies, opportunities and challenges toward responsible AI,” Information Fusion, 58, 82-115, 2020

For more information contact: Martin Gjoreski