While modern AI performs well in certain and closed-world contexts, these AI systems may fail in uncertain and open-world contexts. For example, individual sensors of an AI system may be noisy or intermittently fail causing incorrect or missing values. Beyond uncertainty, the open-world nature of real-world contexts means that there are complex changes in the operating environment. Some environment changes are exogeneous to the system such as a change in foreign policy or a world event, while others directly affect the inputs to the system such as a change in weather, location, or sensor network. Given these deficiencies in modern AI, human oversight is required for practical systems. To alleviate the problems above, a contextual AI system should be able to explain its uncertainty and changes in its environment.
This project aims to fill these gaps in contextual AI by (1) providing a self-explainable neural networks, (2) combining efficient input and parameter uncertainty methods into a unified uncertainty framework and (3) developing a general method for explaining distributions shifts between environments. Specifically, we incorporate Shapley feature attribution values [Lundberg & Lee, 2017] as latent representations in deep models thereby making Shapley explanations first-class citizens in the modeling paradigm [Wang et al., 2021]. We plan to combine the probabilistic circuit method for handling input uncertainty [Khosravi et al., 2019] with an efficient method for handling parameter uncertainty. We explain distribution shifts via transport maps between two distributions [Kulinski & Inouye, 2022a, 2022b]. We plan to also explore the connection of distribution alignment to invariance and causal discovery so that we may enable partially causal explanations. Access to causal structure is expected to unlock both counterfactual explanations and much simpler explanations.
References
Khosravi, P., Choi, Y., Liang, Y., Vergari, A., & Van den Broeck, G. (2019). On tractable computation of expected predictions. In Neural Information Processing Systems (NeurIPS), 32.
Kulinski, S., & Inouye, D. I. (2022). Towards Explaining Image-Based Distribution Shifts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops.
Kulinski, S., & Inouye, D. I. (2022). Towards Explaining Distribution Shifts. arXiv preprint arXiv:2210.10275.
Lundberg, S. M., & Lee, S. I. (2017). A unified approach to interpreting model predictions. In Neural Information Processing Systems (NeurIPS), 30.
Wang, R., Wang, X., & Inouye, D. (2021). Shapley Explanation Networks. In International Conference on Learning Representations (ICLR).
