Opening the" Black Box": Tools to Improve Understanding of Neural Network Reasoning for Geoscience Applications

摘要: Artificial neural networks (ANNs) are emerging in many geoscience applications for a large variety of tasks, including prediction, classification, anomaly detection, and potentially representing subgrid processes in climate or weather models. We highlight methods developed within the field of explainable AI (XAI) for the interpretation of ANN models. This effort extends recent work (McGovern et al., 2019) on interpretation of ML methods for geoscience applications. However, we focus on a different set of methods, namely layer-wise relevance propagation (LRP), which we believe is particularly useful for geoscience applications. LRP methods (Bach et al., 2015), such as Deep Taylor decomposition, seek to explain decision making of ANNs by identifying which elements of the input data are most important for the model to lead to the corresponding output. Understanding those details is important to be able to a …