Council Post: Explainability and Interpretability in Machine Learning Models, Bridging the Gap Between Accuracy and Transparency

In the ever-evolving landscape of machine learning, the pursuit of accurate predictive models has been a central focus. However, as models become increasingly complex, there arises a critical need to comprehend and trust their decision-making processes. This imperative has given rise to the crucial concepts of explainability and interpretability in machine learning models. The quest to bridge the gap between accuracy and transparency is a pivotal journey, acknowledging that the reliability of AI systems not only hinges on their predictive prowess but also on the capacity to demystify their inner workings. This exploration into explainability and interpretability unveils the significance of understanding how algorithms arrive at their conclusions, fostering trust, accountability, and ethical deployment in diverse applications. As we delve into this nuanced realm, we embark on a journey to dissect the inner workings of machine learning models, seeking a harmonious equilibrium between cutting-edge accuracy and the transparency essential for real-world applicability.

We had a roundtable discussion on the topic with a set of experienced and distinguished leaders in the industry. The session was moderated by Kashyap Raibagi, Associate Director – Growth at AIM along with panelists Satheesh Ramachandran, Head of AI and Analytics Product at Charles Schwab, Farhat Habib, Associate Vice President at Mphasis, Avijit Chaterjee, Head of AI/ML and NextGen Analytics at Memorial Sloan Kettering Cancer Center, Arjun Srinivasan, Director - Data Science at Wesco, Rishi Bhatia, Director - Data Science at Walmart Global Tech and Deepak Jose, Global Head and Senior Director for One Demand Data and Analytics Solutions at Mars.

Exploring the Trade-off Between Explainability and Accuracy in AI

This tradeoff between explainability and accuracy is not a new conversation. If you step back 20-30 years ago, with the classic credit scoring model, for example, you needed to explain why somebody got rejected for a loan application, another example, from having been working in fraud for a long time. In fraud, when you flag something as a lead for an investigator to pursue, explainability is important for two reasons: a) the investigator wants to know why this model gave it a high score - it is very important that you have buy-in/trust from the end users of this model, because you are completing against prebuilt bias of the investigator, and b) regulatory reasons - when rejecting somebody's credit card transaction, we need to explain to internal auditors that the decisions that the model is making is not biased and discriminatory.

So, explainability has been there for a long time. It's just that it's evolving. Before the advent of some of the de-facto explanatory algorithms these days such as Shap, LIME, and..there were the rock solid metrics from old school predictor contribution metrics from logistic regression scorecards that were the mainstay of financial, insurance and other regulated industries for a long time.