Explainable AI/ML (XAI) for Accountability, Fairness, and Transparency
Page: https://xai.kdd2019.a.intuit.com/
Keynote: Responsible Use of Explainable Artificial Intelligence
Patrick Hall
https://github.com/jphall663/kdd_2019/
Intro
Understanding the trust
The dark side
Surrogates
Intro
What is an explanaition in ML?
What do I mean by explainable ML?
Mostly post-hoc techniques users to enhance understanding od trained model mechanism and prediction
Direct measures od global and local feature
Responsible use of explainable ML can enable
- Human learning from machine learning
- Human appeal of automated decision
- Regulatory compliance
- White-hat hacking and security audits of ML model
Even logistic regression is often “explained ",or post-processed for credit scoring, eg)Max points lost method and adverse action notices.
Why propose guidelines
- Misuse and abuseof explainable ML can enable
- ExplainableML is already in-use
- Regulatory guidance is not agreed upon yet
Proposed guidelines for responsible use
- Use explainable ML to enhance understaning
- Learn how explainable ML is used for nefarious purpose.
- Augment surrogate models with direct explanations
- Use highly transparent machanism for high stakes application
Use explainable ML to enhance understaning
Use explainable ML to enhance understaning
corollary2.1: White-hat attackes can crack potentially harmful black-box
corollary2.2: Explanation is not a front line fairness tool
Surrogates models
BUT many explainable ML techniques have nothing to do with surrogate models
Augment surrogate models with direct explanations
corollary3.1: augment LIME with direct explanations
Use highly transparent machanism for high stakes application
corollary4.1: explanations and interpretable models are not mutually exclusive.
Interrude: an ode to the shaply values
corollary4.3: explanations and fairness techniques are not mutually exclusive
Contributed Talk: Parametrised Data Sampling for Fairness Optimisation
Carlos Vladimiro Gonzalez Zelaya, Paolo Missier, and Dennis Prangle
Slides: https://github.com/vladoxNCL/fairCorrect/blob/master/Presentation.pdf
outline
- Method for correcting classifier fairness
- Model and definition agonistic
- Tune correction level to optimize fairness
Train set correlation
Sampling strategies
- Original data
- Under
- SMOTE[chawia et al 2002]
- Preferential
Equality form
Ratio form
Some fairness ratios - Demographic parity
- Equality of opportunity
- Counterfactual (proxy)
Effect on test set (COMPAS, Undersampling) - Effect is correlated with correction
- But it occurs to a different effect
- Intersection is not at d=0
Optional correction by fairness and sampling
Accuracy vs fairness trade-off
Conjecture: A LR model is FTU <> CFR = 1
Conclusion
- Fairness-agonistic optimization with a relatively small loss in accuracy
- ideal correction level is definition dependent
- Different sampling strategy produced similar result
Contributed Talk: What Makes a Good Explanation? The Lemonade Stand Game As a Platform for Evaluating XAI
Talia Tron, Daniel Ben David, Tzvika Barenholz, and Yehezkel S. Resheff
Title: What can selling lemonade teach us about XAI
What to explain:
Nothing > the data > the model > the prediction
Model transparency
Global explanation
local explanation
What makes a good explanation
Explanation quality
Users understanding
user behavior adoption trust, satisfaction
The challenge
- Behabioal measures
- Dynamic
- Real financial
- Counrolled environment
- Uncertain conditions
Research questions
- Evaluate the influence of different explanations on financial alporism advice adoption
Experiment: the situation of selling lemonede.
Keynote: Interpretability - now what?
Been Kim
Sea of interpretability
- Where are we going ?
- What do we have now ?
- What can we do better ?
- What should we be carefull ?
We need to ensure to
1, Our values are aligned
2, Our knowledge is reflected for everyone.
Not out goal
- About making all models interpretable
- About understanding every single bit about the model
- Against developing highly complex models
- Only about gaining user trust of fairness
What do we have now ?
Investing post-training interpretability
Saliency Maps
Some confusing behavior of saliency Maps
Sanity checlk1
When prediction changes do explanations change ? NO
Network rained with true and random labels do explanations deliver different explanation
What can we learn from this?
- Confirmation bias
- Others who independently reached the same conclusions
- Some of these methods have been shown to be useful for humans why ? More studyes are needed .
- Resent work by Gupta Arora 19’ suggests a simple fix
Goal of TCAV: Testing with concept activation vectors
Defining concept activation vector (CAV)
TCAV core idea: derivative with CAV to get prediction sensitivity
Guiding against spurious CAV
Quantitive validation : guarding against spurious CAV
Recap TCAV: testing with concept activation vectors.
Limitation of TCAV
Concept ohas to “exoressible” Using examples
User needs to know which concepts they want to test
Explanation is Not-casual
What should we be carefull ?
Proper evaluation
Remember that humans are biasied and irrational
Design the right interaction (HCI)
Try to criticize (think about what wasn’t talked about in this talk but should have )
keep checking