Final Exam
General information
• Public Artifact:
▪ Link - https://www.technologyreview.com/s/613508/ai-fairer-than-judge-criminal-riskassessment-algorithm/
• Application/Scenario/Domain of Misuse: Criminal Risk Assessment (Predictive Algorithm)
• Regulated Domain/Protected Class: Education/Race
• Evidence: Dataset - https://www.propublica.org/datastore/dataset/compas-recidivism-risk-scoredata-and-analysis
Task 2 (20 pts): Provide a 1-2 paragraph summary of the bias that is identified by the public artifact. In your description, your summary should be specific and reference definitions, concepts, ideas discussed during this course (i.e. in lectures, assignments, cases, etc.). Please BOLD all references to definitions, concepts and ideas discussed in class.
Task 3 (30 pts): Provide specific details (in bullet or table format) on any and all quantifiable metrics that are available or can be derived from compiling together information from the artifact and associated evidence. There should be enough metrics (at least 6) and details provided for us to validate your ability to synthesize course concepts based on the overarching topics:
• Privileged/unprivileged groups
• Misleading graphs
• Sources of Data Bias
• Sources of Sampling Bias
• Sampling Methods Used to Collect Data
• Correlations found in the data
• Outcome measures: Averages, Standard Deviations, Quartiles, Frequency Distributions, Margins of Error
• Bias & Fairness (or other) metrics used to identify differences in outcomes
Task 4 (25 pts): Identify an issue related to one of the quantifiable metrics listed above (Task 3) that, if addressed, might help mitigate bias and/or unfairness. Design a method to help address the issue identified. The method should relate to a concept discussed in the lectures. You can explain the method using pseudo-code with an explanation or a python script with comments. Remember to identify the issue, the data inputs and outputs (based on the evidence), and the anticipated change in outcomes. Note: You do not need to have a working code or quantitative results.
