A year ago, the AI Now Institute released its inaugural report on the near-future social and economic consequences of AI, drawing on input from a diverse expert panel representing a spectrum of disciplines; now they’ve released a followup, with ten clear recommendations for AI implementations in the public and private sector.
The first of these is “Core public agencies, such as those responsible for criminal justice, healthcare, welfare, and education (e.g “high stakes” domains) should no longer use ‘black box’ AI and algorithmic systems.”
The remaining recommendations deal with operational details, like examining training data for bias and validating the performance of the models to ensure that they aren’t misfiring; and areas where work needs to be done, like evaluation of the impact of AI on hiring and HR, setting data-set quality standards; bringing cross-disciplinary expertise to bias evaluation; and the active inclusion of women, minorities and other marginalized populations in systems design and evaluation.
1 — Core public agencies, such as those responsible for criminal justice, healthcare, welfare, and education (e.g “high stakes” domains) should no longer use ‘black box’ AI and algorithmic systems. This includes the unreviewed or unvalidated use of pre-trained models, AI systems licensed from third party vendors, and algorithmic processes created in-house. The use of such systems by public agencies raises serious due process concerns, and at a minimum such systems should be available for public auditing, testing, and review, and subject to accountability standards.This would represent a significant shift: our recommendation reflects the major decisions that AI and related systems are already influencing, and the multiple studies providing evidence of bias in the last twelve months (as detailed in our report). Others are also moving in this direction, from the ruling in favor of teachers in Texas, to the current process underway in New York City this month, where the City Council is considering a bill to ensure transparency and testing of algorithmic decision making systems.
The 10 Top Recommendations for the AI Field in 2017
[AI Now Institute/Medium]
b