Who Needs Reasons for AI-Based Decisions?

Deep learning systems, which are the most headline-grabbing examples of the AI revolution—beating the best human chess and poker players, self-driving cars, etc.—impress us so very much in part because they are inscrutable. Not even the designers of these systems know exactly why they make the decisions they make. We only know that they are capable of being highly accurate…on average.

Meanwhile, software companies are developing complex systems for business and government that rely on “secret sauce” proprietary data and AI models. In order to protect their intellectual property rights, and profitability, the developers of these systems typically decline to reveal how exactly their systems work. This gives rise to a tradeoff between profit motive, which enables rapid innovation (something government in particular isn’t known for), and transparency, which enables detection and correction of mistakes and biases. And mistakes do occur…on average.

pay no attention to the man behind the curtain
Photo by Andrew Worley on Unsplash

On the one hand, a lack of transparency in deep learning and proprietary AI models has led to criticism from a number of sources. Organizations like AI Now  and ProPublica are surfacing circumstances where a lack of transparency leads to abuses such as discriminatory bias. The EU has instituted regulations (namely GDPR) that guarantee its citizens the right to an appeal to a human being when AI-based decisions are being made. And, last but not least, there is growing awareness that AI systems—including autonomous driving and health care systems—can be invisibly manipulated by those with a motive like fraud or simple mischief. Continue reading “Who Needs Reasons for AI-Based Decisions?”

The BaDFun Podcast

I started a new podcast! My co-host, the talented Ben Rogojan, and I are interviewing people who are figuring out how to use data in business, so that we can share what they are learning.

Joe Heck Podcast photo

Here’s the official description:

“The BaDFun “Business and Data Fundamentals” podcast is hosted by Ben Rogojan, a data guy focused on business, and Bruce Wilson, a business guy focused on data. In each episode we interview someone making data actionable in their organization, getting an inside look at some of the strategies, successes, and false starts of working with data in business. Our goal is to discover and share the steps organizations are taking to use data more effectively in analytics, operations, products, and services, ranging across finance, retail, manufacturing, technology, healthcare, agriculture, the public sector, and beyond.”

Episode 1 is live now, featuring my friend Joe Heck (a serial entrepreneur who is now a Director at EMC because they bought his startup)—check it out. And subscribe if you like the concept.


%d bloggers like this: