Be it the core of their product, or just a component of the apps they use, every organization is adopting machine learning and AI at some level. Most organizations are adopting it an ad hoc fashion, but there are a number of considerations—with significant potential consequences for cost, timing, risk, and reward—that they really should consider together.
That’s why I developed the following framework for organizations planning to adopt machine learning or wanting to take their existing machine learning commitment to the next level.
Does your organization have well-defined problems that can be solved using machine learning?
• Have you assessed opportunities for using machine learning, organization-wide?
• Do your people need training or input from data science experts to help them identify machine learning opportunities?
What are the potential benefits of investing in machine learning solutions? e.g.
• speed up processes / fewer mistakes / lower risk / lower cost
obtain new insights [see a high level overview here]
• deliver new solutions/features
Who will evaluate these opportunities?
• in house data scientists
• outside data science consultants
• “owners” of these opportunities—the individuals or teams who want to use machine learning
• internal stakeholders responsible for revenue, technology, data, innovation
Does your data lend itself to machine learning solutions?
Is there enough data for a machine learning solution?
Is the data accessible to your data science platform? (via data warehouses, federation, data lakes, APIs)
Is the data suitable (in quality and structure) for machine learning?
• should you clean, restructure, or amalgamate it
• should you employ a vendor to improve your data
Could using certain data potentially lead to brand damage or legal liability?
• privacy issues (GDPR, etc.)
• bias issues (adverse impact and/or embarrassing incidents)
Can machine learning algorithms be meaningfully applied to your data? [check out some of these cheat sheets to put this in perspective]
What methods can be meaningfully applied to your data?
• supervised (classification, regression), unsupervised (clustering), reinforcement, hybrid, etc.
Is interpretability (human understanding of machine decisions) needed?
• expected in many outward-facing uses in financial services, healthcare, hiring, etc.
Do you need training data? (for supervised learning)
Do you already own labeled training data?
Is training data publicly available (e.g. Image Net)?
Do you need to generate training data?
• should you label it yourself
• should you employ a vendor to label your data (e.g. MightyAI)
Will the training data incorporate cultural biases into your models? (e.g. race, gender, age, religion, SES)
• could the organization’s use of biased models potentially lead to brand damage (negative publicity) or legal liability
What does your machine learning platform consist of?
What set of tools and workflows will make machine learning available cost effectively across your organization (for customer care, marketing personalization, supply chain, HR, finance, etc.)?
• make available a wide range of machine learning methods
• avoid creating data silos
• avoid redundant solutions / redundant tools & licenses
• avoid platform components that don’t scale to fit your use
• avoid creating solutions with unacceptable latency (e.g. customer experience can’t depend on long-latency machine learning decisions)
Build or buy?
What’s your organization’s existing platform for machine learning?
• what tools do your business analysts or in-house data scientists already have access to
• what data science enabling storage and processing will you need (like data lakes/warehouses)
How many low-hanging fruit problems can be solved with your current platform?
Is your current platform robust enough to take on long term strategic challenges?
Will your current platform scale as machine learning opportunities grow across your organization?
Are any machine learning solutions part of your organization’s core competency?
• do you want to own a proprietary methodology (e.g. because no one else has done it right, because you’d have to pay to have it done from scratch, to productize and resell it, to gain a competitive advantage)
Do if any capabilities does your in house data science team already have?
How long will it take to develop (lead / train / hire) the team you need?
Is there a viable consulting / outsourcing alternative for near-term/long-term opportunities?
Is there a vendor solution for some or all of your existing machine learning solutions?
• can you farm-out just a piece of your solution to a Partner? (e.g. data reconciliation / federation such as Amperity)
If you decide to build a custom solution, will it be on-premises, cloud, hybrid, serverless?
What steps lead to implementing your machine learning vision?
• prioritize machine learning opportunities, company-wide
• create and maintain a comprehensive data platform development plan
• allocate resources
• research and select tool and knowledge vendors
• put metrics in place
• put policies in place (e.g. privacy, security, bias, brand communication)
• schedule roll-out (avoid resource conflicts)
What’s working, and where can you make improvements?
Record and share
• success stories
• lessons learned
• opportunities discovered
• cost benefit analysis
• projected benefits
Inquire and prioritize: where do we need to improve?
• internal communication
• training (did your teams have the knowledge they needed to identify machine learning opportunities?)
• machine learning opportunity spotting
• data quality
• data accessibility
• data privacy
• fairness (avoiding discriminatory bias)
• external communication
• data science team
• access to our machine learning platform
• breadth/depth of our machine learning platform (did your team have access to the tools they needed to take advantage of their opportunities?)
• cross-functional access to solutions our teams have created
• vendor management
• risk management
This a work in progress. Please contribute—leave your comments below, or tweet them to @bruce_2b.
I’ll flesh out some of twigs and leaves of this outline by blogging about them in more depth (for example, here’s my blog about interpretability). Please let me know which topics you want to see more information about, and share resources you know of that cover any of the above topics.