The AI Assessment Canvas, Version 1
AI is overhyped. What was once machine learning is this year, artificial intelligence. The VC rush is in full force, funding AI startups at a record pace. And while generative AI products like ChatGPT and MidJourney flood the markets and we face the pressure to ship AI-powered features for our products, the hype train is also making assessing the risks associated with AI more difficult.
Why? Because we’ve all already been using ML, or ahem, AI, for years, even if we haven’t even noticed, but generative AI in particular creates new problems.
The first major concern all businesses are facing isn’t really about the right AI capabilities for product lines, it’s about the implications AI methods have for data privacy and security. Then, after overcoming that, the challenges with AI’s hallucinations, inaccuracies, and ethics kick in.
So as a business or product team, how do you make sense of this first concern and facilitate a good risk assessment?
There are lots of reasons to be careful about where sensitive and personal data go, the imposed regulations being one, but doing right by customers is another.
It’s just that AI makes things a little odd that break the typical control patterns.
An Simplified Lifecycle of Sensitive Data
The biggest sticking point for businesses today is awareness and control of where sensitive data flow, particularly personal data and company data.
If you work with your data, security, and legal teams, the default reaction to anything AI-related will likely be to treat it all as high risk. This is natural; the stories of how AI is a black box that we can never understand strike fear into most risk-minded folks.
Here, I’ll talk about a data lifecycle with eight key components:
- Generation + authorship
- Storage + custody
- Encryption + decryption
- Transmission
- Processing
- Consumption + access
- Destruction
- Incorporation into AI models
The last point is of course the subject of this post, particularly because it changes the topology of data flow. How do the peculiarities of new AI technologies change the custody and lifecycle of sensitive data? And how do we simplify this when we’re defining new products? And this doesn’t even cover the rapidly evolving governance and regulatory landscape.
Where do the data go?
First, consider the application. Just because it says “AI” doesn’t mean the app is ready to share private data with the world in some surreptitious way.
Your legal, security, and data privacy teams are certainly the experts in compliance, but some aspects of this technology make their typical processes a bit ineffective.
Whatever can be highlighted to show how data flow through an AI-powered app is really what those teams need to provide the right risk assessment.
Incorporation into AI Models
This is the part that starts to break the typical controls on data custody.
- Inference
- Chat history
- Fine-tuning
- Access patterns like RAG
- Training
- Reinforcement learning patterns
The Canvas
But with a little diagramming, we can cut through this. This is for anyone wanting to make sense of the state of AI.
How to use the canvas
(TBD)
After The Data Lifecycle, The Real AI Problems Emerge.
Once this is ironed out though, companies will need to wrestle with the problems that come from AI itself, things like hallucinations.
Once your strategy team are plugged into a comprehensive system using your proprietary data, they could inadvertently generate some analyses that don’t reflect reality. It may be so subtle that it passes our notice, and we may make product and business decisions from false conclusions.