AI is a transformational tool that can lead to significant improvements in key operational metrics. Yet 95% of implementations are failing according to MIT’s Nanda research group and that’s because there’s a gigantic skill gap between individuals who drive the idea teams and those who actually implement AI systems.
Unless this gap is addressed through meaningful upskilling and training throughout firms of all sizes, companies will continue to throw bad money after good in hopes of chasing returns that will never materialize or rescuing projects doomed to fail from the start.
AI Ideas Are a Dime a Dozen
The first issue we need to address is that ideas for implementing AI whether it’s cleaning up a paper based accounting workflow, speeding up creation and delivery of marketing assets or automatically writing custom emails to sales prospects are a dime a dozen.
It’s not an insult or dig at anyone building or selling enterprise level engagements but there’s a blindness that any process in effect today can magically be transformed by integrating AI and that’s where the issues begin.
In my previous consulting work, we had thousands of ideas for how AI could impact every aspect of our clients’ operations and we were constantly pitching AI for (insert business process here) and how the manual, grind heavy work of today could be completely automated tomorrow.
The disconnect was that we would build this beautiful dream for our prospects … “Imagine this 14 day process condensed into 4 hours and how that would transform your marketing department” but there was no thought to how it would actually get done.
It was the art of the possible with just a small footnote of implementation.
Just Plug AI into the System
Generally teams think of AI implementations as:
- Hooking into ChatGPT
- Training a custom model based on your assets and brand voice
- Leveraging an existing proprietary AI
- Leveraging a 3rd party vendor that says they can do this
The problem is that very few people at the idea level have any concept of how this works in real world environments. In the US, people in charge of AI projects have little to no training in AI deployment.
It’s one thing to understand the basic concept of how AI reasons output when you ask it a question – but it’s another thing entirely to understand what training a model really entails and how to design a working AI system diagram with inputs/, outputs, gating, security protocols and compliance measures.
Teams often sell an idea with the implicit assumption that someone smarter in some remote team could take that input and magically design a working system that met all of the project needs. Deferring the cognitive and implementation workload to unknown people on unknown teams somewhere in the organization is the cardinal sin of project management.
Data is Important?
The one thing teams never discuss is the role that clean, structured data plays to enable AI’s ‘magic’. Unfortunately employees on the consulting and client side are often so caught up in the demands from higher ups for rapid ASAP implementation or the lucid dream of a single click being able to create a tidal wave of productivity that the idea of rigorous data cleanup and evaluation gets lost in the shuffle.
The selling point to prospects was “We’ll take your data and …” “We can use your existing data to transform your …” “Leveraging your data we can” … when the reality was that client data was often, for compliance reasons, siloed and gated among systems. Projects that needed to tie an interaction in one part of a data ecosystem to an outcome somewhere else were dead in the water because compliance wouldn’t allow those pieces to interact.
Furthermore a lot of data was missing or incomplete and training a model using large datasets where 20% of the fields are blank is a gigantic waste of time and money. In fact a data error rate of 0.0001% is enough to poison an entire model in certain instances.
There was a lofty idea that as long as data exists, somewhere in the ether, that you just connect it to AI and tell it to predict outcomes or generate content and voila your work is done and the fee can be collected.
Someone Somewhere Can Do It
We need to dive further into the cardinal sin of assuming someone somewhere can do something.
Organizations across every industry instill the mentality of “Be scrappy, make calls, get to someone who can get to someone who can” … and that mentality is the exact opposite of what you need when working with AI.
Greenlighting an idea on the assumption you can just Google a vendor or throw a requirements document to someone in India or Poland and they’ll magically build an A-team for implementation is an all too common practice that kills progress.
You don’t just make a ‘webhook into a GPT’ or ‘build a local model’ and assume that because it sounds smart to say – that it actually does anything. AI implementations require significant planning, team building, data cleanup and testing just to get basic output that will conform to a single acceptance value.
There’s nothing wrong with having an idea and wanting to execute it at scale but without the foundational knowledge of how this happens, your project won’t go anywhere. It’s like having an idea for a new car and instead of partnering with an automotive design studio, you go to a junkyard and try to piece something together.
Where do we go from here?
We need mass investment in upskilling at both the brand and consulting level to get team members up to speed on the mechanics and requirements of implementing and operating AI enhanced systems.
A small course asking someone to write a prompt and use Midjourney to make an image are in no way sufficient for large sclae operations.
We need to train teams on data, compliance, model function, prompting and so much more than how teams are trained right now at the same enterprises priding themselves on selling the ideas of the future. Failing to upskill teams will lead to even more wasteful spending and at some point someone somewhere will pull the plug after another dead in the water project with no ROI launches.