Photo by Hitesh Choudhary on Unsplash
Artificial Intelligence (AI) has strongly matured as a technology in recent years, gaining broad acceptance across the business world and driving significant benefits in a wide range of applications and industries.
The art and science of managing AI projects, however, is still evolving. In fact, certain AI project failures can be directly attributed to a lack of tailored AI project management practices.
To address this issue, PMI and NASSCOM, India’s National Association of Software and Services Companies, have combined their expertise to develop a Playbook for Project Management in Data Science and Artificial Intelligence Projects. The playbook presents a framework with recommendations on resources that organizations can use to bolster their capability for Data Science (DS) and AI projects, as well as a best practices toolkit to apply to different project stages.
In this post, Snehanshu Mitra, CEO—Center of Excellence, Data Science & AI at NASSCOM, and Srini Srinivasan, Managing Director, PMI South Asia, discuss how and why the playbook came together.
Playbook for Project Management in Data Science and Artificial Intelligence Projects
The Playbook for Project Management in Data Science and Artificial Intelligence Projects presents a framework that recommends resources that organizations can use to build capability and a best practices toolkit for different stages of a DS/AI project.
DOWNLOAD THE PLAYBOOK
Why is there a gap in project management practices for DS/AI initiatives? Why don’t traditional project management practices apply?
Snehanshu: Data Science is a complex and multi-disciplinary domain and unlike traditional IT projects, the aspect of problem discovery is more nuanced and takes much longer. The techniques and application of machine learning frameworks are carefully assessed in light of the problem at hand and any misalignment at this stage could prove to be costly given that DS/AI projects are resource intensive.
The project teams are also fairly diverse—looking into specialized areas of data management, preparation and exploration, machine learning, product development and designing customer experience. These are mini projects by themselves and yet are expected to work in sync with a well-established handover-takeover process.
Srini: There’s a certain mystique surrounding AI projects. They call for high levels of innovation, creativity, speed and agility, and they’ve always been seen as being in a class by themselves—a cut above traditional business projects. We’re aiming to penetrate that mystique—to understand what makes an AI project tick and to define approaches that will enable practitioners to effectively and efficiently manage these projects. Our goal is to develop a “fit for purpose” framework that allows organizations to derive maximum benefit from their DS/AI initiatives.
It’s important to note that our model is the result of expert consultations with practitioners across a range of organizations who have a deep understanding of managing all aspects of DS and AI projects. We are truly grateful for their insights.
What are the specific challenges involved in managing DS/AI projects?
Snehanshu: Research among these expert practitioners uncovered three primary challenges in managing DS/AI projects:
Traditional project management practices seem to be of limited effectiveness when applied to DS/AI projects.
- There is a significant need for experimentation in DS/AI projects. This makes adhering to a given project management process extremely difficult.
- Defining and measuring success can be problematic because setting KPIs and pegging them to a business value depend on the availability of data, model behavior and other factors.
The DS/AI experts who shared these insights came from some 25 organizations cutting across different geographies and types of organizations—from GCCs (Global Capability Centers) to start-ups to service companies.
Srini: Respondents also represented a range of industries, including information technology enablement services, semiconductors, consumer packaged goods, computer hardware, agritech, financial services, chemicals, management consulting, telecom and electrical equipment.
Of note, 76 percent of these organizations reported using their own customized methodologies for managing DS/AI projects—highlighting the absence of a standardized methodology. And 88 percent reported gaps in their practices for AI projects. These organizations also said that 21 percent of the total wastage in AI projects in 2023 could be recovered with effective project management practices.
What does the playbook cover?
Snehanshu: Broadly speaking, the framework we’ve developed in the playbook aims to strengthen teams’ capabilities in three areas:
- Enabling experimentation via recommended “unstructured practices.”
- Supporting alignment on “what success means,” including recommendations around workflow training, success metrics and communications checklists.
- Sharing best practices on managing data and model maintenance through the adoption of an organization-wide data strategy, appropriate technologies and the reusability of models and databases.
Essentially, we’ve developed a best-practices-based toolkit for each stage of a DS/AI project, including business understanding, data preparation, modelling, implementation and closing. We also address overarching practices applicable to all stages of project management, such as quality assurance and risk management. With this toolkit we’re seeking to raise the maturity level of DS/AI project management to keep pace with the growing maturity of DS/AI technologies.
Srini: The playbook also addresses one other important need: capability-building for individuals and organizations—so they can fully realize the benefits of transformative projects. We’re addressing this at three levels:
- Capability-building for practitioners new to DS/AI, including building technical knowledge of programming, statistics and domain knowledge.
- Capability-building for existing DS/AI practitioners, focusing on the basic work-flow of DS/AI projects.
- Capability-building for organizations, providing specialized training for dedicated roles to manage complex DS/AI projects at the organizational level.
In developing this unified framework—both in terms of capability-building and best-practices—we are indebted to NASSCOM’s Center of Excellence, Data Science & AI. Not only did they bring deep subject matter expertise to the task, they also contributed a practical wisdom that would be difficult to find anywhere else. It’s been a hugely rewarding experience.
In part two of our two-part series, we take a deeper dive into the playbook and share some of the best practices we’ve uncovered.