A recent Open eLMS discussion explored the critical challenge facing Learning and Development (L&D) professionals: proving the return on investment (ROI) of their initiatives in an era increasingly shaped by artificial intelligence. The conversation featured Emil Reisser-Weston, host of the Open eLMS podcast, and Derek Mitchell, founder of Gallus Insight, an analyst known for his direct approach to L&D measurement. Mitchell posited that L&D budgets are under severe threat unless teams can demonstrably link their efforts to clear, provable business value.
The dialogue underscored a significant shift in expectations, where the C-suite, now empowered by AI tools like ChatGPT for rapid content generation, questions the traditional pace and cost of L&D departments. This perception, Mitchell argued, is potent: if AI can deliver a solution faster and cheaper, L&D teams must prove their human-led initiatives are unequivocally better.
Key Takeaways from the Discussion:
- Imminent Threat to L&D Budgets: AI's ability to generate content quickly means traditional L&D functions face budget cuts by 2027 if they cannot prove superior, measurable impact.
- Redefining L&D's Role: L&D must shift focus from content creation to strategic roles, such as facilitating coaching, skill development, and on-the-job learning, by leveraging data.
- The End of Traditional Metrics: Standard LMS metrics like enrolments and completion rates are no longer sufficient to demonstrate business value.
- Focus on Behavioural and Business Metrics: True ROI measurement requires tracking changes in employee behaviour (via situational judgment tests) and direct business KPIs (e.g., productivity, attrition).
- Vendor Accountability: L&D technology providers must offer systems that integrate with wider business data to enable real impact measurement, moving beyond simple activity tracking.
- AI for Enhanced Analysis: AI significantly simplifies the collection and analysis of complex data, allowing for deeper insights into learning effectiveness and causation, not just correlation.
- Standardised Measurement: The industry needs a unified approach to measuring learning experience and impact to foster transparency and allow for meaningful comparisons.
The discussion highlighted that while L&D teams might argue AI lacks the nuanced understanding of a specific business, this argument falls flat when senior leaders perceive AI as a faster, more cost-effective alternative. Mitchell emphasised that a machine, with access to vast amounts of data and studies, can often compile information more comprehensively and rapidly than any individual L&D professional. The core challenge for L&D, therefore, is not to compete with AI on content generation, but to demonstrate unique, quantifiable value that AI cannot replicate alone.
A New Mandate for L&D: From Content Creators to Strategic Enablers
With AI poised to handle much of the content creation, the role of L&D teams must evolve. Mitchell proposed a shift in focus from merely delivering training content – the '10' in the 70/20/10 learning model – to concentrating on the '20' (coaching and mentoring) and '70' (experiential, on-the-job learning). This means L&D professionals should become strategic enablers, using data to:
- Identify Skill Gaps: Pinpoint individual and organisational skill deficiencies.
- Forge Network Connections: Deliberately connect employees with complementary skills to foster cross-functional collaboration and knowledge sharing.
- Design Experiential Learning: Place individuals in projects or roles where they can develop specific skills on the job, directly addressing identified weaknesses.
This human-centric approach, supported by data, moves L&D beyond attendance lists and completion rates to truly understanding and developing employees. Platforms like Open eLMS, with its robust personnel management and skills monitoring features, are designed to support this strategic shift, enabling L&D managers to map skills to roles, track competency development, and identify gaps for targeted interventions that boost employee performance and ensure compliance.
The Power of AI in Content Generation and Personalisation
While L&D's strategic role grows, the landscape of content creation is fundamentally changing. Mitchell asserted that for L&D teams, "content generation is done." AI is now capable of producing bespoke, personalised learning content at speed and scale. By leveraging rich employee data – from work behaviours to system interactions – AI can generate tailored e-learning courses, videos, podcasts, and even gamified experiences that cater to individual needs and learning styles.
Open eLMS Learning Generator exemplifies this innovation, allowing organisations to create custom e-learning from existing documents or simple prompts in minutes. This AI-powered course generator transforms the content creation process, making it faster, more efficient, and highly personalised, freeing L&D teams to focus on higher-value activities.
Beyond Activity: Measuring True Impact and ROI
The podcast discussion critically examined the inadequacy of traditional L&D metrics. Emil Reisser-Weston noted that many businesses still prioritise "ticking boxes" over demonstrating real value, often relying on basic completion rates or generic course libraries. Derek Mitchell agreed, highlighting that most L&D leaders are not truly accountable for business results, such as direct cost reduction or revenue increase.
Mitchell proposed a focus on two primary data points for measuring true impact:
- Behavioural Change: Measured through situational judgment tests before and after training. These tests assess how an individual responds to specific scenarios, revealing maturity shifts in desired behaviours. This method offers high validity, avoiding the biases of self-reported surveys or the ethical concerns of intrusive behavioural monitoring.
- Business Metrics: Direct, quantifiable business outcomes relevant to the training. Examples include production rates in a warehouse, sales conversion rates, or employee absenteeism.
By combining these two data points, L&D can perform statistical analysis to determine which behaviours most effectively drive desired business outcomes. This allows for targeted investment in specific behaviours for specific groups, maximising the impact of training. Open eLMS's custom reporting with Power BI integration provides the advanced analytics capabilities needed to connect learning activities with these critical business KPIs.
The Role of Technology Vendors in Driving Accountability
Mitchell argued that many content and platform vendors historically have had little incentive to measure true impact, often focusing their analytics on activity rather than outcomes. The truly meaningful data, he suggested, often resides outside the LMS – in HR systems, social platforms, or departmental spreadsheets.
For a supplier to provide true metrics, particularly for something like a leadership development course, they must:
- Define Target Behaviours: Clearly identify the specific behaviours the course aims to affect (e.g., assertiveness, active listening).
- Utilise Situational Judgment Tests: Implement these tests to measure behavioural shifts.
- Integrate Business KPI Data: Provide a straightforward mechanism for L&D teams to upload and link relevant business performance data (e.g., employee attrition rates) to learner IDs and training dates.
Mitchell simplified the data requirement, stating that only four columns of data are needed to measure business impact for any metric, for any course: Learner ID, Date of Metric, Numerator, and Denominator. This seemingly simple structure, when integrated with an LMS, allows for powerful pre- and post-training analysis, moving towards establishing causation rather than mere correlation.
A Vision for the Future: Centralised Outcome Repositories
Looking ahead, Mitchell envisioned a "centralised outcome repository" – a normalised dataset of what training worked (and didn't work) across different companies, topics, and seniorities. Such a repository, he argued, would be invaluable, providing EdTech providers with an API into data that informs the creation of highly effective, evidence-based learning experiences. This would not only increase business productivity but also enhance the learning experience for individuals globally, even impacting education in developing economies.
Measuring Experience: Five Key Questions
To standardise the measurement of learning experience, Derek Mitchell proposed five simple yet powerful questions:
- Describe the training in a word: Elicits an adjective, enabling sentiment analysis and comparison of courses.
- How confident are you in the topic? A strong predictor of actual behavioural and KPI change at a population level.
- Why do you say that? Provides qualitative data for thematic analysis, revealing factors influencing confidence.
- What will stop you applying what you've learned? Identifies practical barriers (e.g., lack of time, IT equipment), informing business interventions beyond L&D.
- Is there anything else you wanted to tell us? An open-ended question for general feedback.
These questions, when combined with KPI data, allow organisations to quantify the impact of barriers and understand what truly drives performance improvement.
The Path Forward for L&D
The discussion concluded with a powerful call for change in the EdTech arena. L&D professionals looking to invest in a learning management system or content creation platform should demand systems that can connect with their business KPI data. While building in-house solutions is an option, the ideal scenario involves a collective industry shift towards standardised measurement of experience and impact.
The era of merely tracking enrolments and completions is over. AI is not just a tool for content generation; it is a catalyst demanding a complete re-evaluation of how L&D demonstrates its value. By focusing on measurable behavioural change and direct business outcomes, supported by intelligent data analysis, L&D can secure its place as a strategic driver of organisational success.
Ready to transform your L&D strategy with AI-powered solutions that prove real ROI?
Discover how Open eLMS LMS can centralise your learning, track skills, and integrate with your business KPIs for unparalleled insights. And unleash the power of rapid, personalised content creation with Open eLMS Learning Generator, our AI-powered e-learning tool.
Visit www.openelms.com to explore our one-supplier solution for modern workplace learning.