The world of workplace learning is constantly evolving, with new technologies promising revolutionary shifts. Yet, amidst this rapid change, a fundamental question persists: does any of this actually work? This critical inquiry formed the bedrock of a recent "Future of Learning" podcast discussion from Open eLMS, featuring host Andy Howie and Dr Robert Brinkerhoff, a distinguished expert in programme evaluation from Promote International and the Brinkerhoff Evaluation Institute.
For over four decades, Dr Brinkerhoff has challenged conventional wisdom in learning and development (L&D), advocating for a profound shift in how organisations assess the true impact of their training initiatives. His insights, born from extensive experience evaluating programmes for entities ranging from the US government to global corporations, suggest that the success of training lies less in the content itself and more in the intricate system surrounding it.
Key Takeaways from the Discussion:
- Beyond Averages: Relying on average training results obscures both exceptional successes and critical failures, hindering genuine understanding of impact.
- The System Matters Most: Approximately 80% of training effectiveness is determined by "before and after" factors, such as preparation, management support, and post-training application, rather than the training content itself.
- Managerial Influence: A manager's actions, or inactions, can swiftly amplify or negate weeks of training.
- AI for Performance: While AI offers immense potential for "learning in the flow of work" and just-in-time support, its true value is realised when focused on improving performance, not merely generating vast amounts of content.
- Rapid Evaluation: Effective evaluation should be quick, continuous, and centred on observable behaviour change, allowing for immediate adjustments and improvements.
- Value Over Activity: L&D leaders must transition from selling training programmes and hours to demonstrating tangible value and solving business problems.
The Flaw in Averages and the Power of Story
Dr Brinkerhoff's journey to developing the Success Case Method began with a startling observation: identical training, delivered by the same facilitators with the same materials to different parts of an organisation, yielded wildly divergent results. Some teams excelled, while others gained nothing. This led him to question the validity of averaging results, a common practice that often masks critical insights into corporate training effectiveness.
He recounted an early experience in the New York City public school system, observing teachers using a new reading methodology. One teacher demonstrated brilliant application, while others had not even opened the training materials, using the box instead as a stand for a classroom aquarium. Averaging these experiences, Dr Brinkerhoff realised, would paint a misleading picture of the programme's efficacy, failing to highlight both successes and areas for improvement in the learning management system (LMS) implementation.
Further reinforcement came from a report to the US Congress on a Native American development programme. Despite a thick report filled with statistical data, it was the compelling personal story of a programme participant, who had become a brain surgeon, that moved Congress to approve funding. This highlighted the profound impact of storytelling in conveying value and driving belief, a lesson that became central to his evaluation approach for employee training platforms.
The Success Case Method: Digging Deeper
The Success Case Method deliberately rejects the "survey and average" approach. Instead, it focuses on the "tails of the distribution," examining both the most successful applications of training and instances where it failed to make an impact. This involves asking critical questions: when training works, what good does it do? How often does it work? How and why does it work? Who makes it work? And when it does not work, why not?
By interviewing those who achieved significant success, L&D professionals can uncover what factors enabled their positive outcomes. Conversely, speaking with those who struggled reveals systemic barriers. Dr Brinkerhoff shared an example from the postal service, where employees trained to maintain complex sorting machines made "zero use" of their new skills because their union forbade them from touching the equipment. Such insights are invaluable for refining training delivery and ensuring relevance, a key aspect of effective corporate LMS solutions.
Beyond the Training: The System's Crucial Role
A key insight from Dr Brinkerhoff’s extensive research is that the training itself is rarely the sole determinant of success. He estimates that the "before and after" components of a training programme – how it is prepared for, implemented, supported, and followed up on – account for approximately 80% of the variance in its success.
This places significant emphasis on the organisational system in which training is embedded. A manager, for instance, can undo two weeks of training in mere minutes through lack of support or contradictory directives. Conversely, a supportive manager who provides coaching and feedback can turn training into a valuable asset. This underscores the need for L&D to work hand-in-hand with leadership to create an environment conducive to learning transfer and application, a critical factor for any modern employee training platform.
AI in Learning: A New Chapter, Old Lessons
The discussion naturally turned to the advent of AI in L&D. Dr Brinkerhoff drew parallels to previous technological shifts, such as computer-based training and the internet, noting that while the technology changes, the fundamental challenges of adoption and impact remain similar. People need to understand not just what AI is, but what it can do, its risks, and why, when, and where to use it.
AI, he argued, can significantly enhance "learning in the flow of work," echoing the traditional apprenticeship model where learning happens organically during tasks. This directly addresses the "transfer problem," where up to 80% of learning acquired in formal settings fails to translate into workplace practice. Just-in-time, just-enough learning, facilitated by AI, can provide instant support precisely when and where it is needed, making it a powerful tool for modern e-learning.
However, Dr Brinkerhoff cautioned against using AI merely to generate a "heap of content." The risk, he noted, is that cheap and fast content production could lead to "quantity over quality," bludgeoning learners with information that has no tangible impact on performance. The focus must remain on leveraging AI to impact performance, not just to create learning content. The Open eLMS Learning Generator, for example, is designed to create custom eLearning courses, videos, podcasts, and gamification in minutes, but its true power lies in its ability to quickly produce targeted, relevant content that addresses specific performance needs, fostering a culture of continuous improvement and positioning Open eLMS as a leading AI course creator.
Measuring Impact: Focus on Behaviour Change
For L&D leaders, the imperative is clear: move beyond measuring activity (hours trained, programmes delivered) to measuring tangible behaviour change and performance improvement. Evaluation, Dr Brinkerhoff stressed, should be rapid and continuous, not delayed by months or years. He suggested that after the first cohort of learners completes a programme, L&D should immediately assess if behaviours are changing.
The focus should be on identifying early adopters – those 15-20% who quickly implement new skills – and understanding what enabled their success. Their stories and methods can then be used to inspire and support others. Similarly, understanding why some individuals struggle allows for prompt adjustments to the training or the supporting system. Tools within the Open eLMS LMS, such as Learning Tracking, Skills Monitoring, and Custom Reporting with Power BI integration, offer L&D professionals the granular visibility and analytical power needed to track behaviour change, identify skill gaps, and measure the real impact of their interventions, making it an ideal L&D platform for skills management and compliance training.
Shifting L&D's Focus: From Activity to Value
Dr Brinkerhoff challenged L&D professionals to change how they "sell" training to managers. Instead of presenting programmes, they should initiate conversations by asking about business problems: "How consistent is job performance in your unit? What's slowing down work? What's impeding productivity?" The goal is to identify a problem that training can genuinely help solve, thereby selling value, not just features of an online learning software.
He lamented that the L&D system often rewards content creation and hours in training, rather than actual results. This perpetuates a cycle where training is delivered without sufficient consideration for its real-world application or impact. The industry, he concluded, has gotten better at designing programmes, but must now focus on engineering the performance environment to elicit and reward desired behaviours.
A Journey, Not an Event
The COVID-19 pandemic, for all its challenges, inadvertently pushed L&D to rethink learning as a continuous journey rather than a one-off event. This shift, which Dr Brinkerhoff believes is crucial, allows for guided processes of learning and application, moving away from "sheep-dipping" learners in content for a few days and then leaving them to their own devices.
Ultimately, Dr Brinkerhoff encouraged L&D professionals to embrace curiosity. "Be curious, ask questions... I wonder what's happening with the people who took it and go find out." This simple yet powerful advice underscores the need for continuous inquiry and adaptation in the pursuit of truly impactful learning.
By adopting an approach that prioritises understanding the entire learning ecosystem – from initial preparation and management support to ongoing application and rapid evaluation – organisations can move beyond superficial metrics and unlock the genuine potential of their L&D investments.
Transform Your L&D Strategy with Open eLMS
Are you ready to move beyond averages and unlock the true impact of your training programmes? Open eLMS offers a comprehensive, one-supplier solution designed to support a performance improvement journey, not just isolated training events. As a leading Edtech provider, we empower L&D managers and HR directors to achieve measurable results.
- Measure Real Impact: Leverage Open eLMS LMS with its robust Learning Tracking, Skills Monitoring, and Power BI-integrated Custom Reporting to gain deep insights into behaviour change and training ROI. This powerful corporate LMS helps you demonstrate tangible value.
- Create Engaging, Relevant Content: Utilise the AI-powered Open eLMS Learning Generator to rapidly create just-in-time, just-enough courses, videos, and interactive content tailored to specific performance needs. Our AI course creator transforms content generation, saving you time and resources.
- Streamline Your Operations: Centralise all your learning and development activities, from personnel management and live learning to compliance training with our 4,200+ CPD-accredited courses from the Open eLMS eLearning Store. This comprehensive employee training platform ensures all your needs are met in one place.
Discover how Open eLMS can help you build a system that supports continuous performance improvement and delivers measurable value. Visit https://www.openelms.com to learn more about our AI-powered learning management system.