Designing learning for engineers who build spacecraft requires a unique approach, as Firefly Aerospace's Learning Designer, Aurelia O'Neil, revealed in a recent "Future of Learning" podcast from Open eLMS. She highlighted the surprising extent of on-the-job training necessary for such a high-stakes environment, where hands-on application and project-based learning are paramount. This insight, shared alongside Firefly's Lead Technical Trainer Johnny Mendias, underscores the critical need for robust, data-driven learning strategies that move beyond traditional course completion metrics to ensure mission success.
Key Takeaways from the Discussion:
- On-the-Job Training is Critical: For high-stakes industries like aerospace, hands-on, project-based learning and simulations are crucial, often surpassing formal online courses.
- Talent Architecture as an Ecosystem: Moving beyond a "course factory" to a holistic learning ecosystem that blends approaches, focuses on governance, and provides ongoing support through departmental trainers and coaches.
- Data-Driven Decisions are Paramount: Collecting granular skills data from every department and using tools like skill matrices and "rubric centres" to identify gaps, critical risks, and inform targeted training.
- Rubrics for Objective Assessment: Shifting from subjective "confidence ratings" and simple completion counts to structured rubrics with proficiency legends, observable behaviours, and non-vague descriptors to accurately measure capability and inform quick operational decisions.
- AI as a Quality Assurance Tool: Exploring AI's potential not just for content generation, but as a "soundboard on quality" to review and validate learning outputs, trained on instructional design principles and theories.
- Human Oversight is Non-Negotiable: While AI can enhance efficiency and speed, human judgment remains essential for alignment, complex assessments, and critical analysis of AI-generated content, especially in risk-averse environments.
- Cultivating a Learning Culture: Fostering cross-functional collaboration, aligning L&D vision with company mission, and creating psychological safety by separating trainers/coaches from evaluators to encourage learning from mistakes.
- Strategic Design for Pressure: Layering standards, creating modular learning, and embedding safety directly into tailored course designs to respect productivity time and ensure quality learning in fast-paced, high-consequence settings.
From Rocket Science to Learning Science: The Firefly Approach
Firefly Aerospace, a company that designs, builds, and tests rockets, lunar landers, and spacecraft, operates in an environment where getting training right directly impacts hardware integrity, safety protocols, and mission success. This unique context demands a highly adaptable and precise approach to corporate training.
Aurelia O'Neil, with a background in university-level educational technology, noted the significant shift to on-the-job training at Firefly. Here, learning is deeply embedded in hands-on application and project-based simulations, reflecting the practical demands of building cutting-edge aerospace technology. Johnny Mendias, an engineer who transitioned from testing rocket engines at SpaceX to leading technical training at Firefly, emphasised his role in bridging the gap between the technical demands of the manufacturing floor and the strategic needs of the human resources department. This dual perspective is crucial for developing efficient and relevant learning solutions.
Building a Data-Driven Learning Ecosystem
Firefly's approach to learning and development extends far beyond a traditional "course factory." Aurelia described their "talent architecture" as an entire ecosystem, blending online modules for foundational knowledge with extensive hands-on application. This ecosystem is not only about content development and delivery but also about robust governance and sustained support. Departmental trainers and coaches play a vital role in guiding employees through their learning journeys.
A core tenet of Firefly's strategy is the belief that "we are only as strong as the data that we're able to collect to make our informed decisions." With 20 to 30 departments and up to 90 unique work centres, Firefly has meticulously collected skills data from every single area. This data feeds into a comprehensive skill matrix, providing clear visibility into existing skill gaps and identifying critical risks that require targeted training interventions. This level of granular insight is precisely what a modern learning management system (LMS) like Open eLMS is designed to support, offering custom reporting and Power BI integration for advanced analytics.
The Power of the Rubric: Beyond Completion Rates
A key innovation discussed was Firefly's move away from simply counting course completions, which Aurelia described as "misconstrued data." In an industry where errors can lead to severe consequences, leaders need accurate, real-time information on process coverage and individual capabilities. Traditional "confidence ratings" by mentors, while well-intentioned, were found to be subjective and biased.
To address this, Firefly developed "OG rubrics." These structured assessment tools feature a proficiency legend (typically one to five), criteria derived from subject matter experts' skills, and clearly defined, observable behaviours. The descriptors within the rubric are designed to be non-vague, moving beyond generic feedback like "did great" to specific, measurable observations. For instance, achieving a level five proficiency might mean an employee not only performs a task flawlessly but can also teach it to colleagues. This objective approach, supported by a comprehensive LMS for tracking and reporting, allows for precise identification of strengths and weaknesses, informing rapid, data-backed decisions about personnel deployment and future training needs.
AI as a Quality Co-Pilot, Not Just a Content Creator
While much of the AI conversation in corporate training focuses on speed and content generation, Aurelia's current project explores a different facet: can AI assess the quality of what's been built? She is developing an AI agent to review the quality of training rubrics, demonstrating a commitment to leveraging AI for quality assurance rather than solely for rapid content creation.
Aurelia explained that training such an AI agent involves feeding it instructional design principles, learning theories (like Bloom's taxonomy), and specific success criteria. The goal is to enable the AI to perform a self-review of its own output, reducing the human effort required for validation. Johnny Mendias cautioned that while AI is a powerful tool for expediting processes and scaling content, its output should never be taken at "face value." In Firefly's risk-averse culture, every AI-generated suggestion is scrutinised, reinforcing the critical role of human subject matter experts in validating information. This perspective aligns with Open eLMS's philosophy, where AI-powered tools like the Open eLMS Learning Generator are designed to enhance and accelerate human creativity, not replace it, ensuring quality and accuracy in e-learning course creation.
Cultivating Experts and Retaining Knowledge
Firefly's journey also highlighted the importance of nurturing internal expertise. Johnny Mendias discussed the "train the trainer" programme, which addresses the challenge of converting experienced engineers into effective mentors and teachers. A key insight was the need to differentiate between a subject matter expert, a mentor, and a leader. Firefly established two distinct career paths: a leadership path and a contributor path. Trainers thrive in the contributor path, free from the protocols and priorities that can restrict innovation and passion for teaching.
Crucially, Firefly ensures that coaches and trainers are not in a position to evaluate those they are teaching. This separation fosters psychological safety, encouraging employees to make mistakes and learn without fear of impacting their performance reviews. This model ensures that invaluable "tribal knowledge," which was once lost when experienced staff moved on, is now standardised and retained through structured training and the use of rubrics, thereby improving employee training platform effectiveness.
Designing for High-Stakes Environments
Operating under constant delivery pressure, Firefly's learning design is inherently strategic. Aurelia O'Neil detailed how learning is designed to be efficient, respecting employees' productivity time. This involves layering standards, consolidating skills into thematic modules, and creating modular, "plug-and-play" learning pathways. Safety training, rather than being delivered in standalone, generic courses, is embedded and tailored directly into specific job functions and activities, making it more relevant and impactful.
The discussion reinforced that in environments where mistakes have severe consequences, human judgment remains non-negotiable, particularly for ensuring the alignment of learning objectives with assessment methods. The collective support and adaptability within Firefly's culture, coupled with a deep focus on learning from past "non-conformances," drive continuous improvement in their corporate training strategy.
Open eLMS, established in 2003, understands the complexities of corporate training in high-pressure environments. Our AI-powered LMS and learning solutions, including the Open eLMS Learning Generator and Open eLMS OS, are designed to empower organisations to build robust, data-driven learning ecosystems. We offer a one-supplier solution for learning and development, helping L&D managers, HR directors, and business leaders to improve employee performance, reduce training costs, and ensure compliance.
Empower Your Learning Strategy with Open eLMS
Are you looking to transform your corporate training from a course factory into a dynamic, data-driven talent architecture? Open eLMS provides the tools and expertise to build a learning ecosystem that drives measurable outcomes.
Discover how Open eLMS can help you:
- Create engaging, AI-powered e-learning courses in minutes with the Open eLMS Learning Generator.
- Manage personnel, track skills, and generate custom reports with the comprehensive Open eLMS LMS.
- Access over 600 CPD-accredited courses from the Open eLMS eLearning Store.
Visit www.openelms.com to explore our solutions and partner with an EdTech leader committed to your success.