By Andrew Park | 2024-10-31
This article is based on a discussion I had with Adam Ellsworth, a Senior Product Manager at Lucid Software. Adam has extensive experience in cross-functional product development, market research, and M&A integration from his roles at Verisk and Endurance International Group. At Lucid, he drives product innovation and strategy for SaaS solutions, aligning product teams with business goals.
The Hidden Challenge: Misaligned Expectations in Agile and Scrum
A key reason Agile and Scrum frequently fail is the implicit assumption that engineers already possess strong product and domain knowledge. This assumption overlooks a critical gap: many engineers are trained primarily in technical execution (the how), rather than understanding the product’s purpose (the what and why). Adam explained it this way:
“Agile assumes engineers have the maturity to understand product context, but in many cases, the reality is different. Without that context, it’s hard to deliver what’s really needed.”
This lack of context can lead to suboptimal design choices, misalignment, and wasted effort.
Standard Agile and Scrum frameworks expect seamless collaboration between product and engineering teams. However, when engineers don’t grasp the broader product context, product managers are forced to provide exhaustive details in user stories, manage dependencies, and clarify goals at every step. As Adam noted:
“It often feels like product managers have to babysit the process, filling in every gap for engineers who don’t have the necessary product background.”
This diverts product managers from strategic work, such as product discovery and customer engagement, as they become bogged down by tactical details.
The Traditional Approach: Overemphasis on Technical Skills
During our conversation, Adam and I both agreed that the traditional approach to hiring and developing engineers tends to overemphasize technical skills—coding abilities, problem-solving techniques, and specific programming languages—at the expense of broader product knowledge. Adam emphasized:
“Most job descriptions for engineers focus only on tech skills. But if engineers don’t understand the product’s why, they’re just coding in the dark.”
This focus reinforces the notion that engineers are responsible solely for the how of product development, not the what or why. As a result, engineers may struggle when expected to understand the product’s strategic goals or the customer’s needs.
This disconnect is problematic in Agile and Scrum, which aim to foster collaborative work environments. Product managers often spend excessive time detailing tasks, managing dependencies, and resolving misunderstandings—time that could be better spent refining the product vision and engaging with customers.
A New Approach: Hiring for Broader Product Knowledge
Adam observed that the engineers he has interacted with at Lucid generally possess a higher degree of product domain knowledge compared to other companies he has worked for. While it’s not necessarily a formalized part of Lucid’s hiring or development processes, there are cultural efforts that encourage engineers to accumulate product knowledge.
“Lucid’s engineers tend to have a better sense of what the product is trying to achieve,” Adam noted. “That’s made a big difference in how we collaborate. It feels more like a true partnership.”
Engineers who understand the what and why of the product can make more informed decisions, aligning their technical work with strategic goals. This shift has impacted the dynamics of collaboration, resulting in:
Contextual Understanding: Engineers are better equipped to make informed decisions that align with both the what and the why of the product, reducing misinterpretations and the need for micromanagement.
Reduced Micromanagement: Product managers can shift from managing minute details to focusing on broader strategy, thanks to engineers’ greater familiarity with product goals. As Adam explained, “When engineers get the bigger picture, you don’t have to hold their hand every step of the way.”
Improved Scalability: The approach fosters a more sustainable workload for product managers, as engineers contribute more meaningfully to product discussions.
Cultural Shifts: From Execution to Strategy
Reimagining Agile and Scrum isn’t just about changing processes—it’s a shift in culture. It requires a rethinking of how product and engineering teams interact, with a focus on shared understanding and collaborative problem-solving. By hiring engineers who can comprehend the broader context, organizations can achieve:
Better Decision-Making: Engineers make design choices that align with product strategy, reducing rework and inefficiencies.
Empowered Product Managers: Freed from micromanaging tasks, product managers can focus on strategic activities, such as discovery, customer insights, and market analysis.
More Collaborative Environments: Product teams work more effectively together, with engineers contributing meaningfully to product discussions and product managers focusing on high-value tasks.
Adam described the transformation this way:
“When engineers understand the product’s mission, it creates a sense of ownership. It’s no longer just about shipping features; it’s about achieving product success together.”
Achieving Sustainable Success
For Agile and Scrum to be truly effective, organizations need to rethink how they cultivate talent and structure collaboration. This evolution includes:
Revamping hiring processes to emphasize product and domain knowledge alongside technical skills.
Training engineers to understand product strategy through direct exposure to customer insights and business objectives.
Creating informal communication channels that enable flexible, real-time collaboration rather than rigid documentation of tasks.
Implementing these changes can unlock the full potential of Agile and Scrum, creating a more scalable and sustainable work environment for product teams. This results in faster development, better products, and higher team satisfaction.
Final Thoughts: Introducing Product Oriented Software Engineering (POSE)
While Agile and Scrum were designed to enhance collaboration, they must evolve to meet today’s complex product development landscape. This is where Product Oriented Software Engineering (POSE) comes in—a framework I established as a Product Leader and Engineering VP in 2006. POSE not only leads to better products but also significantly reduces workloads for product managers by transforming software engineers into product engineers. You can learn more about POSE in my upcoming book, Product Oriented Software Engineering, here.
As Adam put it, “When engineers start thinking like product people, the whole team wins.” Companies like Lucid demonstrate that culturally encouraging engineers to accumulate product knowledge can create a more effective, collaborative, and sustainable framework for innovation and long-term success.