Strategic product research: Vision validation & design guidelines
Researchers: M. Pederson & L. Westra
CASE STUDY
Background & Problem
The client’s executive leadership team completed a multi-month visioning exercise to define a three-year roadmap. The roadmap aimed to unify disparate products while significantly expanding capabilities for software engineering teams.
Before moving forward, leadership needed to validate the roadmap with top customers. At the same time, product and design were exploring how to unify legacy and new capabilities within a single ecosystem. To advance the initiative, the organization needed clarity on how to align products, workflows, and stakeholders around a shared vision.
Research Questions & Methodology
Our goal was to identify strengths, gaps, and risks within the client’s three-year roadmap from the perspectives of engineering leaders and end users. Our research questions were:
How does the existing 3-year roadmap align, or not align, with the ways engineering leadership anticipates their company’s technology portfolio evolving?
How do engineering teams currently work with the company’s various products? What other tools do they currently use to complete their work?
What are the potential barriers for software developers to engage with a unified development experience?
What are users’ context-specific goals, needs, and pain-points when interacting with a unified experience?
How does their behavior change as the context changes?
Given the complicated nature of the project, we completed the work in two separate phases, using a mixed methods approach.
Phase One: Strategic Validation
We conducted in-depth interviews with engineering leaders from the client’s top customers. To broaden our perspective, we also performed a comprehensive review of documentation and recordings from the customer support team.
Phase Two: Workflow & Experience Design
We conducted a moderated and unmoderated card sort to understand how developers conceptualize their tools. We then ran a moderated and unmoderated tree test to arrive at an initial information architecture for the unified experience. In total, 48 software engineers participated in the study.
After analyzing each phase independently, we synthesized the findings into a cohesive set of strategic recommendations.
Key Insights & Impact
Our research validated the overarching direction the company was headed. Based on customer pain points, we highlighted the areas of the roadmap the company must immediately focus on, and recommended areas to deprioritize. We were able to provide the company with a refined, customer-driven version of their vision and roadmap. The full report cannot be shared for confidentiality reasons, but here are two samples of vision slides.
Vision Revision: Decision makers for enterprise customers want an automation system that is adaptable, scalable and secure.
Not a single individual contributor participant asked for a tool that will make their work more adaptable, scalable and secure. Developers want an automation tool that affords them a dedicated, safe workspace to write code, so they do not have to worry about adaptability, scalability, and security.
Significant Research Findings Sample
Significant Research Findings Sample
Vision Revision: Customers are seeking an adaptable, scalable, secure invisible platform. In simple terms, the platform is the code that enables automation for customers’ repeatable tasks, including integrations and pieces of automation. The ideal solution allows customers to select their preferred third-party integrations, and customize how the system pieces come together.
Significant Research Findings Sample
In addition to the refined vision, we identified and built out the behavioral archetypes that would work within the client’s unified product suite. We provided information on each archetype as well laid out as the archetypes that would need the most design accommodation. The product and design teams were the primary consumers of this section of the report. Here are samples of two of the overview slides. Specific product capabilities and proprietary information have been removed from the diagrams.
Confident Incompetent
I think I am an expert in everything, but in reality I only have deep knowledge of my own silo.
Fast & Frustrated
I move quickly through the experience and will leave when I cannot figure out the tool.
Logical & Methodical
I read tool tips, help docs, and keep trying until I figure it out.
Autonomy Deficient
I am forced to be the human connection when automation is not working as intended.
Booksmart
I understand theoretical best practices, but less about how to get the job done in real life.
Skeptic Expert
I know theoretical best practices, and actually how to get the job done.
Significant Research Findings Sample
Capability 1
Build in ways to keep these folks from unintentionally messing up.
Avoid including extra components on the UI that are not related to the primary task.
Roles: Application developers, highly technical shared services individual contributors
Confident Incompetent
Capability 2 & 3
Fast & Frustrated
Provide a non-code view to address the blocker or access to make dev-done, done-done.
Capability 1 roles: Release managers, product managers, shared services individual contributors
Capability 2 roles: product managers, app dev managers
Capability 4
Fast & Frustrated
Direct access to issues and direct, actionable next steps.
Tools to get to the deeper data to do their own analysis.
Tool lives up to the sales pitch.
Roles: Shared services, product managers, app dev managers, engineering leadership
Autonomy Deficient
Booksmart
Skeptic Expert
Finally, we provided the client with an initial information architecture for the unified experience as well as product value statements for both buyers and users. Samples are not shared to protect client confidentiality. We also suggested areas for further exploration and testing throughout the design and build process. Based on the key insights and artifacts provided, the client was able to successfully design, build and launch their new product.