Strategic product research: Good developer experience

Researchers: M. Pederson & L. Westra

CASE STUDY

Background & Problem

The client, a business software company focused on development tools for software engineers, had one successful product line and plans to expand its portfolio. While teams across the organization had deep experience with developers—many were developers themselves—knowledge was fragmented and inconsistent.

Product and design leadership recognized that critical decisions were being shaped by a narrow set of customer voices and internal preferences rather than representative data. To expand confidently into new product lines, the company needed a broader, more rigorous understanding of developers—their workflows, decision-making influence, and expectations.

Research Questions & Methodology

Our objective was to define what “developer experience” and “developer-first” should mean within the context of the client’s current and future capabilities.

We focused on four core questions:

  • How should the client approach developer experience from a product strategy perspective?

  • What differentiates “good” from “great” developer experience?

  • How do developers influence software purchase decisions?

  • Which developer segments represent the highest strategic value for the new product lines?

We used a mixed-methods approach:

  • Review of third-party research on developer workflows and preferences

  • 30 in-depth interviews with software engineers across target markets

  • A follow-up survey to validate qualitative findings (n = 512, representative sample)

  • Parallel survey with internal developers (n = 40) to compare internal assumptions with external realities

Qualitative and quantitative data were analyzed independently before being synthesized into a unified set of findings and strategic recommendations.

Key Insights & Impact

Note: This study was conducted prior to the widespread integration of AI into developer workflows.

We defined “good developer experience” specifically for the client’s product ecosystem and translated that definition into clear prioritization guidelines for the roadmap. This clarity unlocked alignment across product and design as well as influenced the go-to-market motion.

While detailed findings remain confidential, the following slide illustrates the level of insight used to support strategic recommendations.

Significant Research Findings Sample

The order participants ranked their definitions of “good developer experience” varies slightly based on years of coding experience, but the categories they chose are consistent regardless of experience. “Understand how the tool is technically operating” is the outlier, but this is understandable as it was chosen by developers who are still learning how the foundations.

Top definition

Years of coding experience

Less than 1 year

Fast - tool runs processes quickly and get what I need

Understand how the tool is technically operating

Active community

Integrates with existing systems

Modern looking UI

1 to 4 years

Active community

Good documentation

Predictable, consistent availability / up-time

Fast - tool runs processes quickly and get what I need

Modern looking UI

5 to 9 years

Active community

Good documentation

Fast - tool runs processes quickly and get what I need

Predictable, consistent availability / up-time

Modern looking UI

10 to 14 years

Good documentation

Active community

Predictable, consistent availability / up-time

Essential information is clear

Integrates with existing systems

15 to 19 years

Good documentation

Integrates with existing systems

Fast - tool runs processes quickly and get what I need

Active community

Modern looking UI

20+ years

Good documentation

Predictable, consistent availability / up-time

Essential information is clear

Integrates with existing systems

Active community

We also painted a nuanced picture of software engineers and how they influence the purchase of new tools. The following graphics are similar to the types of information included in the final report.

Significant Research Findings Sample

Behavioral archetypes focus on our users’ needs, motivations, and pain-points.  Archetypes capture how they think, feel, and act in particular situations or scenarios, so we can make accommodations for potential pain points through empathic design.  The six archetypes below focus on developers’ behavior towards tool changes, and they help define the targets for the new funnel.

Ladder Climber

I learn about new tooling because I want others to view me as the expert.

Coffee Break & Coder

I use social opportunities to learn about new tools, and I am excited to try them when I have time.

Pragmatist

I prefer to use tools that I know inside and out, but I will use different tools if it is required of my job.

Peacekeeper

I prefer using tools that I know, but I will move into new tools if it makes it easier to collaborate with my team.

Tinkerer

I try all the new tools I learn about, but I rarely switch to them as my go-to tool.

Tool Seeker

I enjoy learning about new technology and frequently switch my primary tool to gain the full experience.

Significant Research Findings Sample

We detailed the role of each archetype in the software purchase cycle.

Coffee Break & Coder

I use social opportunities to learn about new tools, and I am excited to try them when I have time.

"I do personally believe that, you know, the best news is to talk to fellow engineers. I have [a] very good relationship with my university alumni….. So, you know, the best thing [is] if they say, 'oh, we tried this too.' And it's a breeze because obviously you sit down with a beer for friends, you're gonna talk about it. And that to me is very good..." Tech Lead, UK

Coffee Break & Coders’ motivations for tool seeking:

  • Social butterfly, seeking out opportunities to spend time with people while also learning about tooling

  • Often an official, or unofficial, leader of the group

  • Comfortable trying new things

Coffee Break & Coders’ decision making bias:

  • Best source for gauging their peers’ feelings on tool changes

  • Strategic impact must be clear before they can make any decisions

  • Support change management initiatives

  • Will have a good handle on the latest tool trends

Considerations for working with Coffee Break & Coders

Appreciation: Coffee Break & Coders will bring the team together to make the final tooling decisions and keep people motivated during implementation.

Coping Mechanism: They tend to seek out informal discussions regarding changes. While this may not be premeditated, it can influence the final tooling decision, and lead to the rest of the team feeling like they were not included.

Significant Research Findings Sample

We also broke down influence in the tool process by job function.

The independent proportion of each role was compared using a right-tailed test after the spider plot was created to see if there was a significantly greater number of decision makers or influencers in any one group.  DevOps specialists and AppDev leadership reported making significantly more decisions on tooling for their teams than other roles.  DevOps specialists and AppDev make significantly more decisions for their company tooling than other roles. There were no significant differences between the roles for influential power.

Alt. Hypothesis: DevOps Specialists have significantly more TEAM decision makers than other roles.
AppDev DevOps Leadership AppDev Leadership Architect
Z-scores 3.11 2.28 1.77 3.11
Reject H₀ Reject H₀ Accept H₀ Reject H₀
Alt. Hypothesis: DevOps Specialists have significantly more decision makers FOR TOOLS THAT IMPACT THEIR COMPANY than other roles.
AppDev DevOps Leadership AppDev Leadership Architect
Z-scores 2.62 0.99 -0.01 1.35
Reject H₀ Accept H₀ Accept H₀ Accept H₀
Alt. Hypothesis: DevOps Specialist has significantly more tooling influencers than other roles.
AppDev DevOps Leadership AppDev Leadership Architect
Z-scores 0.36 0.89 0.95 1.82
Accept H₀ Accept H₀ Accept H₀ Accept H₀