Revolutionizing and Scaling Experimentation & Feature Management
The Challenge
Optimizely’s Web experimentation product was widely adopted for its simplicity, but the transition to the more powerful Full Stack platform presented significant barriers for customers. The fragmented user experience and steep increase in technical complexity prevented them from effectively scaling their experimentation practices.
Initially, leadership proposed a migration tool to bridge this gap, but I recognized early on that this would reinforce rather than resolve the underlying friction. The real challenge was to fundamentally rethink Full Stack experimentation—making the platform more accessible, streamlined, and scalable from the outset.
The Strategy
Based on extensive user research and market analysis, I led a strategic pivot away from the migration approach toward a complete reimagining of feature management and experimentation. Several critical insights shaped my strategic direction:
- Declining usage of Web Experimentation and increasing adoption of Full Stack indicated a shift in customer needs and expectations
- Technical complexity and accumulating debt were major pain points for customers
- Effective collaboration between technical and non-technical team members was lacking
Our strategic approach centered on four objectives:
- Rapid Time-to-Value: Accelerating the pathway from sign-up to meaningful results.
- Scalability: Allowing high-volume enterprise users to increase experimentation volume without technical debt
- Simplicity: Reducing complexity in both UI and underlying concepts
- Cross-functional Collaboration: Enabling seamless interactions among diverse team roles
I also established secondary goals including clarifying information architecture, better representing the experimentation lifecycle within the product, and incorporating meaningful moments of delight that could be measured and improved.
The Process
To kick off the project (internally named “Project Ozone”), I led a comprehensive design sprint alongside our UX researcher. Actively participating in every session, I guided the team through brainstorming, whiteboarding, ideating solutions, and early-stage prototyping. My hands-on involvement set clear expectations for the direction, provided direct mentorship, and demonstrated practical approaches to complex UX challenges.
Our deep-dive user research across startups, SMBs, and enterprises, covering multiple roles (from developers to marketing leaders), surfaced key findings:
- Teams struggled with cross-functional collaboration
- Technical debt rapidly accumulated with increased experimentation
- Transitioning from Web (for less technical users) to Full Stack (for developers) was unnecessarily painful
This foundational research informed a significant strategic pivot that I championed after securing buy-in from our Chief Product Officer. We reconceptualized feature flags—not merely as toggles but as essential decision points that could flexibly support various experimentation types (A/B tests, multivariate tests, multi-armed bandits). This innovative approach dramatically reduced technical debt by enabling easy conversion of winning experiments into simple variables, eliminating experimental overhead.
One critical UX innovation I personally contributed was a three-panel approach for configuring flags. This design provided users a centralized space to manage all aspects of a flag seamlessly, maintaining clarity without clutter or overwhelm—directly addressing user pain points identified in research.
Additional improvements developed by the team under my direction included:
- Streamlined navigation to increase access speed to common features
- Enhanced user permission models with abstracted system entities
- Implementation of a flexible taxonomy system to overcome project structure limitations
- Unified help resources to improve user self-service
I oversaw iterative prototyping and user testing sessions, working hands-on with the design team in Figma to refine the experience until we achieved user flows that satisfied all stakeholders.
The Results
I was deeply involved in the implementation and rollout of this fundamental reimagining of feature flags and experimentation. One of my biggest contributions was rallying stakeholders to understand the goal, problem, and approach, while strategizing how we could iterate on the solution to balance customer value with other roadmap priorities.
I directed a phased implementation over quarterly iterations, ensuring key customers received early access after the second quarter release. This approach resulted in overwhelmingly positive feedback and an NPS increase of nearly 2 points. The validation provided momentum for further streamlining in subsequent releases, cementing Optimizely's position as the innovation leader in experimentation platforms.
The project successfully transformed how teams implement experimentation, reducing technical debt, improving cross-functional collaboration, and making sophisticated testing accessible to more users – precisely addressing the critical challenges I identified at the project's outset.