← Selected work · All case studies · How I work
Making experimentation a way of working
Ulta Beauty’s published case study reports growth from 20 to more than 65 tests a year. I led the work of making good ideas easier to prioritize, deliver, and learn from.
My role: Led the Digital Optimization Team’s strategy, operations, stakeholder relationships, and vendor partnership.
Team: Product, UX, Design, Development, Analytics, and business stakeholders.
The full story
The work behind the tests
As Ulta Beauty’s appetite for experimentation grew, ideas needed a clear route from hypothesis to a decision—and someone responsible for what happened after the test.
Give each idea a useful starting point
I introduced problem-and-solution mapping, a prioritization matrix, and team voting on test candidates. In weekly planning, we compared customer problems, business goals, visibility, and effort. Team members helped shape both the work and how we delivered it.
I maintained relationships across product, UX, merchandising, loyalty, and customer experience. Those conversations helped connect our testing work to the questions the business actually needed answered.
Make the work repeatable
I helped establish agile practices and documented the route through requirements, design review, development, quality assurance (QA), launch, and analysis. During the transition from Qubit to AB Tasty, I led the team and managed the day-to-day vendor relationship. A more flexible platform mattered; so did the working practices around it.
I created a shared process map to make that route visible: who owned each step, where a decision was needed, and when another kind of research would be more useful than an A/B test. It connected the work before intake with development, launch, analysis, and the handoff into production.
Make room for support
One useful piece of feedback from my manager was to ask leaders for help when direction was unclear or the team was facing challenges. The growing pains were not all mine to carry. A useful process also makes it clear where support is needed.
Design the handoff as carefully as the launch
A test result needed an owner and a next step. The recommendation overlay made the cost of an unfinished handoff especially clear. We kept the winning experience live through AB Tasty while waiting for production implementation. Unannounced API changes could break the patch, and bug reports sometimes became our first warning. Repairing it pulled my small team away from new experiments.
I developed a more rigorous patch audit process with Product: monthly check-ins to review temporary experiences, follow up on roadmap commitments, and move proven changes off the testing tool and into production. The work after a test needed to be visible, too.
At our peak, we were managing about 35 patches simultaneously. By the time I left Ulta Beauty, none remained. Regular audits and follow-through with Product helped bring that ongoing maintenance workload to an end.
Published Ulta Beauty case study
Before we built a test
Bring the idea forward. Complete the DOT intake form, then bring the right partners into a kickoff.
Make the work visible. Gather requirements and development estimates, and share both across the team.
Make a decision. Decide whether the idea is worth A/B testing before committing to the build.
Our original intake flow made requirements and effort visible before development began.
Intake detail: requirements, estimates, and the decision to test. Open the original image →
What happens after a test ends?
The conclusion flow made the next decision explicit: share what we learned, decide whether a winning experience should move into production, and review temporary experiences with product partners. It also left room to pause or iterate when the evidence called for it.
Conclusion detail: learning, iteration, and ownership after the test. Open the original image →
Explore the full DOT experimentation process (PDF) →