Context

Lead Product Designer · 2018–2019

Services: Design & team strategy · Research, testing & test-follow iteration · UI/UX design · Growth experimentation · Cross-functional relationship building and communication · Design systems collaboration

Status: All of these experiments have ended. Some were integrated into the product experience.

Objective

The International Growth team wanted to run tests on their pricing page and checkout flow in order to promote new monthly packages and pricing on the platform.

The goal of these experiments specifically was to create a more targeted package and checkout experience for users in Italy, Spain, and France.

Overview

As our international efforts became more of a focus, the International Growth Team was tasked with hitting an 18% year-over-year growth target in 2019 with expected acceleration in 2020.

In addition to this ambitious goal, the International Growth team had never worked with a dedicated product designer, so the majority of the experiments they had been running were based on copy and engineering improvements.

My contributions

I was always supporting two other teams, so my time had to be spent efficiently when working with this team. The team was almost completely made up of members who were new to the company. The growth director, Senior PM, Engineering lead, content strategist and engineers had all joined the company within the last six months. Additionally, two of the engineers had never worked directly with a product designer before.

Team rituals & processes

Worked with leadership and teammates to develop rituals, cadence workflows, and strengthen communication best practices to help the team better integrate into the company's processes.

Design efficiency

Focused on designing experiences that were good enough to learn from in testing, but not too polished that it slowed down the team's progress.

Cross-functional partnerships

Collaborated and built relationships with the engineering, product management and design systems team partners in order to keep forward progress as efficient as possible.

Experiment planning

Planned experiments by helping to identify key performance metrics, phasing tests to isolate variables and learnings, and developing a thorough understanding of business goals and how to analyze test results.

Business rationale

There was significant customer demand for monthly plans on higher-tier packages in the International market, and we believed that by offering new monthly packages, it would generate incremental new paid seats, increase the basic-to-paid rate, and have a positive effect on AOV by shifting demand for Standard Monthly packages to Advantage or Premier Monthly packages.

The success metric we focused on was the basic-to-paid rate for both existing and new users.

Design strategy & user research

I worked with the team to create some variations on the current user experience for paid plan sign-ups. We needed to answer the following questions in our research:

Do users understand the differences between monthly and annual plans?
Is the toggle component the best way to navigate between the two categories of plans?
Does the 'NEW' label highlight the new packages effectively?
Do users feel they understand the annual saving percentages?

Onsite team training

I traveled to SurveyMonkey's Dublin office for 10 days to work onsite with the team and we ran two sprints (team kickoff, designs, user testing, iteration, development, experiment launch, learnings, repeat).

Experiment metrics

Audience

France, Italy, and Spain

Key performance metric

Conversion — specifically the basic-to-paid rate for existing and new users.

Hypothesis

New monthly packages will generate incremental paid seats, increase B2P rate, and shift demand from Standard Monthly toward Advantage or Premier Monthly — improving AOV.

Next steps

Finance to model results and launch new designs to mitigate the drop in annual package mix.

Experiment results

Our B2P (basic to paid user) upgrade rate was significantly higher in the treatment we tested. Additionally, ARPU (annual revenue per user) went up 9% and refunds went down to 0 (400%).

The next round of iterations would include:

In-product messaging to upsell advantage monthly to advantage annual
Highlighting features such as unlimited responses with advantage annual
Include US test in FR, IT, ES for annual savings on checkout
Add new currencies as we can test other lower B2P markets
Revisit the features for monthly packages to shift mix
Remove standard and rename advantage/premium to just display 2 packages
Increase standard monthly price, test increments

Learnings

Communication is invaluable when working with a team: talk through scenarios, discuss things easily and often, and work with a shared understanding of your goals
Nothing beats working onsite with a team
Clearly prioritize features for experiments to let engineers know what it will take to stop something that's good enough to learn from but not so polished that we're losing time and resources to experiments that aren't winning