Case Study  |  First Time User Experience Redesign

In order to increase Week 1 retention of new Pantheon users, we redesigned the first time user experience (FTUE) to communicate to new users the way Pantheon can solve current workflow challenges. We started with user research to discover the common roadblocks new users encounter, and through prototyping and testing we defined and implemented an approach to help developers more quickly realize the time and effort they can save by using the platform. The project resulted in a 10% key metric improvement.

User Research

Pantheon's primary market segment consists of agency and freelance developers who build websites for clients. We started with (1) interviews with newly registered agency developers to learn why they tried Pantheon; and (2) observation with brand new developers who had never tried Pantheon before. This research helped us define which problem areas we should focus on improving in order to have the most impact.

These are the common roadblocks new developers encounter.

Developers are unsure where to start

We learned most developers want to try Pantheon by creating a new site so they can start experimenting with how the platform works, but it can take a while to find how to do this.

After signing up for Pantheon, developers land on a home screen that doesn't point them to the first action they should take.

Value proposition is unclear

When developers create their site, they choose which CMS to use (Content Management System - on Pantheon this is either WordPress or Drupal).

Developers commented that this screen is too generic, and doesn't help them understand why they'd choose to run their site on Pantheon instead of using another platform or running the site themselves.

Developers are thrown into the deep end...

Once their first site is created, developers land on the site's dashboard. Here it is likely they will be overwhelmed with all of the options to choose from and no direction for what to focus on first.

At this point, developers who have past experience with WordPress or Drupal know they need to finish setting up the CMS, but they get stuck trying to find the way to do so.

We also observed one user trying to commit a code change for 8 minutes before giving up.

...with no guidance

If developers successfully make it past the CMS installation step, they next want to access the code respository so they can start developing. It is unclear to developers how to get to the site code, and we observed several testers give up at this stage.

We also observed several users miss the command they need to paste into their command line in order to clone a local copy of the repository, due to the noise in the dropdown.

Developers have to learn how to use Pantheon's differentiators on their own

Pantheon's out-of-the-box Development, Test, and Live environments and workflow are differentiators according to current customers, but new users are uncertain how to use this workflow and what the best practices are.

Conclusion

Through the interviews and observations with newly registered developers, we concluded that the majority's first goal is to start experimenting on a testing site so that they can learn how Pantheon will improve their own workflow and address the problems they deal with every day.  But all of the blockers we identified come before developers can even finish setting up a local copy of that testing site, meaning many may not even make it to that firsthand experimentation step at all.

Prototyping

In order to remove roadblocks to site development, we created several prototypes to test different ways of showing developers how Pantheon can improve their site development workflows. We also added in feedback loops to make next steps obvious so that developers can move forward unimpinged. I ran point on two approaches (below), while my designer teammate worked on others.

Prototype 1

Goal: Integrate guiding moments to unblock the first user goal

Prototype sample: Add CTA to install the CMS

After users create a test site, developers get stuck trying to find how to install the CMS.
In the prototype, we added a button to prompt testers to install WordPress (or Drupal), instead of relying on them to figure it out.

Prototype sample: Point users towards the next step after WordPress or Drupal is installed

After WordPress is installed, the prototype adds a confirmation and a message guiding them to the next step.

Prototype sample: Help users find what they need by removing unnecessary information in this dropdown.

Developers can then use the new "Clone or Connect" button to access their site code.

To make what they're looking for more obvious, we distilled the dropdown down to the essentials and separated Git vs. SFTP mode so users can select their preferred method of working.

Prototype sample: Confirm success, and point users to a next step

Once testers cloned the code and made a commit, we again reinforced success and suggested a next step.

Prototype 1 Result: We observed large improvements in testers understanding of what to do through success confirmations and guidance to next steps. Sequencing actions together also helped testers start an activity and see it through to completion, instead of dropping off.

Prototype 2

Goal: Teach users how Pantheon can improve their workflow challenges

Prototype sample: This new screen contains a limited set of recommended actions

In this prototype, we tested this new screen that users would land on after they create an account, rather than landing them on the main dashboard where they were overwhelmed with all the options. This screen instead contained a limited set of recommended actions based on the most common user goals we learned during research.

Using users' own reasons, we also highlight Pantheon's value props in the descriptions. For instance, the "Set Up Your Team" option addresses how Pantheon can enable developers to work collaboratively without overwriting others' work, which was a major pain point learned during research.

If users aren't sure what to do, we make a suggestion based on what we know most developers do -creating a new site to test out the product. This line helped several testers decide which option to choose.

Prototype sample: Call out the benefits of running their CMS on Pantheon vs. other hosting solutions

We tested new copy on the Choose CMS screen to communicate the value proposition of using Pantheon to run WordPress and Drupal, and directly address the concerns surfaced by users during the research. Testers found this screen more informative compared with the existing version.

Prototype sample: Explain how to use each environment

Once users arrived at the site dashboard, we updated the environment empty states to describe how each environment fits into Pantheon's workflow and how it should be used. Here, we call out that the Test environment should be used to stage and QA code changes once development is finished.

Prototype 2 Result: This prototype resonated with testers through moments that communicate how Pantheon solves workflow challenges they keenly feel, coupled with actions the tester can immediately perform to move them towards a better process.

Team Collaboration

Throughout this project, I worked with cross-functional teams consisting of engineers, product managers, and another designer

Workshops

Workshop excerpt: Slide created during the discussion.

In a series of onboarding teardown workshops, I led a discussion critically evaluating other products' onboarding flows, and what makes a successful new user experience.

Brainstorms

I also led whiteboarding and brainstorming sessions to come up with ideas addressing the user roadblocks observed from the research.

Product Goal

We took all of the learnings, the best of the prototypes, and rough engineering estimates into account in order to define our approach for the FTUE product update:

The first goal of new developers is to experiment with Pantheon's workflow to learn how it can help them. To help developers meet this goal faster, we should communicate to first time users how Pantheon can solve current workflow challenges and suggest actions for users to take, so that they can make progress in solving their challenges.

New First Time User Experience

The product goal led into a final design pass followed by implementation, resulting in these product updates.

This screen was made in collaboration with designer Nathan Lee

Give users a set of entry points to choose from

Rather than throw new developers into the deep end of the main dashboard, they land on this new welcome screen immediately after creating an account, containing a limited set of starting actions from which to choose from. These actions also highlight product benefits in our users' own words.

Existing design was updated with new copy rooted in user concerns

Communicate how Pantheon is unique

We updated only the copy on the existing "Choose CMS" set up screen to communicate the benefits the developer will receive by running a CMS on Pantheon vs. other hosting services or even rolling their own. (Updating the implementation of this screen was out of scope given the project timeline).

For example, we found WordPress users tend to have less experience with version control even though they know it's a best practice, but they can quickly realize a benefit of Pantheon by using the product's built-in version control system. 

Create guiding moments

We added a new empty state for the Dev environment to communicate why and how to use this environment. The empty state includes guidance on how to start development by first installing WordPress, with a clear call to action.

Explain Pantheon's differentiating workflow

Because the Dev/Test/Live environments and workflow are a Pantheon differentiator according to users, we extended the Dev empty state design to the Test and Live environments, in order to communciate how to use the entire workflow.

Provide assurance

Once WordPress is installed and the developer returns to the dashboard, we added this new message in the green box to confirm the installation succeeded.

Lead to the next step

Then we guided them to a next step with a new CTA to clone the site code (the yellow "Clone with Git" button). 

We also simplified the UI in area surrounding the button so that this action is more discoverable.

Dropdown design by designer Nathan Lee

When opened, dropdown itself was simplified to focus only on the information developers need, and no more.

Impact

From the research, we uncovered several roadblocks impeding most new users' primary goal to experiment directly on a testing site, so that they can learn how Pantheon will improve their workflow. But developers often abandon their attempt, and don't make it to that first site in the first place.

To track the impact of the first time user experience improvements, we tracked the number of new developers who make at least one code commit on a first site, and observed a +10% improvement compared to the old design. This positive result signals that more developers are able to reaching their first goal of trying out the workflow by experimenting directly with code a new site, and evaluating firsthand how Pantheon can improve their workflow challenges.

© Kim Kiser Ramirez 2020