saúl vega

Car Del Mar:

International A/B Testing

The Challenge: Selling Rental Insurance

A major challenge in designing an e-commerce site that will scale internationally is that a user’s needs differ across different cultures and different markets. What your team might know as “best practices” may only be best practices for your context. In such cases, the US and European car rentals teams had to do deeper digging before making a design choice.

One such example is the treatment of rental insurance. Car rental insurance is one of the largest drivers of revenue for CarRentals.com and the German company Car Del Mar. However, when it came to deciding the proper way to present insurance on the site, the teams had significant disagreements.

The US team’s proposed insurance flow is straightforward. Either you wish to purchase insurance or you don’t. There is only one insurance rate, and it is presented on the order page, once a car has been selected. This design is meant to reduce friction on the results page, and thus increase clickthrough to the order page.

The German team’s design features multiple insurance rates, all presented earlier than in the US flow: on the results page. This offer is then re-iterated on the order page, as in the US flow.

Conflict

The insurance sales treatment proposed by the team in Germany was in opposition to much of the knowledge that the US product team had learned about car rentals user behavior:

• It has more prices displayed, and features higher prices. Adding too many price points to an experience had overwhelmed US users during prior tests.

• The juxtaposition of different insurance products creates a sense of less-than for the standard rate, potentially discouraging users who would opt for that rate.

• There is more information for the user to process in order to make a car choice. This increases the risk that users don’t even make a car choice before bouncing out from cognitive overload

• Fewer results can be shown above the fold. It takes more effort for users to see the same number of options, in the German team’s preferred design.

To help decide this stalemate, the teams ran user research labs dedicated to understanding the user journey with insurances. We ran these out of Hamburg and SF. I designed a script for the labs to help us reach our specific research questions, and presented the users with prototypes.

The most salient points found:

• US average car rental length was 3-5 days

  • Mostly vacation and work rentals, driving in-country

• German car rentals averaged 1-2 weeks

  • Mainly outside of Germany

• People who rent cars in Germany give more importance to insurance because they are likely to be taking longer trips in foreign countries.
  Thus, insurance is of high priority, and can sit higher in the funnel.

• In the US, Insurance is an afterthought. Many people already have it, or their credit card offers coverage as a benefit.

A/B Testing

The qualitative research helped us understand the users’ mental models and rationales, in addition to pointing us to the optimal layout for each version. Next, we wanted to test these layouts against each other in terms of their conversion rates.

We performed an A/B test of each version on the US site and the German site.

Learnings & Conclusion

As corroborated by the qualitative research, the US team’s design choice had higher conversion rate in the US and lower in Germany. The German team’s design preference won in Germany and lost in the US.

• Thus, we modularized the insurance sales sections and altered the design template to accommodate for this. The data corroborated the need for the additional investment in development.

• Through this experience, we confirmed that although you should avoid re-inventing the wheel, it is always key to bring it back to the user, especially when designing for newer contexts.