Multiple Facilitators in One Study: How to Establish Consistency
Provides guidelines, with case study examples, for establishing consistency in multiple-facilitator studies. Presented at the 2010 Usability Professionals’ Association (UPA) Conference.
Can I Hear You Now? Designing the User Interface of a Product Improves Cell Phone Reception
Describes how TecEd, working together with engineers and industrial designers, created and tested the UI for a new product that improves cell phone reception in people’s homes. The project culminated with field usability testing of a functional prototype in two single-family houses with non-technical users. Presented at a 2008 World Usability Day event.
Learning about Users When You Can’t Go There: Remote Attended Usability Studies
Describes a method for conducting usability test sessions remotely and presents details on facilitating remote user sessions based on two case histories. In the Proceedings of the 2008 IEEE International Professional Communication Conference (IPCC).
A Clever Startup: Combining Qualitative and Quantitative UX Research
This article describes how TecEd helped a young startup company collect both qualitative and quantitative research data to inform their decisions about a social networking website for movies. In User Experience volume 7, issue 2 (2008), the magazine of the Usability Professionals’ Association.
Shoes for the Shoemaker’s Children
A usability company redesigns its website to market itself and better support clients, associates, prospects, employees. The process involves obstacles and benefits that give fresh insights into customer experience and usability processes for customers and practitioners. The process — and the site — demonstrates how usability works when it happens at home. Presented at the 2006 Usability Professionals’ Association Conference.
Balancing Rigor, Adaptation, and Mentoring: Field Study at Customer Sites to Initiate a Corporate Usability Program
Describes how TecEd mentored new usability staff at a client company in performing a field usability test with the client’s customers. co-authored with the client company. In the Proceedings of the 2004 Usability Professionals’ Association Conference.
Ethnographic Interviews Guide Design of Ford Vehicles Website
Describes ethnographic interviews with vehicle buyers to learn how they make purchase decisions. Despite a tight schedule, limited budget, and difficulties in finding suitable participants, we obtained valuable data that helped J. Walter Thompson refine the Ford Vehicles website. In the Proceedings of CHI 2003.
Combining Usability Research with Documentation Development for Improved User Support
Describes two case studies where TecEd leveraged usability research and documentation activities to create solutions that met the needs of both our clients and their customers. In the Proceedings of the 2002 Association for Computing Machinery’s Special Interest Group on Documentation (ACM SIGDOC).
When the Field is Far Afield: Multiple-Country Observations of Complex System Use
Describes the challenges of usability studies of complex systems that are used internationally, based on a case study of a multinational company’s enterprise-wide call management system. In the Proceedings of the 1998 Usability Professionals’ Association (UPA) Conference.
Multiple-User Testing: When One Person Can’t See Everything
Describes the pros and cons of the methods used in designing two usability tests where constraints prevented observation of all participants by one person. In the Proceedings of the 1997 Usability Professionals’ Association (UPA) Conference.
Testing the Sizzle of the Steak: Usability Testing of Packaging
Provides the rationale for usability testing the packaging of high-tech products, including lowered support costs, improved customer satisfaction, and increased sales. In the Proceedings of the 1997 Usability Professionals’ Association (UPA) Conference.
User testing is an integral part of web design as it gives impartial and quantifiable insights into the customer’s experience.
As discussed at our recent JUMP multichannel event, usability testing removes any ego or opinion from product design which helps improve customer satisfaction and ultimately increase revenue and growth.
There are a number of different methodologies for user testing, including A/B or multivariate tests, heatmapping, or videos of real customer journeys.
As these case studies show, each different methodology can identify small changes that lead to massive increases in CTR, conversions and sales.
For more information on this topic, checkout our new Conversion Rate Optimization Report 2012 in association with RedEye.
The research looks at tools, strategies and processes employed for improving conversion rates, as well as examining different areas of best practice.
Sidelines increases landing page conversion from 5% to 55%
When sports website Sidelines first started out it found that, despite a steady flow of traffic, only 5% of visitors actually signed up to its service.
So to try and increase the conversion rate co-founder Vinay Kuruvila ran six A/B tests.
1. The original landing page showed three screenshots of the product and one sentence below each describing a benefit of using Sidelines.
Due to the poor conversion rate they experimented with a simplified version that just had a random picture from the 2012 Super Bowl and a couple of sentences about the product.
As a result conversions went from 5% to 17%. Kuruvila suggests this is because if a screenshot doesn’t immediately show a customer what your product does, then it’s better to go for a simple, eye-catching photo.
2. For the next test they changed the landing page so it had a large background picture that covered the entire screen and reduced the text to a single tagline and one sentence explaining the product.
This test saw conversions creep up to 25%.
3. After trying a number of different taglines, Sidelines found that the deliberately vague option of “Follow Sports Together” was the most successful.
The theory is that it intrigues people and convinces them to sign up so they can find out more.
4. Initially the landing page only highlighted the sharing functions of the site, but when they added the words “Follow your favourite teams” conversions jumped from 25% to 40%.
This is because it made new visitors aware of the brand’s value proposition, so they knew what they stood to gain from signing up to Sidelines.
5. Like many startups Sidelines originally required new users to signup using their Facebook or Twitter accounts.
However after adding an email sign up option the number of people signing up using their social account increased by 7%. Kuruvila says this is because email signup option increased trust for the brand.
6. Kuruvila initially thought that Sidelines would appeal mostly to men, but traffic from Facebook ads proved that assumption to be incorrect.
The highest conversion rate “by far” was women aged 26-40 with the messaging “share photos, videos, articles and opinions about your favourite teams.” In addition, these women were inviting 35% more friends to join Sidelines than men were.
Finally, men who received email invites from women converted about twice as well as men who received email invites from other men.
It was clear that women aged 26-40 loved to share, but Sidelines wanted to ensure the landing page message appealed to the vast majority of users who start off as pure content consumers.
The compromise was a message that started off with the consumer value proposition and then include the sharing aspect: “Follow your favourite teams, join the hottest discussions and share photos, videos and articles with other fans.”
This helped landing page conversions increase to 55%.
DHL uses A/B testing to achieve a 98% uplift in conversion rate
DHL ran an A/B test to try and increase signups to a free Import Tool Kit. It created a new page that made the signup form the main focus by making it larger and moving it to the top right of the screen.
All the text remained the same, but the image was changed from a cityscape to a picture of a friendly courier.
The new form design
The test ran for around four weeks in July, and as a result DHL achieved a 98+% conversion rate increase in two countries at a 95% statistical confidence.
ASOS reduces abandonment rate by 50% by changing one CTA
ASOS ran A/B testing at the sign-in page of its checkout to try and reduce the number of abandoned baskets. The original page had a CTA for new customers saying ‘Create your account’ that it changed to simply ‘Continue’.
This simple change reduced basket abandonment by 50% at this stage as consumers associate creating an account with a long process of form filling.
The new checkout design
In fact ASOS left the checkout process for new customers largely unchanged and still required them to choose a password and create an account, but this simple change in copywriting had a massive impact on the customer’s perception of what they were being asked to do.
The old checkout design
Hyundai increased requests for test drive by 62% using multivariate testing
As with other case studies on this list, Hyundai set out to improve conversions on its product landing pages.
This actually included three different goals that it wanted to optimise:
- Primary goal was a brochure request and/or request for a test drive.
- Secondary goal was a clickthrough from the car page to the first step of the funnel.
- As a check, engagement (inverse of bounce rate) was also measured.
To test these different criteria Hyundai used multivariate testing, which is different to A/B testing as each change you make creates a new variation to be tested.
Original landing page
This allows you to track how each single change impacts conversion rates so you can find the best combination of factors. Using Traffic4U it tested eight different iterations by changing the following sections of the page:
- New (SEO friendly) text versus control text. The hypothesis was that if Hyundai changed its normal text to SEO friendly text and it didn’t impact conversion rate, it could permanently implement it for SEO benefits
- Extra call-to-action buttons versus no extra buttons. The thinking was that an extra call-to-action would encourage the user to complete the desired action.
- Large photo of the car versus thumbnails. Hyundai thought that larger photos would entice the visitor and also confirm to them that they are on the right page.
The results showed that the combination of SEO text, extra CTAs and larger images increased the conversion rate (request for test drive or brochure) by 62% and there was a 208% increase in CTR (step 1 to step 2).
New landing page
User testing videos help increase sales by 9.5%
Appliances Online commissioned 125 user testing videos to help identify potential improvements to its product pages, giving it 250 hours of footage of customers browsing the site before making a purchase.
The company used the videos in conjunction with other tools, such as Click Tale, which provided heatmaps showing which elements of product pages users were interacting with the most.
Thanks to this insight it uncovered several issues with its product pages. For example, 70% said that pages were too busy, 17% said service information needed to be clearer, while 13% thought the video experience could be improved.
As a result, Appliances Online made these changes to its product pages:
- Banners touting special offers were incorporated into the copy on the page, allowing the buy button to be moved above the fold.
- The buy button was changed from dark blue to a more eye-catching green colour, while the text was altered from ‘Buy’ to a more descriptive ‘Add to basket’.
- Previously product videos opened in a pop-up screen that took too long to load. The solution was to embed the video into the product pages, which is less interruptive, while consumers can also scan up and down the page looking at reviews and product specs, and the video stays still.
- Creative copywriters were employed to draft unique product descriptions.
These changes resulted in a 9.5% increase in sales, while 37% more visitors viewed the product videos. As viewers of these videos are 57% more likely to add items to the basket, this was a big improvement.
In addition, the number of reviews left by customers increased by 11%, while there was a 33% reduction in calls about delivery, as the information was more clearly visible on the product page.
Veeam changed a single word an increased CTR by 161%
In an effort to increase conversions Veeam Software asked all visitors to its product pages what other information they would like to see.
A number of visitors answered “pricing” however Veeam sells through partners so does not publish pricing information on its site. However it does have a ‘Request A Quote’ link that leads to a sales inquiry form.
Simply by changing the CTA from ‘Request A Quote’ to ‘Request Pricing’ Veeam achieved a 161.66% increase in CTR from 0.54% to 1.40% with 100% statistical confidence.
This is a great example of increasing conversions by using customer feedback to improve the website.
Amazon made $2.7bn by asking one simple question
Consumer reviews are a proven sales driver in e-commerce, with research showing that 63% of customers are more likely to make a purchase from a site that has user reviews.
Amazon receives hundreds of reviews so it was faced with the challenge of how to curate them into a useful order for its customers.
It struck upon the best system by adding the question: “Was this review helpful to you?” This allows users to endorse the best reviews, with the top three being featured on the product page.
Other touches, such as allowing users to easily view the best negative or positive reviews, as well as some handy charts that summarise review ratings, make the large number of reviews manageable for users and more useful for Amazon.
Displaying the most helpful reviews has increased sales in the media products category by 20%, meaning this feature was worth $2.7bn to Amazon.
VOIP telephone service increases quote requests by 262%
In this test an anonymous B2B VOIP telephone services company set about increasing its number of quote requests by performing an A/B test of different page designs.
The original signup page
The company redesigned the page to simplify the form and reduce friction in the sign up process. The number of fields that users had to fill in was reduced from six to three, and it removed any requests for personal information.
The new signup page
Other changes included:
- Changing the CTA from ‘Get Quote’ to ‘Show Me My Instant Quote’.
- Giving an interactive quote online rather than getting a salesperson to call the customer.
- Adding third-party security logos.
- Adding a chart comparing its service versus traditional phones.
The result was a 262% increase in conversion rate from 2.4% to 8.8%.