Published on March 5, 2003
Digital Web: In the lean-and-mean times of the post-dot-com period, return on investment (ROI) has become much more important. What are some of the immediate effects of this on Web development?
Kelly Goto: In the “golden” days, millions of dollars were thrown at Web redesigns. Forrester Research reported that Fortune 1,000 companies averaged between $1.5 million and $2.1 million per year on site redesigns (circa 2000) but that “their shot-in-the-dark efforts often worsened the user experience.” It is no surprise those days are over.
Today, companies large and small have seen the benefits of applying a user-centered methodology to their Web development approach. It seems like an obvious solution to a sluggish economy to turn to a straightforward approach—make it easy for people to use and interact with your site, and your company. Despite the preachings of Jacob Nielson and Jared Spool, companies still feel ill-equipped to manage their own usability testing processes, thinking there is some “magic” out there in execution and analysis they do not possess. To quote Steve Krug (author of the fabulous book, Don’t Make Me Think) “it is not rocket surgery.” Get out there. Test. Gather results. Make recommendations. Implement changes. Measure results.
The focus of ROI has also forced businesses to attach redesign projects to more discrete business goals rather than more generalized business goals such as “brand recognition” or “audience penetration.” Because of this, businesses in their second and third generation Web sites are more inclined to focus on smaller initiatives tied to particular revenue streams rather than go through complete Web redesigns. Web development has become a fast-turn, reactionary process. Development/design companies and in-house teams are frantically turning around initiatives, trying to generate profits, cut losses and manage business goals along with customer expectations within an 800 × 600 environment. With the change in times and the need to show profit at every turn, redesign timelines have changed from 12–18 months to 3–6 month cycles targeting specific results.
Digital Web: In the business world, ROI is defined as percentage based on what the investor received divided by the initial investment. Does this definition hold for the Web design world?
KG: The simple answer to this question is yes. Projects are judged according to their efficacy. The real challenge that a Web design studio has is whether the design that they are creating has a realistic business goal. However, it is not the role of a studio to question the business logic of the client. If the business logic is faulty (in terms of revenue produced by a solid design / redesign implementation) then no matter how successful a design company is in meeting the interaction goals of the client, the project itself will fail since it will not provide the desired ROI against which success or failure will be determined. For in-house teams, the same holds true.
A good design studio (or in-house team) can, however, help the client understand how certain design elements are directly tied to the business goals, and how business goals are tied to assumptions of ROI. The Web world is the business world. ROI (return on interaction) as we see it differs from traditional ROI (return on investment) by focusing on customer experience and metrics of satisfaction and success rather than traditional spreadsheet formulas. The traditional methods of analysis and measurement used by mainstream businesses are being actively applied to the Web—however even with solid metrics companies have a tendency to design haphazardly and without measurable goals in mind.
We have been using the term “return on interaction” to target measurable responses and results based on user experience. We think of this as “real-world ROI” for the small to medium sized company without the research, marketing and user experience support as the larger fortune 1,000 group. A combination of efforts, from interviews, online surveys and user experience studies, usability testing and expert reviews, inform the process. Jacob Nielson claims a 10% project budget focused on usability will yield a 135% return on usability response in site redesigns.
Digital Web: What are some ways to measure a ROI for a Web design project?
KG: As a smaller company, we partner with companies like Vividence who offer services to measure customer experience. For one of their clients, Red Envelope, they were able to measure an increase in customer satisfaction, with measurable data such as a 45% decrease in customer calls related to confusion in the use of the site, decreasing overall operating costs of the call center by 2%. By focusing on the checkout process, the search tool and global navigation, users indicated they were 12% more likely to return to the site to make purchases.
We have also found great partnerships in companies focusing on measurement via search engines and tracking results. Barbara Coll of Webmama.com focuses on the niche of search engines. Her company offers dedicated services to companies to help set up and track usage based on search results and placement. San Francisco-based Fathom Online, a Search Engine Marketing Company, uses end-to-end tracking solutions to measure ROI. They begin by tracking each keyword on each search engine all the way through conversion at the advertiser’s site—the ultimate acid test. Chris Churchill, the CEO of Fathom Online, explains:
Keyword buying has been around since the first search engines began to put up banners, but in the last year or two, the convergence of database technology, XML, and paid search has yielded a breakthrough. Now, marketers can see, in simple reporting terms, which keywords are converting into sales. Commonly called “click to conversion tracking,” this technique is an extension of what marketers learned in the 80s about direct mail and measuring their ROI by response rate.
Sometimes it is not necessary to attach specific percentages to redesign goals. For some companies, simply getting the decision-makers to agree on a focused action from the home page targeted to a specific audience is an accomplishment. For one client, our goal was to reduce the ordering process from 12 screens to 5. For another client, putting the company’s 1-800 number on every page of the site, customer sales increased immediately. Conversely, by removing the 1-800 number and improving the site’s FADigital Web area, calls to customer centers can decrease. There should always be some means of tracking in place, and incorporating Web traffic analysis software such as Web Trends and Hits Link will allow for stats and measurements to be gathered over time.
Digital Web: What steps can you take to ensure the business objectives of the project are met?
KG: To effectively meet the business goals of a project, the studio should insist that the client (or decision-making team) articulate its goals clearly. These goals should be documented, and in some cases, prioritized. Only once they have been integrated, into an agreed Statement of Work and/or requirements document, should work on the design proceed. After all, if you do not know where you want to go, you cannot begin to map out a strategy. With a redesign, these goals are easier to identify. When there is a starting point, information that can be gathered from current data, along with site logs, tracking software and customer feedback.
Once these goals have been agreed upon, the studio can begin to match up business goals with discrete features and functionality to be designed—both in the look/feel and layout, and in the user flow. Again, this should be agreed to and signed off on by the client. Oftentimes a client has numerous and sometimes conflicting business goals, and may want the studio to help them distill these into a single vision.
While this may be a useful exercise for the studio, in the end, only the client can determine what their business goals are, and how they relate to ROI. Whether internal or external, the development team should restrict themselves to proposing improved means of interaction that facilitate the business goals.
Digital Web: Should all of these be applied to a project already in progress?
KG: Yes. It is never too late to step into a project and bring clarity to the goals, even if there are political issues to navigate. There are a variety of ways to broach this subject, (e.g., "Let’s take this time to reaffirm our goals for this project.") without exposing the studio to issues of liability. No matter how delicate the situation might be, it is worthwhile to take this stand, since in the end, if a project does not meet the client’s expected business goals, the studio will be held responsible.
Digital Web: Can these be applied to smaller Web design firms or projects?
KG: Absolutely. Clarity of audience goals can be outlined in a simple document for projects large and small. It is a matter of proper client engagement rather than overhead. Larger companies, such as Expedia and Amazon have dedicated research, marketing and usability teams. These companies have the ability to track audience behavior and site metrics at an extremely high level. For the most part, smaller companies do not have the same resources , budget or expertise for testing and analysis. Instead, smaller companies need to focus on the end users and their needs, and adjust business, sales and marketing goals to align with audience objectives.
Digital Web: How can you ensure both the client and the Web design team understand those objectives?
KG: We create a communication brief which outlines the overall goals of the project, audience profiles, targeted redesign goals and desired audience perception. This focused, one-to-two page document is based on a client survey and interviews to ensure all team members are on the same page. We ask that all decision-makers on the team sign off on the communication brief, and, if it needs to be updated, we post each version on our staging area with sign-off. For one indecisive client, we posted 12 versions of the communication brief.
Digital Web: Can you share any anecdotes about miscommunication in this area?
KG: One initiative, by a high-level CEO in a fortune 1,000 company, was to add graphics of his favorite hobby, rock climbing. This direction was purely at the “whim” of the CEO and incorporated much to the chagrin of the design team. Sometimes we joke about creating the CEO site, one that has everything they have asked for—that appears at a special URL.
Another time we sat around a round table of a Web company offering dining and food services. About 20 people sat around discussing the pros and cons of adding salad dressing to the Web site, and we must have talked about this for 2 hours. I added up the people and time and determined we spent nearly $6,000 of billable time just thinking about this. Imagine how much time would have been put into implementation, marketing and distribution of this product. The time vs. cost (and profit) need to be weighed early on to determine if it is even worth the discussion.
For a pet company we worked with in the dot-boom years, we set up monthly usability tests, with targeted users identified and set up for testing and interviews. Unnecessary round table discussions and arguments (like leash selection, size, color and length of leashes...) were kept at bay by tabling discussion and adding it to the interview questions and usability tests each month. Once the results were in, a comprehensive discussion could be held based on credible information rather than individual guesses and preferences. While this is not perfect science, it does help to avoid costly, circular discussions.
Digital Web: Is there anything you’d like to add?
KG: Why complicate things so much? The beauty of the Web is that it can be modified and infinitely updated. This is both positive and negative. Unlike the print world, you are forever in blueline stage, able to update and correct the site on the fly. This means the fast stages of development have been encapsulated into very short-term development cycles.
Digital Web: Finally, other than ROI, what’s your favorite initialism?
KG: I think nothing is more appropriate these days than making sure to CYA. ;-)
Kelly Goto is principal of gotomedia, inc. a San Francisco based company focused on web redesign and user centered methodology. Kelly also continues to lecture and teach internationally on the topics of usability, information design and workflow. Kelly, along with Emily Cotler, recently co-authored Web ReDesign: Workflow that Works.
Craig Saila has been working the Web since 1996, and is now a Web producer for Bell Globemedia’s financial sites, including Globeinvestor. He’s worked in the past with the Ontario Science Centre’s Digital Media Publications group, and as an associate producer at one of Canada’s biggest news sites, CANOE. Throughout his work, he’s divided his time between client-side development and online journalism—dual interests which are apparent at his site, saila.com.