Dr. Jakob Nielsen

In: Interviews

By Meryl K. Evans

By Nick Finck

Published on November 13, 2002

Then and Now
Companies, Clients, CEOs, and Cost
Usability Attitudes

Then and Now

Digital Web: What surprised you while doing recent studies on Web site usability?

Dr. Jakob Nielsen: Probably the most surprising recent finding came in our study of the usability of Web-based applications in Flash. Even though the Flash designs themselves had plenty of usability issues and could be improved in many ways, the single biggest factor in the study turned out to be the integration between the application and the hosting Web site.

More of a third of the test users never even made it into the applications because of the bad way many of them were linked with the main workflow of the Web site. This means that a company that has invested in developing a Flash application can increase its return on investment by about 50% without any changes to the Flash: all they need to do is make better links.

Digital Web: What are the common problems of today? What past common problems have become scarce today?

JN: Trust is a huge problem. Users are justifiably very cynical about their privacy and about the extent to which they can trust Web sites. In our recent study of newsletter usability, we saw a lot of people being very hesitant to sign up for a totally honest and legitimate newsletter because they were afraid of spam and shady marketing. It’s going to be a big challenge to find ways to reaffirm the credibility of honorable websites that respect users’ rights.

A continued problem that has always been bad and continues to be bad is the writing on Web sites. Very rare to see writing that’s to the point and focuses on answering customers’ questions without smothering the info in marketese.

We have almost succeeded in eradicating annoying splash pages and Flash intros. There are also not nearly as many bloated designs as there used to be. These are great advances from a design perspective. From a technical perspective, there has been great progress in searching, even though these advances have mainly manifested themselves on public search engines and not yet on site-specific search engines or intranet-internal search engines.

Digital Web: What areas are you focusing on in research from a usability perspective today? Why?

JN: Much of my current research focuses on intranet usability because it has been such a neglected field in the past. In our latest study, we tested 16 common employee tasks across 14 intranets. I estimate that if we could get all companies in the world to follow good usability practices for these 16 tasks, then the annual productivity gains would be $311,294,070,513. Since there are many more tasks on most intranets and since it’s surely possible to do better than what we found in testing 14 designs, the total amount of money at stake in intranet usability is in the trillions. Even small improvements have great leverage in this area.

The next round of research will focus on mobile usability. We have done a small amount of work in this area for several years, but mobile computing and wireless Internet access are not sufficiently mature yet for us to develop the comprehensive list of guidelines I would like to publish. One of the downsides of our methodology is that we have to wait until there is a broad spectrum of designs in the world before we can learn enough from user testing to generalize and derive guidelines. Even so, just being aware that this field is developing will allow us to do the research at a time when less than 10% of the design projects have been completed, so the remaining 90% of projects will still benefit from the guidelines.

Digital Web: How do you create a site that is highly usable?

JN: The biggest factor is a management decision to prioritize usability and consistency. Especially on intranets, we find tremendous usability problems caused by a lack of design standards that allows every department to put up pages in weird and conflicting designs with poor integration.

A corporate policy to promote usability, enforce standards, and fund user-centered design is the only way to quality user interfaces. Individual designers can do their part, and major aspects of quality do come from the myriad of details on every single page, but to fully support users requires a global approach across the company.

Digital Web: What methods do you find most effective? Why?

JN: Simple user testing: get four to five real customers to sit down by a computer (one at a time) and use your design while they think out loud. It takes almost no time to run such a study to get a list of the top ten (or more) changes that need to be made to the design. We can teach a team to conduct these simple tests in three days. An incredibly effective method that usually has 1000% return on investment because it’s so cheap and yet generates important quality improvements.

Digital Web: What methods do you find least effective? Why?

JN: Measurement studies: they cost five times as much as qualitative user testing and yet generate less knowledge about user behavior and fewer insights that can set directions and drive the design.

Digital Web: How do non-Web developers perceive usability?

JN: Software developers used to be enemies of usability because of their assumption that they knew what was best for users. The basic thinking was “I am developing for humans; I am a human; I can make something that’s good for humans.” Unfortunately, average users are quite different from programmers, so this approach has failed except for a small number of tools for geeks made by geeks (Apache and Linux are good examples).

Today, most professional software engineers accept that quality methods are needed to develop good software and that you have to employ life cycle-oriented processes for a project to succeed. Some of these processes relate to code quality and others relate to the quality of the user experience, and mature software organizations have usability methodologies in place to handle the latter.

This has been a huge change over the last twenty years: from usability being an outsider to usability becoming an integrated part of development. Unfortunately, Web development is much less mature than software development and there has been a blatant disregard for good process and quality methods in the past, but let’s hope that history repeats itself for this new type of development.

Digital Web: What is your biggest pet peeve regarding usability on Web sites? Why?

JN: No prices.

No B2C ecommerce site would make this mistake, but it’s rife in B2B, where most “enterprise solutions” are presented so that you can’t tell whether they are suited for 100 people or 100,000 people. Price is the most specific piece of info customers use to understand the nature of an offering, and not providing it makes people feel lost and reduces their understanding of a product line. We have miles of video tape of users asking “Where’s the price?” while tearing their hair out.

Even B2C sites often make the associated mistake of forgetting prices in product lists, such as category pages or search results. Knowing the price is key in both situations; it lets users differentiate among products and click through to the most relevant ones that they are likely to buy.

Companies, Clients, CEOs, and Cost

Digital Web: When working with clients how does one translate usability engineering into ROI, or does it translate?

JN: Usability completely translates into dollars, but the metrics vary, depending on the type of project. For intranets, ROI comes from increased employee productivity: making it faster for people to do their jobs. For e-commerce, ROI comes from increased sales: making it easier for customers to find the product, understand the product, and complete the checkout transaction. For other types of sites, ROI is harder to estimate. For example, for a marketing-oriented site, greater usability means that prospects will view the company as being easier to do business with, but who knows how many sales it delivers down the road.

Roughly speaking, usability will cost about 10% of the total budget for a development project if one takes a life cycle approach and employs the full range of methods throughout the stages of the project. (Most companies start cheaper by running a small user test over a few days, but eventually you have to take a life cycle approach and conduct multiple rounds of user research with a variety of methods.)

Also roughly speaking, good usability will approximately double the effect of the project because users can accomplish about twice as much with an easier interface. Usability increases the benefits of the design by 100% at a cost of 10% of the investment in developing it. Thus, the benefits are ten times the cost, leading to an average ROI estimate of 1000%.

Digital Web: In lean times, people sacrifice resources including hiring information architects and usability experts, and instead expect one person to do it all. How do you see this impacting the usability of Web sites?

JN: Could be a benefit, since the biggest problem facing usability is communicating findings and recommendations to designers and developers. Quite often, people don’t want to hear the findings from user research if it invalidates their pet design. Even if they listen, they may not understand because the usability expert did a poor job of explaining the findings. In contrast, if the designer or developer ran the usability study, they are going to believe their own findings and they are going to have first-hand knowledge of what happened instead of the filtered view they would get from reading a report.

Smaller teams ought to mean smaller projects, which may also be good for usability. Simplicity is rule #1 for usability, and fewer features means that those features that do remain in the design will automatically be easier to understand because there are fewer other features to compete for the user’s attention. The only problem is companies that don’t understand the need to focus their projects on the key features. If the company insists on doing the same number of things with fewer people, then of course, quality will suffer.

Digital Web: How can a Web site do usability testing on a shoestring budget?

JN: Anybody can do usability testing. We run a workshop where we teach a team how to run studies in three days: we do this by actually conducting a complete, if small, usability project in those three days, complete with planning the test, running the test, and analyzing the test results and turning them into redesign recommendations. The key point is to do simplified testing.

Of course, there are reasons to employ advanced usability methods as well, and it would be better to have a bigger budget that would allow, for example, field studies and international testing. Still, if you only have a tiny budget, you can still do usability. The cost of the simplest test is about $1,000 to recruit and compensate the test users plus about three days of your time. If you don’t even have $1K, you can still get some usability data from friends and family, as long as you stay away from fellow designers and developers, but I don’t recommend this approach. It’s worth the money to get truly representative users for a test.

Digital Web: What sites walk the talk on usability? Fail?

JN: Amazon.com remains my favorite site. They run everything though quick usability tests with a handful of users and also do more elaborate A/B testing on the live site. As a result, it’s the easiest place to shop online, year after year.

I also like the redesigned homepage CNN introduced about a year ago, with more of a focus on the top stories and a scannable list of smaller stories. For mobile access on a small PDA screen, MSNBC has better usability, though.

TimBuk2 is an interesting example of both good and bad usability: there’s a great Flash application where you can design your own bag and experiment with different color schemes and accessories. And then they lose about half of users because they don’t have a decent error message when people forget to pick some of the colors. They could double their sales if they fix the one small usability failure.

Sites that completely fail? Most big corporate websites are still so arrogant that they refuse to communicate to customers and answer their questions in plain language. They still have information architectures that mirror the organization chart, and they still have the style of bloated graphics that are long gone from any site that actually does business on the Web.

Usability Attitudes

Digital Web: In October of 2000 you published an Alertbox entitled “Flash: 99% Bad” and it infuriated the Flash community and some of the Web design community. Today your name holds a stigma with several designers on the Web. With the advent of Flash MX can you tell us what has changed since October of 2000?

JN: There have been some technical changes in the Flash software itself. Most notably support for accessibility and the Back button as well as standard GUI components so that users don’t have to puzzle out the meaning of homemade scrollbars and the like. Even more important, there has been a change in strategy from focusing on annoying users with intrusive bells and whistles to a much more constructive focus on developing rich Web-based applications. Flash is now a GUI construction kit with support for client-server functionality across the Internet, which is something we have needed for years.

Taken together, the changes in Flash over the last two years are a major triumph for usability and proof that big companies can be made to listen. Usability will not always win, but it does describe the path of least resistance because it’s based on the simple idea that easy is better than difficult. The recent victory in the Flash case certainly motivates me to continue my fight for human mastery over technology.

Digital Web: Macromedia claims Flash MX to be more accessible and usable than its predecessors yet many Web Designers have failed to take advantage of these features in Flash MX. What will it take to facilitate a change in the way Web Designers think about creating accessible and usable Web sites?

JN: It’s good that Macromedia has included more accessibility features in the underlying technology of tools like Flash. A necessary, but not sufficient, step is to make the actual sites easier for users with disabilities. We conducted a study of current Flash designs with users with disabilities and derived a list of 21 design guidelines that individual Web sites should follow if they want to facilitate access for users with disabilities.

So far, the main push for accessibility has come from government regulations and threats of lawsuits, but I think it’s a more constructive approach to start thinking of users with disabilities as a big group of potential customers and work to make it easier for them to conduct business with you. This is just an extension of the general usability ideology, which aims to make customers feel that *their* needs are the drivers of the design. Anybody who wants to sell to senior citizens needs to prioritize accessibility, but even in younger age groups, there are about 10% of users with some kind of impairment.

Digital Web: A lot of what you have said and published has contributed to the evolution of the Web today as we know it, however much of the industry feels that while your approaches do have core values behind them they often come across more authoritarian-like. Most Web Designers are not that docile. Wouldn’t a more compassionate and constructive approach go further in educating others about usability?

JN: I can hardly be authoritarian since I don’t hold any authority. I don’t run any design projects and I can’t fire anybody for making a Web site too hard to use. All I can do is to report the findings from usability research and recommend that companies follow the guidelines that have resulted from these studies. I admit that I tend to be outspoken in defending users’ rights to simplicity, but I simply think that it’s time to stop being so accepting of difficult technology.

In any real development project, one obviously has to compromise on quality and deliver something that will fit within the budget even if it doesn’t follow all the usability guidelines. The point is that any design project is a negotiation between different needs and constituencies.

My job is to negotiate on behalf of the world’s 500 million Internet users since they are the only stakeholders without representation on the design team. And it’s one of the oldest tricks of negotiating that you never start out asking for a compromise deal. You start out asking for the ideal and then you negotiate down from there. Thus, I have to promote ideal usability and the full set of usability guidelines, even though I know that any project will end up violating some of them. It’s the project manager’s job to make compromises, not my job.

Digital Web: What would you change about usability?

JN: The main elements of usability need no change since they are already working exceptionally well. I would like to find cheaper ways of conducting field studies so that more companies would do them. Same goes for international user testing. It would also be great to discover the solutions to some of the unsolved problems of usability such as the fact that most users tend to plateau at a fairly low level of performance with user interfaces instead of gradually getting better and better. People don’t have the motivation to learn once they know enough to accomplish their main tasks and we don’t have good ways of overcoming this barrier and making users into true experts.

Related Topics: Usability, User Experience, Web Guru