usability

About

Syndication

Categories

Archives

On Being a User Researcher

society & sociology, usability

May 21, 2004, 07:18 PM

I've been thinking recently about user research (by which I mean formative research techniques that attempt to understand how potential users live their lives, what problems they have, what their goals are, etc.) and what it takes to be a good user researcher.

There are many skills that a person who hopes to do user research must acquire. For instance, a good user researcher must learn how to put interviewees at ease while still maintaining sufficient objectivity by avoiding misleading questions, etc. A good user researcher must be able to moderate a focus group by keeping it on topic without stifling discussion, avoiding group-think, and ensuring everyone's opinions get heard. A good user researcher needs to be a good listener. He must keep good notes. He must keep his biases out of the results. And so on.

But there is one quality that I believe all user researchers need to have to be effective, and it's never taught nor even mentioned in any training programs I know of. A good user researcher has to like people.

And I mean he has to like people, all people, not just his friends, peers, and colleagues. He has to be intrinsically interested in who they are, what work they do, how they feel about life, what they want to achieve, and so on. Even though user research is generally conducted to inform product design or marketing, in order to do his job well, a good user researcher must possess an interest in learning about people that transcends these goals. Gaining a deep understanding of how others live their lives must be an end in and of itself.

This is different from the goals of a psychologist or social scientist. Scientists who study humans hope to obtain generalizable knowledge about how humans behave. They uncover facts about human behavior; often these are abstract and somewhat removed in their statistical generality from the real thoughts and feelings of individuals. User researchers, on the other hand, do not uncover generalizable facts; instead they hunt for an understanding of the qualia, the what it's like to be a member of their target population. Though useful in and of themselves, psychology and social science aren't user research professions, nor can they be expected to adequately train user researchers.

I rather suspect that a mature field of user research would more closely resemble journalism than psychology. A good journalist goes out into the world and hunts for stories that will be interesting to their intended audience. A user researcher's audience is the design team. Their job is to uncover stories about their users that would be interesting to the designers while also accurately portraying the people they are designing for. To do this well requires many skills, but foremost among them is an intrinsic interest in and love for people that few other professions in this world truly call for.

[ Permanent Location ]

Design, Usability, and Innovation

design, usability

April 18, 2004, 08:54 AM

A few weeks ago, I went to a talk given by Hugh Dubberly on models for design (Dan has a very nice summary, as usual). During the talk, Hugh made a point about design and innovation that's stuck with me; he remarked You don't get innovation from the design process. You get quality from the design process. At the time, that struck me as quite true.

Since then, however, I've had second thoughts. Let's consider an "innovation" (an overloaded word) to refer to "some technology-related change that has a significant impact on the ways people behave, generally for the better". I think that captures it reasonably well for my current purposes. Usually when we think of innovations, we think of some brand new ideas that spark new technological developments, like televisions or automobiles. And its true that the design process won't get you these; great new ideas are elusive things and so far no one has found a process to repeatably elicit them (though there are some brainstorming techniques that can help).

But there are times when innovations are not spawned by brand new ideas. There are times when the innovation involves taking an existing technology and making it accessible to a new (probably wider) audience. Or in taking a technology that used to be complex and cumbersome to use and making it quick and simple. Suddenly, the technology is accessible and it becomes an innovation.

The GUI desktop metaphor is a classic example. Affordable computers were available before PARC's innovation got commercialized, but only hobbyists bought them, because they were just too tough for the average person to use. With a well-designed interface, however, personal computers became a reality and an innovation was born.

iChat AV is a more modern (if also more modest) example. Voice-over-IM has existed for a long time in AIM, but nobody used it. All Apple did was make it dead simple to use. Suddenly it becomes useful to the masses, and we have innovation.

Style sheets and the single-sourcing paradigm might be a design innovation waiting to happen. Style sheets are useful to some but inaccessible to most due to the complexity of dealing with the badly designed interfaces for creating and applying them. If someone were to create a more usable interface, we may have innovation.

This is, I believe, the role that design plays in innovation. Not in consistently generating world-changing ideas (these come from a multitude of disciplines) but in turning these ideas into something simple enough to affect the lives of thousands.

[ Permanent Location ]

Marketing and User Research

design, usability

April 16, 2004, 11:01 PM

I've been thinking recently about the relationship between market research and user research.

Market research is conducted by marketing departments, who are mainly concerned with figuring out what sorts of products people are willing to buy and what sorts of people are willing to buy them. I am very ignorant of marketing research techniques, but my current understanding is that they tend to involve conducting surveys and possibly focus groups with the intent of nailing down a market demographic that the company can sell some new product to. Marketing may also collect information on what features and attributes of this product are important to swaying this demographics' buying decisions.

User research is conducted by user-centered designers, who are mainly concerned with producing the interface and interaction design of the new product. As Beyer and Holtzblatt point out in Contextual Design, the kind of data necessary to drive product design is different from the kind of data collected by marketing; market research alone cannot drive research-informed product designs. UCDs require more detailed information about what the people behind the statistics are like, what their goals are, how they are currently performing their tasks, etc. B&H have already made this argument eloquently so I won't bother rehashing it here.

All this isn't to say that marketing's data is useless to UCDs, however. In fact, a reasonably thorough market analysis is absolutely necessary to have ready before design research begins. If marketing hasn't narrowed down the target user population, design won't know where to start looking to collect their data. They may wind up taking a scattershot approach and (at best) waste time talking to the wrong types of people or (at worst) design a product from data on the wrong users.

Just as marketing's data can't substitute for design data, design shouldn't be expected to collect market data. Because design research involves more detail-oriented studies, these studies cannot uncover a broad picture of the market needs on a large scale. A mistake I've seen among novice design researchers is to spend time sending out surveys to uncover a market for the product when this information was basically provided to them by the marketing team. This is a waste of time, not to mention an inappropriate application of skill sets. Design should be focused on how the real people behind the demographic numbers marketing generated are working, communicating, and feeling, for this is the information they need to create great products.

[ Permanent Location ]

The Usability of Security

usability

March 30, 2004, 04:07 PM

Neema is working on a project to improve the usability of wireless network security. As I've touched on before, I think the usability of security is an underexplored but very important topic, so I'm glad to see that Neema is thinking about it.

Since he asked for feedback, I'm giving my view (for what it's worth) of how security should work as a user-centered design paradigm. Then I'll try to relate it back to wireless networking.

There is basically only one UCD problem in the security space. Users, organizations, and systems (security people call them "principles" I believe) need to identify what other users, organizations, and systems they trust, and which ones they do not. They may also need to specify further what information they trust each other party with, and what information should be kept hidden. The big interface design question, then, is how to make it quick and easy (remember that specifying security settings is rarely the user's main goal for any system) for people to make these distinctions. Once the appropriate distinctions are made, the software should determine what technologies (WEP, Kerberos, SSL, or whatever) are appropriate to guarantee the required level of trust, and employ them. This part shouldn't concern the user.

I know this is abstract, and determining how to realize this in any individual system is a decision that the designers of each system must make on a case-by-case basis. But it's important to think of the problem in these terms, and not in terms of "how can I teach users the advantages of using SSL?" Users don't know about SSL and don't want to know about SSL. They just know who they trust, and that must be enough.

So what might this mean for wireless security? Well, when a user logs on to a wireless network, their user agent (probably their computer) needs to know whether they trust the person who is administering that network. If the user is at Starbucks, then they must indicate whether they trust Starbucks Corp. to faithfully transmit their data. Maybe the interface must allow the user to directly specify this information. Maybe it only needs to show the user what other people they are currently "trusting" with their information. But these are the sorts of design questions I'd like to see addressed. Everything else should Just Work™.

[ Permanent Location ]

Sorting Environments

design, usability

March 14, 2004, 09:28 PM

My friend Jeff Howard was showing me a research project he'd been working on for his miLife project a couple weeks ago, and now he's posted about it. He's basically conducting a card sorting exercise, except the cards contain pictures of various kinds of environments like conferences halls, bus interiors, scenic natural vistas, etc. His team is asking users to arrange the cards along a continuum of which environments they feel more or less mobile in. I believe they're also asking users to arrange the environments along another continuum of how remotely connected to others they feel and how connected they wish to feel. Note that Jeff's online prototype doesn't work as well due to the small size of the cards; the actual physical cards are larger.

It's an interesting spin on the typical card sorting procedure; the pictures may evoke a more emotional response from users which is more appropriate for this particular study than the short words used in navigation hierarchy studies. It also helps give the users more context than a typical interview question such as "name five places you feel mobile in" would, while avoiding the cost of having to find users actually sitting around in all these environments.

[ Permanent Location ]

Newsable Redesign Mockups and a Request for Feedback

design, usability

March 08, 2004, 10:07 PM

I'm working on giving Newsable a facelift soon, both to improve the interaction and visual design. I've posted some mockups of the new key screens; if you're the sort of person who likes to speak words about other people's work, then here's your chance to speak some words about mine. My goals with this release are to make the visual design a little more pleasing and to improve the interaction for people who are subscribed to a large number of feeds. Also, I hope to convince Kenneth that the three-pane format isn't the be all and end all of aggregator design.

I'm also working on the design of a community system that will help facilitate a usable open source software process. The design is based on the research I did last semester in CSCW. If this sounds intriguing to you, stay tuned for more.

[ Permanent Location ]

Eric Raymond: Open-Source Usability Advocate?!

software development, usability

February 27, 2004, 10:01 PM

I was idly surfing the blogspace (doesn't that sound so much nicer than "blogosphere"?) this afternoon when I happened across Julian Missig's blog (a fellow CMU student and a co-worker of Dana's, of all things). He points to an article hot off the press by Eric Raymond where the famous OSS guru actually argues in favor of fixing the open source software usability problem! I could barely believe it until I read the article; frequent readers will recall that I've been studying the OSS usability problem for several months now. But though I've found some encouraging evidence of OSS developers who are starting to get it, I never, ever thought I'd read an article like this from ESR. Some of his suggestions were right on, others I'm not so sure about, but the core ideas about the ways open source software needs to change are right on the money.

I've sliced out some particularly interesting bits below, with commentary.

Aunt Tillie, at this point, is either resigning herself to another session of being tortured by the poor UI choices of well-meaning idiots or deciding to chuck this whole Linux thing and go back to the old Windows box. It blue-screened a lot, but at least it allowed her the luxury of ignorance — she didn't have to know, or care, about what a JetDirect or a CUPS might be.

OSS advocates often emphasize superior stability or security of OSS tools over commercial alternatives. Whether these things are true or not, they are irrelevant to average home users; if Raymond's fictional everyuser, Aunt Tillie, can't figure out how to set up and use her PC, it's worthless to her. Putting up with the occasional crash or theoretical security glitch is worth it to her if it means she doesn't have to be a tech wizard or Unix hacker to get her machine to print photos and send email.

I am not ignorant, but I have my own equivalent of Aunt Tillie's problem. I know I want one of the top two methods, but I don't know which one. And I don't want to know or care about the difference either; I have better things to do with my brain than clutter it with sysadminning details.

Another good point; even though OSS gets most of its support within technical circles, it's not just nontechnical users that software with poor usability alienates. Raymond is far from representative of the technical skills of the average computer user, and even he barely succeeds at performing this seemingly simple operation with CUPS. I, too, am far more technical than your average Joe, but I've played out scenarios similar to Raymond's with my own linux server many, many times. Sometimes, after much wasted time, I succeed at my task. Other times I fail, and figure out an ugly workaround or just give up and go back to the rest of my life. Like ESR, I have better things to do with my brain.

The thing to notice here is how far behind we have left Aunt Tillie. Rule 1 of writing software for nontechnical users is this: if they have to read documentation to use it you designed it wrong . The interface of the software should be all the documentation the user needs. You'd have lost the non-techie before the point in this troubleshooting sequence where a hacker like me even got fully engaged.

Huzzah! Good documentation does not replace good interface design. And if the interface is poor, it's documentation is necessarily confusing, since a convoluted interface requires complicated instructions on its use. Too often in OSS, the solution to poor design is lengthy documentation. Proprietary software companies learned long ago that user's don't read documentation. It's time OSS learned the same lesson.

GUI tools and voluminous manuals are not enough. You have to think about what the actual user experiences when he or she sits down to do actual stuff, and you have to think about it from the user's point of view.

I could hardly have put it better myself, if someone asked me to capture in a nutshell the problem with OSS efforts to bring usability to their software. Open source projects are copying the trappings of well-designed software interfaces without understanding the substance. The result is lots of GUIs and manuals, but interaction designs that all too obviously had little to no thought put into them. This must change, if OSS is to compete with proprietary software in anything other than the (shrinking) niche of highly technical audiences.

I'll note that I had such a hard time believing my eyes while reading this article that I checked the domain name repeatedly to make sure it was really published by Eric Raymond.

None of this is rocket science. The problem isn't that the right things are technically difficult to do; CUPS is already supposed to have discovery of active shareable queues as a feature. The problem is that the CUPS designers' attitude was wrong. They never stepped outside their assumptions. They never exerted the mental effort to forget what they know and sit down at the system like a dumb user who's never seen it before — and they never watched a dumb user in action!

Again, this is right on (although I might take exception with referring to users as "dumb", if I wasn't in such a good mood). I've advocated bringing usability professionals and interaction designers into the open source community precisely to solve this problem; it's hard for anyone to step outside their assumptions and see a system they created as it is seen by those who are unfamiliar with it. It's especially hard for software developers who, while extremely skilled at producing quality code, are unskilled at understanding human limitations and seeing the system from the user's perspective. Hence the need for more heterogeneous skill sets within the OSS community.

CUPS is not alone. This kind of fecklessness is endemic in open-source land. And it's what's keeping Microsoft in business — because by Goddess, they may write crappy insecure overpriced shoddy software, but on this one issue their half-assed semi-competent best is an order of magnitude better than we usually manage.

When I read this section, I wasn't quite sure whether to dance for joy or go into shock.

It's clear that, beyond all hope, Raymond recognizes that there is a problem. With his influence, perhaps others will recognize it as well. And that, of course, is the first step towards a solution.

Because I believe it is a solvable problem. Although many OSS developers are currently naive about usability and interaction design methods, this can change. Although most interaction designers and usability professionals do not currently get involved with OSS projects, this too can change, if a place is made for them. And what's more, I highly suspect that a fully integrated, properly honed open source software community, with interaction designers, usability professionals, software developers, and end-users all working together, could produce software far superior in both design and technical quality than any proprietary company has yet been able to manage.

But that future does not yet exist, and it will take a lot of work to get there.

As I said before, the point of this essay is not especially to bash on the CUPS guys. They're no worse than thousands of projects out there, and that is the point. We talk about world domination, but we'll neither have it nor deserve it until we learn to do better than this. A lot better.

Amen, brother.

[ Permanent Location ]

Robotic Walker at CHI 2004!

announcements, life & times, usability

February 21, 2004, 01:29 PM

At Jodi's urging, Irina and Chung and I submitted an interactive poster to CHI 2004 on our social robotic walker interface design project. We found out just yesterday that it was accepted! Hurray!

Also, shouts out to Neema, Chad, Patrick, and Kevin for the acceptance of their poster on Apeer, their lightweight link sharing system, as well as to the CMU team for their acceptance to the design competition.

[ Permanent Location ]

Ready, Fire, Aim: Parallelizing Design and User Research

design, processes & methodologies, usability

February 18, 2004, 11:48 PM

For our capstone project, we're reaching the point in our process where we agreed to begin thinking about design. As I mentioned earlier, we want to overlap our design and research efforts to avoid getting caught up in wasting lots of time doing up-front research that doesn't turn out to be useful for our design efforts.

Bob calls this approach "Ready, Fire, Aim". The basic idea is to spend some time preparing yourselves by exploring the domain, reviewing literature, talking to domain experts, etc., but then quickly pick a focus and start down a design direction. These early designs may turn out to be totally off-base, but at least they will serve to point out the holes in the team's understanding of the users and suggest fruitful directions for additional research.

I remember a similar concept appearing in programming practice. The Pragmatic Programmers recommend an approach to architecture design they call "tracer bullets". The idea is to attempt to construct a functional end-to-end skeleton system to serve as an initial prototype of the proposed architectural design. Although you may throw this away, you'll learn where the flaws in your understanding of the problem domain lie so that you can design a better architecture and fire your next tracer.

The challenge we'll face, I expect, lies in keeping a clear vision of the team's current understanding of the users and their needs. This is likely to change with the designs as we uncover holes in our knowledge and fill them in. Our plan at the moment is to encode this understanding in our personas and task lists, hopefully using these as living documents as the design process progresses. Perhaps this will be sufficient, perhaps not. It will be interesting to see what this process produces and whether it will be successful.

[ Permanent Location ]

Divvying Up Research

design, usability

February 04, 2004, 11:07 PM

We've started our exploratory user and problem domain research phase for our capstone project. One of the hard things about managing a sizable research project is figuring out how to divide up the labor; research (and design) doesn't lend itself well to task division since all design team members really need to have an understanding of all research conducted so that they can develop appropriate design solutions. Yet if the entire team must be involved in all research activities, then this 1) introduces serious coordination overhead and 2) limits the amount of research the team can conduct since so little can occur in parallel. What to do?

The approach we are taking is to divide up the overall research task into five areas we consider important and assigning leadership of each area to a particular team member. The five areas are:

1. Goals and attitudes of the teachers
2. Goals and attitudes of the students
3. Advantages and limitations of existing competing products
4. Fitting the product into the user's lifestyle
5. Fitting the product into the classroom process

This approach seems to be working reasonably well so far in that every team member seems clear on what his or her responsibilities are and has been able to pick up the task, break it down into action items, and run with it. However, we don't want the group to get too divided; there's a risk that we'll all just go our separate ways without any clear idea of how these subtasks are fitting into the larger goal of understanding the design problem so that we can develop effective solutions. To mitigate this risk, we're going to try overlapping our tasks so that we can all participate in many activities and make a point of reviewing our findings in our meetings.

A second risk during the research phase lies in doing lots of research without knowing what you're getting out of it, then finding at the end that the questions you asked weren't relevant to your design problem whereas there were many questions you needed answers to that you never asked. To avoid research that goes nowhere, we're aiming to ensure that all research activities fit into a set of documents to guide our design:

1. A list of personas from our two target user groups (teachers and students)
2. A set of visionary scenarios describing how our product will be used
3. A list of tasks that the interface must support (the user-centered equivalent of a software engineering functional specification)

We also plan to begin active design ideation after only about three weeks of "pure" research activities. Hopefully, this will help us avoid following research directions that are not actively informing our design.

[ Permanent Location ]

Movable Type Needs Multithreaded Rebuilds

software development, usability

January 31, 2004, 09:40 PM

I've noticed that posting a comment on roBlog takes quite a bit longer than it used to since I released Heimdall. Although it fortunately does not take so long that browsers time out the request, it takes long enough to seriously impact the usability of the system; at best many posters will be annoyed by the delay, and at worst may think the server is not connecting and may try submitting the same comment again. Certainly the lag is far longer than the 1 second optimum response time and even exceeds the maximum 10 seconds. I clocked it at close to 20.

The problem, I believe, is that Movable Type rebuilds all weblog pages that may be altered when a comment is posted. Since roBlog has many index pages and complex template code, this takes some time, and that contributes to the long post time. For this very reason, Movable Type doesn't bother to rebuild archives when it receives a Trackback ping, since the tool that sent the ping may have even less patience than a human (or something...).

So here's my totally unsolicited advice to Ben Trott: don't rebuild the weblog files during the CGI request! If CGI doesn't allow for the execution of code on its thread after the HTTP response has completed (I'll admit I've never tried such a thing), then spawn another thread to finish the rebuild task in the background. Only generate sufficient information to provide the user (the comment poster or trackback sender) with enough feedback for them to realize their action was successful; the rest of the rebuild can wait until the request is closed to proceed.

And if you don't want to do it yourself, keep in mind I'll be looking for a job starting in August...

[ Permanent Location ]

When Reuse Is Bad For Real Use

software development, usability

January 28, 2004, 11:26 PM

Joel has an interesting article up on the hassle of Microsoft's .NET runtime environment. Joel prefers the "old" approach of using a linker to bind up all the required libraries and resources with the program itself. Despite my love for Java, which is also afflicted with a runtime environment, I agree with him.

Although it is widely considered good software engineering practice to separate reusable code into its own modules, the approach of using system-level shared code libraries takes this too far (run-time environments are essentially just collections of these libraries). Having multiple applications access a shared code base foists the task of upgrading and maintaining that code base on the user. Linux is notorious for this sin; users must often upgrade many shared libraries before a new application will work, and a result software installation is a nightmare. Windows is better nowadays since they developed Windows Update, but as Joel pointed out, with the frequent calls for reboots and other task interruptions, it's still a major annoyance at least.

The Macintosh has always considered each application its own separate entity, and hasn't really had a concept of shared libraries. As a result, software installation (and backup) is painless; just drag an application into a folder. I'm no Apple worshipper, but they've had the right idea all along here. Application developers should worry about whether they have the right version of the right libraries available when they ship, and that should be the end of it. Modularization has many benefits, but none of them directly effect the user. It's just none of their business.

[ Permanent Location ]

A Note on Terminology

design, language, usability

December 26, 2003, 12:31 AM

As I was writing my post yesterday, I wanted to add a note on my usage of a few key terms here on this weblog, to clarify any possible confusions that might arise.

When I use the term design in reference to software systems, I am sometimes referring to interface design, i.e. the form and function of the portion of the software system that the user of the product sees and interacts with, and sometimes referring to internal design, the structure and specifications of the logical components that power the behavior of this interface. Although I recognize that this is generally a useful distinction to make, I tend to view these two disciplines as different aspects of the same activity. There is no question that the two influence one another. Because these two concepts are blurred in my mind, the word I use for them is fairly blurry as well. For the most part this is intentional.

Secondly, I tend to use the term usability rather more broadly than others do (or so I've found). It's convention in the field to make a distinction between "useful, usable, and desirable" as quality characteristics of software systems (and many other systems, as well). Useful refers to whether the software does what the user needs it to, usable refers to whether he can learn and remember how to use the software, and use it efficiently, and desirable refers to whether he wants to use the software. I often find this distinction silly, since there's so much overlap between these three concepts, and thus I tend to lump them all under the moniker "usable", i.e., to what extent can people use it? I've already argued that all software quality attributes reduce to usability. Now I feel like I need to argue that all usability attributes reduce to usability (which strikes me as a bit silly).

Finally, I have no good word to refer to the group of people who are members of the broad profession that aims to design products and systems for use by human beings. The general term for this discipline is often "human (or user) centered design" but the term "human-centered designer" often isn't seen as applying to user researchers, UI programmers, etc. who aren't necessarily designing an interface. But no other word will do. "Usability person" tends to get rejected by the designers. There are a plethora of more specific terms (interaction designer, information architect, interaction architect, UI designer, interface designer, user researcher, usability specialist, "gooey" (from Yahoo), user tester, usability engineer, usability professional, or generate your own) My friend Dana made up the word "usable" but nobody seems too fond of that either. So I might use any sufficiently general-sounding term to refer to the whole lot of them, until one emerges that everyone can agree on.

I don't believe that any of these issues are idiosyncratic to me, but rather reveal some inherent ambiguities in the way we talk about these things. However, this post should be taken as more of a clarification than a complaint. Ambiguity in terminology is confusing, but inevitable. Human language is riddled with ambiguity, which is part of what makes it such a beautiful and powerful thing. Some of the ambiguities I've pointed out above are (in my view) necessary, and I have faith that the others will sort themselves out in time.

[ Permanent Location ]