usability

About

Syndication

Categories

Archives

On Being a User Researcher

society & sociology, usability

May 21, 2004, 07:18 PM

I've been thinking recently about user research (by which I mean formative research techniques that attempt to understand how potential users live their lives, what problems they have, what their goals are, etc.) and what it takes to be a good user researcher.

There are many skills that a person who hopes to do user research must acquire. For instance, a good user researcher must learn how to put interviewees at ease while still maintaining sufficient objectivity by avoiding misleading questions, etc. A good user researcher must be able to moderate a focus group by keeping it on topic without stifling discussion, avoiding group-think, and ensuring everyone's opinions get heard. A good user researcher needs to be a good listener. He must keep good notes. He must keep his biases out of the results. And so on.

But there is one quality that I believe all user researchers need to have to be effective, and it's never taught nor even mentioned in any training programs I know of. A good user researcher has to like people.

And I mean he has to like people, all people, not just his friends, peers, and colleagues. He has to be intrinsically interested in who they are, what work they do, how they feel about life, what they want to achieve, and so on. Even though user research is generally conducted to inform product design or marketing, in order to do his job well, a good user researcher must possess an interest in learning about people that transcends these goals. Gaining a deep understanding of how others live their lives must be an end in and of itself.

This is different from the goals of a psychologist or social scientist. Scientists who study humans hope to obtain generalizable knowledge about how humans behave. They uncover facts about human behavior; often these are abstract and somewhat removed in their statistical generality from the real thoughts and feelings of individuals. User researchers, on the other hand, do not uncover generalizable facts; instead they hunt for an understanding of the qualia, the what it's like to be a member of their target population. Though useful in and of themselves, psychology and social science aren't user research professions, nor can they be expected to adequately train user researchers.

I rather suspect that a mature field of user research would more closely resemble journalism than psychology. A good journalist goes out into the world and hunts for stories that will be interesting to their intended audience. A user researcher's audience is the design team. Their job is to uncover stories about their users that would be interesting to the designers while also accurately portraying the people they are designing for. To do this well requires many skills, but foremost among them is an intrinsic interest in and love for people that few other professions in this world truly call for.

[ Permanent Location ]

Design, Usability, and Innovation

design, usability

April 18, 2004, 08:54 AM

A few weeks ago, I went to a talk given by Hugh Dubberly on models for design (Dan has a very nice summary, as usual). During the talk, Hugh made a point about design and innovation that's stuck with me; he remarked You don't get innovation from the design process. You get quality from the design process. At the time, that struck me as quite true.

Since then, however, I've had second thoughts. Let's consider an "innovation" (an overloaded word) to refer to "some technology-related change that has a significant impact on the ways people behave, generally for the better". I think that captures it reasonably well for my current purposes. Usually when we think of innovations, we think of some brand new ideas that spark new technological developments, like televisions or automobiles. And its true that the design process won't get you these; great new ideas are elusive things and so far no one has found a process to repeatably elicit them (though there are some brainstorming techniques that can help).

But there are times when innovations are not spawned by brand new ideas. There are times when the innovation involves taking an existing technology and making it accessible to a new (probably wider) audience. Or in taking a technology that used to be complex and cumbersome to use and making it quick and simple. Suddenly, the technology is accessible and it becomes an innovation.

The GUI desktop metaphor is a classic example. Affordable computers were available before PARC's innovation got commercialized, but only hobbyists bought them, because they were just too tough for the average person to use. With a well-designed interface, however, personal computers became a reality and an innovation was born.

iChat AV is a more modern (if also more modest) example. Voice-over-IM has existed for a long time in AIM, but nobody used it. All Apple did was make it dead simple to use. Suddenly it becomes useful to the masses, and we have innovation.

Style sheets and the single-sourcing paradigm might be a design innovation waiting to happen. Style sheets are useful to some but inaccessible to most due to the complexity of dealing with the badly designed interfaces for creating and applying them. If someone were to create a more usable interface, we may have innovation.

This is, I believe, the role that design plays in innovation. Not in consistently generating world-changing ideas (these come from a multitude of disciplines) but in turning these ideas into something simple enough to affect the lives of thousands.

[ Permanent Location ]

Marketing and User Research

design, usability

April 16, 2004, 11:01 PM

I've been thinking recently about the relationship between market research and user research.

Market research is conducted by marketing departments, who are mainly concerned with figuring out what sorts of products people are willing to buy and what sorts of people are willing to buy them. I am very ignorant of marketing research techniques, but my current understanding is that they tend to involve conducting surveys and possibly focus groups with the intent of nailing down a market demographic that the company can sell some new product to. Marketing may also collect information on what features and attributes of this product are important to swaying this demographics' buying decisions.

User research is conducted by user-centered designers, who are mainly concerned with producing the interface and interaction design of the new product. As Beyer and Holtzblatt point out in Contextual Design, the kind of data necessary to drive product design is different from the kind of data collected by marketing; market research alone cannot drive research-informed product designs. UCDs require more detailed information about what the people behind the statistics are like, what their goals are, how they are currently performing their tasks, etc. B&H have already made this argument eloquently so I won't bother rehashing it here.

All this isn't to say that marketing's data is useless to UCDs, however. In fact, a reasonably thorough market analysis is absolutely necessary to have ready before design research begins. If marketing hasn't narrowed down the target user population, design won't know where to start looking to collect their data. They may wind up taking a scattershot approach and (at best) waste time talking to the wrong types of people or (at worst) design a product from data on the wrong users.

Just as marketing's data can't substitute for design data, design shouldn't be expected to collect market data. Because design research involves more detail-oriented studies, these studies cannot uncover a broad picture of the market needs on a large scale. A mistake I've seen among novice design researchers is to spend time sending out surveys to uncover a market for the product when this information was basically provided to them by the marketing team. This is a waste of time, not to mention an inappropriate application of skill sets. Design should be focused on how the real people behind the demographic numbers marketing generated are working, communicating, and feeling, for this is the information they need to create great products.

[ Permanent Location ]

The Usability of Security

usability

March 30, 2004, 04:07 PM

Neema is working on a project to improve the usability of wireless network security. As I've touched on before, I think the usability of security is an underexplored but very important topic, so I'm glad to see that Neema is thinking about it.

Since he asked for feedback, I'm giving my view (for what it's worth) of how security should work as a user-centered design paradigm. Then I'll try to relate it back to wireless networking.

There is basically only one UCD problem in the security space. Users, organizations, and systems (security people call them "principles" I believe) need to identify what other users, organizations, and systems they trust, and which ones they do not. They may also need to specify further what information they trust each other party with, and what information should be kept hidden. The big interface design question, then, is how to make it quick and easy (remember that specifying security settings is rarely the user's main goal for any system) for people to make these distinctions. Once the appropriate distinctions are made, the software should determine what technologies (WEP, Kerberos, SSL, or whatever) are appropriate to guarantee the required level of trust, and employ them. This part shouldn't concern the user.

I know this is abstract, and determining how to realize this in any individual system is a decision that the designers of each system must make on a case-by-case basis. But it's important to think of the problem in these terms, and not in terms of "how can I teach users the advantages of using SSL?" Users don't know about SSL and don't want to know about SSL. They just know who they trust, and that must be enough.

So what might this mean for wireless security? Well, when a user logs on to a wireless network, their user agent (probably their computer) needs to know whether they trust the person who is administering that network. If the user is at Starbucks, then they must indicate whether they trust Starbucks Corp. to faithfully transmit their data. Maybe the interface must allow the user to directly specify this information. Maybe it only needs to show the user what other people they are currently "trusting" with their information. But these are the sorts of design questions I'd like to see addressed. Everything else should Just Work™.

[ Permanent Location ]

Sorting Environments

design, usability

March 14, 2004, 09:28 PM

My friend Jeff Howard was showing me a research project he'd been working on for his miLife project a couple weeks ago, and now he's posted about it. He's basically conducting a card sorting exercise, except the cards contain pictures of various kinds of environments like conferences halls, bus interiors, scenic natural vistas, etc. His team is asking users to arrange the cards along a continuum of which environments they feel more or less mobile in. I believe they're also asking users to arrange the environments along another continuum of how remotely connected to others they feel and how connected they wish to feel. Note that Jeff's online prototype doesn't work as well due to the small size of the cards; the actual physical cards are larger.

It's an interesting spin on the typical card sorting procedure; the pictures may evoke a more emotional response from users which is more appropriate for this particular study than the short words used in navigation hierarchy studies. It also helps give the users more context than a typical interview question such as "name five places you feel mobile in" would, while avoiding the cost of having to find users actually sitting around in all these environments.

[ Permanent Location ]

Newsable Redesign Mockups and a Request for Feedback

design, usability

March 08, 2004, 10:07 PM

I'm working on giving Newsable a facelift soon, both to improve the interaction and visual design. I've posted some mockups of the new key screens; if you're the sort of person who likes to speak words about other people's work, then here's your chance to speak some words about mine. My goals with this release are to make the visual design a little more pleasing and to improve the interaction for people who are subscribed to a large number of feeds. Also, I hope to convince Kenneth that the three-pane format isn't the be all and end all of aggregator design.

I'm also working on the design of a community system that will help facilitate a usable open source software process. The design is based on the research I did last semester in CSCW. If this sounds intriguing to you, stay tuned for more.

[ Permanent Location ]

Eric Raymond: Open-Source Usability Advocate?!

software development, usability

February 27, 2004, 10:01 PM

I was idly surfing the blogspace (doesn't that sound so much nicer than "blogosphere"?) this afternoon when I happened across Julian Missig's blog (a fellow CMU student and a co-worker of Dana's, of all things). He points to an article hot off the press by Eric Raymond where the famous OSS guru actually argues in favor of fixing the open source software usability problem! I could barely believe it until I read the article; frequent readers will recall that I've been studying the OSS usability problem for several months now. But though I've found some encouraging evidence of OSS developers who are starting to get it, I never, ever thought I'd read an article like this from ESR. Some of his suggestions were right on, others I'm not so sure about, but the core ideas about the ways open source software needs to change are right on the money.

I've sliced out some particularly interesting bits below, with commentary.

Aunt Tillie, at this point, is either resigning herself to another session of being tortured by the poor UI choices of well-meaning idiots or deciding to chuck this whole Linux thing and go back to the old Windows box. It blue-screened a lot, but at least it allowed her the luxury of ignorance — she didn't have to know, or care, about what a JetDirect or a CUPS might be.

OSS advocates often emphasize superior stability or security of OSS tools over commercial alternatives. Whether these things are true or not, they are irrelevant to average home users; if Raymond's fictional everyuser, Aunt Tillie, can't figure out how to set up and use her PC, it's worthless to her. Putting up with the occasional crash or theoretical security glitch is worth it to her if it means she doesn't have to be a tech wizard or Unix hacker to get her machine to print photos and send email.

I am not ignorant, but I have my own equivalent of Aunt Tillie's problem. I know I want one of the top two methods, but I don't know which one. And I don't want to know or care about the difference either; I have better things to do with my brain than clutter it with sysadminning details.

Another good point; even though OSS gets most of its support within technical circles, it's not just nontechnical users that software with poor usability alienates. Raymond is far from representative of the technical skills of the average computer user, and even he barely succeeds at performing this seemingly simple operation with CUPS. I, too, am far more technical than your average Joe, but I've played out scenarios similar to Raymond's with my own linux server many, many times. Sometimes, after much wasted time, I succeed at my task. Other times I fail, and figure out an ugly workaround or just give up and go back to the rest of my life. Like ESR, I have better things to do with my brain.

The thing to notice here is how far behind we have left Aunt Tillie. Rule 1 of writing software for nontechnical users is this: if they have to read documentation to use it you designed it wrong . The interface of the software should be all the documentation the user needs. You'd have lost the non-techie before the point in this troubleshooting sequence where a hacker like me even got fully engaged.

Huzzah! Good documentation does not replace good interface design. And if the interface is poor, it's documentation is necessarily confusing, since a convoluted interface requires complicated instructions on its use. Too often in OSS, the solution to poor design is lengthy documentation. Proprietary software companies learned long ago that user's don't read documentation. It's time OSS learned the same lesson.

GUI tools and voluminous manuals are not enough. You have to think about what the actual user experiences when he or she sits down to do actual stuff, and you have to think about it from the user's point of view.

I could hardly have put it better myself, if someone asked me to capture in a nutshell the problem with OSS efforts to bring usability to their software. Open source projects are copying the trappings of well-designed software interfaces without understanding the substance. The result is lots of GUIs and manuals, but interaction designs that all too obviously had little to no thought put into them. This must change, if OSS is to compete with proprietary software in anything other than the (shrinking) niche of highly technical audiences.

I'll note that I had such a hard time believing my eyes while reading this article that I checked the domain name repeatedly to make sure it was really published by Eric Raymond.

None of this is rocket science. The problem isn't that the right things are technically difficult to do; CUPS is already supposed to have discovery of active shareable queues as a feature. The problem is that the CUPS designers' attitude was wrong. They never stepped outside their assumptions. They never exerted the mental effort to forget what they know and sit down at the system like a dumb user who's never seen it before — and they never watched a dumb user in action!

Again, this is right on (although I might take exception with referring to users as "dumb", if I wasn't in such a good mood). I've advocated bringing usability professionals and interaction designers into the open source community precisely to solve this problem; it's hard for anyone to step outside their assumptions and see a system they created as it is seen by those who are unfamiliar with it. It's especially hard for software developers who, while extremely skilled at producing quality code, are unskilled at understanding human limitations and seeing the system from the user's perspective. Hence the need for more heterogeneous skill sets within the OSS community.

CUPS is not alone. This kind of fecklessness is endemic in open-source land. And it's what's keeping Microsoft in business — because by Goddess, they may write crappy insecure overpriced shoddy software, but on this one issue their half-assed semi-competent best is an order of magnitude better than we usually manage.

When I read this section, I wasn't quite sure whether to dance for joy or go into shock.

It's clear that, beyond all hope, Raymond recognizes that there is a problem. With his influence, perhaps others will recognize it as well. And that, of course, is the first step towards a solution.

Because I believe it is a solvable problem. Although many OSS developers are currently naive about usability and interaction design methods, this can change. Although most interaction designers and usability professionals do not currently get involved with OSS projects, this too can change, if a place is made for them. And what's more, I highly suspect that a fully integrated, properly honed open source software community, with interaction designers, usability professionals, software developers, and end-users all working together, could produce software far superior in both design and technical quality than any proprietary company has yet been able to manage.

But that future does not yet exist, and it will take a lot of work to get there.

As I said before, the point of this essay is not especially to bash on the CUPS guys. They're no worse than thousands of projects out there, and that is the point. We talk about world domination, but we'll neither have it nor deserve it until we learn to do better than this. A lot better.

Amen, brother.

[ Permanent Location ]

Robotic Walker at CHI 2004!

announcements, life & times, usability

February 21, 2004, 01:29 PM

At Jodi's urging, Irina and Chung and I submitted an interactive poster to CHI 2004 on our social robotic walker interface design project. We found out just yesterday that it was accepted! Hurray!

Also, shouts out to Neema, Chad, Patrick, and Kevin for the acceptance of their poster on Apeer, their lightweight link sharing system, as well as to the CMU team for their acceptance to the design competition.

[ Permanent Location ]

Ready, Fire, Aim: Parallelizing Design and User Research

design, processes & methodologies, usability

February 18, 2004, 11:48 PM

For our capstone project, we're reaching the point in our process where we agreed to begin thinking about design. As I mentioned earlier, we want to overlap our design and research efforts to avoid getting caught up in wasting lots of time doing up-front research that doesn't turn out to be useful for our design efforts.

Bob calls this approach "Ready, Fire, Aim". The basic idea is to spend some time preparing yourselves by exploring the domain, reviewing literature, talking to domain experts, etc., but then quickly pick a focus and start down a design direction. These early designs may turn out to be totally off-base, but at least they will serve to point out the holes in the team's understanding of the users and suggest fruitful directions for additional research.

I remember a similar concept appearing in programming practice. The Pragmatic Programmers recommend an approach to architecture design they call "tracer bullets". The idea is to attempt to construct a functional end-to-end skeleton system to serve as an initial prototype of the proposed architectural design. Although you may throw this away, you'll learn where the flaws in your understanding of the problem domain lie so that you can design a better architecture and fire your next tracer.

The challenge we'll face, I expect, lies in keeping a clear vision of the team's current understanding of the users and their needs. This is likely to change with the designs as we uncover holes in our knowledge and fill them in. Our plan at the moment is to encode this understanding in our personas and task lists, hopefully using these as living documents as the design process progresses. Perhaps this will be sufficient, perhaps not. It will be interesting to see what this process produces and whether it will be successful.

[ Permanent Location ]

Divvying Up Research

design, usability

February 04, 2004, 11:07 PM

We've started our exploratory user and problem domain research phase for our capstone project. One of the hard things about managing a sizable research project is figuring out how to divide up the labor; research (and design) doesn't lend itself well to task division since all design team members really need to have an understanding of all research conducted so that they can develop appropriate design solutions. Yet if the entire team must be involved in all research activities, then this 1) introduces serious coordination overhead and 2) limits the amount of research the team can conduct since so little can occur in parallel. What to do?

The approach we are taking is to divide up the overall research task into five areas we consider important and assigning leadership of each area to a particular team member. The five areas are:

1. Goals and attitudes of the teachers
2. Goals and attitudes of the students
3. Advantages and limitations of existing competing products
4. Fitting the product into the user's lifestyle
5. Fitting the product into the classroom process

This approach seems to be working reasonably well so far in that every team member seems clear on what his or her responsibilities are and has been able to pick up the task, break it down into action items, and run with it. However, we don't want the group to get too divided; there's a risk that we'll all just go our separate ways without any clear idea of how these subtasks are fitting into the larger goal of understanding the design problem so that we can develop effective solutions. To mitigate this risk, we're going to try overlapping our tasks so that we can all participate in many activities and make a point of reviewing our findings in our meetings.

A second risk during the research phase lies in doing lots of research without knowing what you're getting out of it, then finding at the end that the questions you asked weren't relevant to your design problem whereas there were many questions you needed answers to that you never asked. To avoid research that goes nowhere, we're aiming to ensure that all research activities fit into a set of documents to guide our design:

1. A list of personas from our two target user groups (teachers and students)
2. A set of visionary scenarios describing how our product will be used
3. A list of tasks that the interface must support (the user-centered equivalent of a software engineering functional specification)

We also plan to begin active design ideation after only about three weeks of "pure" research activities. Hopefully, this will help us avoid following research directions that are not actively informing our design.

[ Permanent Location ]

Movable Type Needs Multithreaded Rebuilds

software development, usability

January 31, 2004, 09:40 PM

I've noticed that posting a comment on roBlog takes quite a bit longer than it used to since I released Heimdall. Although it fortunately does not take so long that browsers time out the request, it takes long enough to seriously impact the usability of the system; at best many posters will be annoyed by the delay, and at worst may think the server is not connecting and may try submitting the same comment again. Certainly the lag is far longer than the 1 second optimum response time and even exceeds the maximum 10 seconds. I clocked it at close to 20.

The problem, I believe, is that Movable Type rebuilds all weblog pages that may be altered when a comment is posted. Since roBlog has many index pages and complex template code, this takes some time, and that contributes to the long post time. For this very reason, Movable Type doesn't bother to rebuild archives when it receives a Trackback ping, since the tool that sent the ping may have even less patience than a human (or something...).

So here's my totally unsolicited advice to Ben Trott: don't rebuild the weblog files during the CGI request! If CGI doesn't allow for the execution of code on its thread after the HTTP response has completed (I'll admit I've never tried such a thing), then spawn another thread to finish the rebuild task in the background. Only generate sufficient information to provide the user (the comment poster or trackback sender) with enough feedback for them to realize their action was successful; the rest of the rebuild can wait until the request is closed to proceed.

And if you don't want to do it yourself, keep in mind I'll be looking for a job starting in August...

[ Permanent Location ]

When Reuse Is Bad For Real Use

software development, usability

January 28, 2004, 11:26 PM

Joel has an interesting article up on the hassle of Microsoft's .NET runtime environment. Joel prefers the "old" approach of using a linker to bind up all the required libraries and resources with the program itself. Despite my love for Java, which is also afflicted with a runtime environment, I agree with him.

Although it is widely considered good software engineering practice to separate reusable code into its own modules, the approach of using system-level shared code libraries takes this too far (run-time environments are essentially just collections of these libraries). Having multiple applications access a shared code base foists the task of upgrading and maintaining that code base on the user. Linux is notorious for this sin; users must often upgrade many shared libraries before a new application will work, and a result software installation is a nightmare. Windows is better nowadays since they developed Windows Update, but as Joel pointed out, with the frequent calls for reboots and other task interruptions, it's still a major annoyance at least.

The Macintosh has always considered each application its own separate entity, and hasn't really had a concept of shared libraries. As a result, software installation (and backup) is painless; just drag an application into a folder. I'm no Apple worshipper, but they've had the right idea all along here. Application developers should worry about whether they have the right version of the right libraries available when they ship, and that should be the end of it. Modularization has many benefits, but none of them directly effect the user. It's just none of their business.

[ Permanent Location ]

A Note on Terminology

design, language, usability

December 26, 2003, 12:31 AM

As I was writing my post yesterday, I wanted to add a note on my usage of a few key terms here on this weblog, to clarify any possible confusions that might arise.

When I use the term design in reference to software systems, I am sometimes referring to interface design, i.e. the form and function of the portion of the software system that the user of the product sees and interacts with, and sometimes referring to internal design, the structure and specifications of the logical components that power the behavior of this interface. Although I recognize that this is generally a useful distinction to make, I tend to view these two disciplines as different aspects of the same activity. There is no question that the two influence one another. Because these two concepts are blurred in my mind, the word I use for them is fairly blurry as well. For the most part this is intentional.

Secondly, I tend to use the term usability rather more broadly than others do (or so I've found). It's convention in the field to make a distinction between "useful, usable, and desirable" as quality characteristics of software systems (and many other systems, as well). Useful refers to whether the software does what the user needs it to, usable refers to whether he can learn and remember how to use the software, and use it efficiently, and desirable refers to whether he wants to use the software. I often find this distinction silly, since there's so much overlap between these three concepts, and thus I tend to lump them all under the moniker "usable", i.e., to what extent can people use it? I've already argued that all software quality attributes reduce to usability. Now I feel like I need to argue that all usability attributes reduce to usability (which strikes me as a bit silly).

Finally, I have no good word to refer to the group of people who are members of the broad profession that aims to design products and systems for use by human beings. The general term for this discipline is often "human (or user) centered design" but the term "human-centered designer" often isn't seen as applying to user researchers, UI programmers, etc. who aren't necessarily designing an interface. But no other word will do. "Usability person" tends to get rejected by the designers. There are a plethora of more specific terms (interaction designer, information architect, interaction architect, UI designer, interface designer, user researcher, usability specialist, "gooey" (from Yahoo), user tester, usability engineer, usability professional, or generate your own) My friend Dana made up the word "usable" but nobody seems too fond of that either. So I might use any sufficiently general-sounding term to refer to the whole lot of them, until one emerges that everyone can agree on.

I don't believe that any of these issues are idiosyncratic to me, but rather reveal some inherent ambiguities in the way we talk about these things. However, this post should be taken as more of a clarification than a complaint. Ambiguity in terminology is confusing, but inevitable. Human language is riddled with ambiguity, which is part of what makes it such a beautiful and powerful thing. Some of the ambiguities I've pointed out above are (in my view) necessary, and I have faith that the others will sort themselves out in time.

[ Permanent Location ]

A Usable Open Source Software Community

design, society & sociology, software development, usability

December 18, 2003, 10:42 PM

I've written a lot before on the subject of bringing usability to open source software. I've been silent on the topic recently, but not idle; I've been working diligently all semester long with my friend Dana Gelman for our CSCW course on a project to design an open source community that has the tools, techniques, and participants to develop products that can effectively target end-users.

Our core recommendation is that existing open source communities that develop applications targeting end-users (for our purposes, "end-users" will be shorthand for "users that are not like the developers") should invite usability professionals and interaction designers to participate in the development efforts. This is harder than it sounds, however, since existing, developer-centric communities will have to change the way they work to make room for these new people with new skills. To begin to paint a picture of what this new, integrated community might look like, we have developed eight specific recommendations for any OSS community that targets end-users.

1. Involve usables in the core development team. Our literature search has shown that most OSS projects have a small team of one or more developers that do the bulk of the work. On large projects, this team is responsible for decisions relating to the high-level architecture and organization of the code. We propose bringing interface designers into this team to take responsibility for the user experience, including defining and maintaining the core metaphors, ensuring interface consistency, and other high-level duties. These participants will also need to interact closely with the core developers to determine implementation tradeoffs, which is why they're part of the same core team.
2. Incorporate personas. I've already argued extensively for this recommendation, so I won't say much more of it here, except to note that to be effective, the personas must be used and valued (in particular, they should be frequently referenced by name) in design conversations throughout the community.
3. Build small, heterogeneous teams of developers and usability people. Small teams are more productive, and can focus on the details of an interface design with a minimum of overhead.
4. Modify the joiner script to require newcomers to justify design decisions with the personas. Open source communities have an implicit cultural barrier to entry called the "joiner script" that sets a standard that new members must live up to in order to become contributors to the project (think of it as a very long interview process). We recommend not admitting contributors (both developers and usability people) who are unwilling to justify design changes and feature requests in terms of the personas' needs.
5. Use synchronous communication (chat) for design meetings. Synchronous communication is more efficient at building shared understanding, which is essential for making effective design decisions.
6. Transmit design decisions back to the public list. The previous recommendation introduces a problem, however. People who are outside the immediate circle of the design team (contributors working on other parts of the project, contributor hopefuls, users, etc) also need to be kept in the loop about the design decisions that are made and the rational for them. For this reason, someone needs to be willing to summarize the synchronous design conversation and post it to the asynchronous communication medium in use by the project (e.g., a mailing list or discussion board) so these other followers can get access to it.
7. Provide a shared visual information space. We recommend using a shared, visual information space (like a shared whiteboard application) in parallel with synchronous chat for design meetings. Design is a very visual process, so having the easily accessible graphical space to work on rather than trying to describe all interface changes linguistically is necessary.
8. Modify bug tracking tools to better support collecting usability issues. Bug tracking tools are often used to record change requests as well as bugs in open source projects. On large projects, however, we found that they often break down as large numbers of UI problems pile in and contributors must spend time sorting through them. We propose developing more efficient bug tracking tools that are capable of handling UI issues as well as code bugs. We have not yet defined the form such a system must take, since this is really another semester project in and of itself.

Dana and I have put together a final report that contains more comprehensive descriptions and extensive arguments for these changes, ties our assertions back to the literature, and describes an experiment we are proposing to test one aspect of our solution (how personas change the design conversation in an integrated open source and usability community). I haven't made this paper publicly available (yet) since we're hoping to publish portions of it at CHI or CSCW and I don't want to run afoul of prior publication rules. But if you're really interested, drop me an email and I might send you a copy.

[ Permanent Location ]

Don't Make Me Flip Out and Design User Interfaces

funny, usability

December 08, 2003, 01:23 PM

You all remember the Official Ninja Webpage, right? That was nothing. HCI students are the Real Ultimate Power!

Read it, or I'll leave you out of my target audience.

[ Permanent Location ]

Maintenance Projects and U&SA

processes & methodologies, software development, usability

November 22, 2003, 07:06 PM

Turns out Dan wrote an article for Boxes and Arrows about a year ago titled "Tackling Maintenance Projects". The article is excellent and I recommend reading it. What I found most interesting, however, is how Dan's advice relates to the impact of software architecture on usability.

Dan draws a distinction between "Atomic Maintenance Projects" (more or less complete redesigns, of potentially both the interface and the underlying implementation) and "Neutron Maintenance Projects" (small changes that serve as point solutions but may not address the larger usability issues). The problem he observes is that often your client thinks they only need a Neutron fix, but after inspecting the interface you realize that an Atomic redesign is necessary to achieve adequate usability. In his words:

And there’s the rub. Often, IAs are left to wrestle with decisions that were made long before their involvement with the project; decisions that people in the company may not even like but are stuck with. Or decisions they have a vested stake in keeping intact. Or decisions various forces in the company (IT, Marketing, and Legal being typical stakeholders) insist must remain, for reasons the IA may or may not agree with.

Our hope with U&SA is that by bringing usability professionals (or designers or information architects or what have you) into the process at the system architecture design stage and giving them the intellectual tools they need to understand how to effectively contribute, we can reduce the need for Atomic projects so that more usability issues can be dealt with at the Neutron project level, which, as Dan correctly points out, is often insufficient given the current state of the practice.

[ Permanent Location ]

A Foundation for HCI?

philosophy, usability

November 22, 2003, 12:10 PM

I've written before about Graduate Design Seminar, a class I'm not actually in but which I'm experiencing vicariously through Dan. A couple days ago, I was talking to Dave Holstius, an HCI PhD student who's also taking the class. He feels that the HCII needs a similar class that provides a firm philosophical foundation for the discipline and an overview of its intellectual history. Sadly, there isn't really anyone in our institute who is qualified to teach such a course. We need a Dick Buchanan of our own.

I have no idea where I'd even begin to uncover such a foundation on my own. Do any courses or texts even exist in the world that would trace the history of ideas behind HCI?

[ Permanent Location ]

Flexible Designs and the Role of the Designer

design, usability

November 18, 2003, 11:58 AM

My good friend Kerry has a post up on an art exhibit on rethinking the role of design in everyday life. The exhibit includes a project called Felt 12x12, which is essentially a collection of felt squares that the consumer can combine to be anything he wants. This got me thinking about the roles of the consumer and the designer and how much influence each should have over the product.

When designing a piece of technology, especially software technology, one thing you have to decide is the extent to which the end user should be able to configure the product. If you make the product completely rigid and allow for no reconfiguration, then you risk alienating the vast majority of your users who have special needs, plus you lose one of the main advantages of digital computers, their flexibility. On the other hand, if you make the product too configurable, you'll wind up with a jumbled mass of options that are hard to use and hard to maintain (from a programmer's perspective), most of which are rarely needed by the majority of your user base.

I've discussed the problem of designing for end-user customization before, in the context of open source interfaces. As I mentioned then, the HCI and Interaction Design literature tends to emphasize that it is the designer's job to make decisions about the form and function of the product; she should do all the work so that the user doesn't have to think. But the literature tends to approach this from a very narrow perspective; the study from Nielsen only looked into the effects of allowing users to reconfigure the interface's appearance on their time to complete a task. There are many other potential benefits of allowing some user customization, such as improving user satisfaction and supporting tasks the interface wasn't designed for. How much of this truism is the way things must be, and how much of it is self-interest on the part of designers?

What is the role of the designer in society? Is she always tasked with constructing fully-formed user experiences and "canned" design solutions, or must she often instead work to produce "a framework, within which consumers can define shape and form for themselves"? When constructing such a framework, the user's tasks by definition are poorly defined, similar to the problems confronted in designing interfaces to support creativity. Most of our methods rely on these tasks being well defined. Will our methods help us rise to this challenge, or must we develop new methods that better support configurability?

[ Permanent Location ]

Discussing Disconnects

design, processes & methodologies, software development, usability

November 15, 2003, 07:08 PM

At the SSS two days ago, I led a discussion on how we can bridge the gap between interaction designers, usability professionals, and software engineers working together on real-world development teams (as I promised I would). I started out with a quick presentation describing the personalities and goals of both "players" (designers and developers) through some rough personas, then opened the session up for discussion on what techniques might help in getting these two players to work together effectively.

Here's a list of the possibilities that were thrown out for consideration. If anyone out there in readerland was at the discussion and notices something I missed, please do add it in a comment.

1. Emphasize more up-front design so design and engineering need not work in parallel. The idea is that this will reduce the amount that the two players need to communicate, thus reducing conflicts and making the process more efficient. It was also pointed out, however, that although spending time on up-front design is good, it doesn't solve all the problems. The final design, no matter how good, will still need to change quite a bit based on engineering's input and concerns, thus reintroducing the communication problem.
2. Design must bring engineering onboard. Those who came from an engineering background emphasized the need for design to justify and "sell" their design solutions to the engineering team. The idea is that although engineering trusts design to be good at their job, they will be more effective participants in the overall development process if they understand what the important usability issues are and why the related design decisions were made.
3. Design and development must "live" together. Basically, the two players should work closely throughout the entire process; there should be no "throw it over the wall" style information exchanges. I've suggested this before in the context of XP, going so far as to recommend that the two players work together in the same room.
4. Designers should contribute to development work. Many designers have enough technical abilities to contribute to parts of the user interface development process. In addition to speeding things along, this gives engineering a better sense that the designers are true members of the team. Cooper and Reimann suggest that one individual cannot fill the role of a designer and developer, due to the fundamentally different mindsets, but everyone in the group thought it was possible to strike a happy medium. In order to facilitate this technique, the software architecture must separate the user interface modules from the rest of the application using a separation pattern such as Model-View-Controller (which still won't handle all concerns, of course).
5. Designers should have engineers sit in on user tests. Watching users struggle with bad designs is often cited as a good way to motivate people who aren't willing to buy in to user-centered design. If this is the situation designers face, this is a technique they can try to bring engineers on board. Randy Pausch, ever famous for mistreating his engineering team, requires them to sit on their hands while watching people use their interface, and demands they keep silent unless they are willing to precede their comments with an apology for designing such a horrible interface. This might be a bit extremist for most industry teams to swallow, however...
6. Both players need case studies on how to effectively work together and how not to ineffectively work together. Design and engineering (and management) can learn from examples. Even experienced industry veterans may not associate usability failures with market failures, since these two events are often separated by a long period of time, and humans are bad at associating effects that occur long after their causes. Of course, someone must put together a collection of these case studies before designers and engineers can learn from them.

Perhaps the most interesting part of the conversation, however, was the discussion of the benefits of "creative tension", or those times when design and engineering don't get along yet wind up with a solution that is better than what they would have come up with if they had. Haven brought up the case of the Harley-Davidson VROD, a case study where there were several examples of the designers putting their foots down and demanding that engineering figure out a way to implement their design, as well as examples of the engineers demanding that the designers come up with workable designs. In the end, this resulted in a better product than Harley-Davidson would have come up with had the two sides come up with a compromise solution.

This raises the question of whether it is necessarily every desirable for design and engineering to get along smoothly, or whether a certain amount of tension and controlled chaos is sometimes necessary to produce a quality solution.

[ Permanent Location ]

SSL Certificates - Unusable and (Mostly) Useless

internet, usability

November 11, 2003, 02:40 PM

Matthew Thomas has an interesting rant about the poor usability of the SSL Security Certificate system, the mechanism pretty much all web browsers, email programs, and other internet clients use to establish secure connections to servers over the inherently insecure internet. He argues that the usability is so poor that the security mechanism is rendered useless in the vast majority of cases, and demonstrates this with a description of what really happens in an Alice-and-Bob scenario.

I tend to agree, and I can vouch firsthand for the absurd level of difficulty involved in procuring and setting up a certificate from a major certificate authority, since I had to perform this very task for my last job. It took me hours, and required a high level of technical sophistication. Had our security requirements not been so high, I would likely have given up and gone back to my programming tasks, which were already behind schedule.

SSL certificates are just one example of a system that has been studied and engineered to death, but has stopped just short of a usable, human-centered solution. By failing to take that step, we've effectively destroyed all the benefits the system was supposed to provide. I've pointed out that all software quality attributes, including security, are related to usability before. Although there's been some work on designing usable security systems, this area is still sadly lacking in answers. How many compromised systems, virus debacles, and internet-smashing worms do we need to suffer through before someone decides to do something about it?

[ Permanent Location ]

Usability and Creativity

aesthetics, software development, usability

November 10, 2003, 02:55 PM

At the last SSS, MHCI student Jesse Kriss gave a talk on digital music and live performance, focusing on the technologies available for changing the way musicians perform. The important feature of all these technologies, according to Jesse, is that they separate the instrument's input from its output. With a regular instrument, like a violin, the form of the instrument is more or less dictated by the sound you want it to make. You can't make a violin with a substantially different form that is played in a substantially different way without changing the sounds the instrument is capable of making. But with digital technology, the "input" (the way the instrument looks and is played) and the "output" (the sound the instrument produces) are entirely separate. This opens up whole new possibilities for instrument design.

While listening to Jesse's explanation, it struck me how similar these concepts are to software architecture patterns like Model-View-Controller, which emphasizes the separation of input, output, and processing responsibilities into three separate components. MVC aims for this separation for exactly this reason; many types of systems require frequent changes and recombination of the input, output, and processing components, and separating them into three distinct modules helps facilitate this change. Jesse also mentioned how important it was to be able to plug together many different devices to build the kinds of instruments he envisions. He prefers Macs because they make plugging things together easy by requiring a minimum of setup hassle. It's easy, of course, because the Mac device architecture supports device discovery and auto-setup very nicely; more evidence of the impact of architecture on usability.

But it's more than just plugging together hardware. Jesse's work also involves reconfiguring that hardware to map behaviors to sounds. The software he uses to do this allows him to build processing capabilities using audio filtering components and logic controls. Essentially, this is a domain-specific programming language that reminded me of LabView's G. The flexibility and complexity of digital audio technologies is astounding.

All this got me thinking about a question I've pondered before (and in the context of digital music, no less): how do we design interfaces to support creative tasks? As I discussed in the previous post, classic user-centered design techniques focus on designing for the user's goals by distilling these goals into tasks that the interface can support. But the very nature of creative endeavors ensures that the users' tasks are difficult to define; after all, creativity by definition involves doing things that have never been done before.

Bill Fulton, the head of Microsoft Games, gave a talk here at CMU last fall and was asked this very question. His answer was that he was no expert, but that he suspected the interface should just "get out of the way" and let the creator work as directly as possible with their medium. This is good advice as far as it goes, but I suspect it's more complicated than that. After all, many programs that support creativity have some of the most complex interfaces available on the market (think Photoshop, sound editing tools, and programming IDEs). Generally speaking, however, the user populations for these tools are willing to put up with this complexity (to an extent) for the power and flexibility that comes with it. Is there even a user-centered design problem here?

I hypothesize that there is, and that we can make these interfaces better by thinking about those aspects of an interface that are essential to creativity. In general, I suspect supporting creativity is related to supporting exploration and problem solving, two aspects that are relatively well understood in interface design. Here's a few rough guidelines:

1. Make plugging things together easy, both in hardware and software. This involves building software abstractions that are robust enough to support current modules as well as future ones, and that can hide the technical complexity of configuring hardware and software interfaces to work with one another from nontechnical users. Plug-and-Play (PnP) technology has partially solved this problem already, and upcoming protocols like Jini, Rendezvous, and UPnP will hopefully take this even further. What we lack is usable equivalents for software components. Some component frameworks like J2EE try to tackle this problem, but we still have a long way to go.
2. Make problems visible and obvious. When users get creative, they tend to make mistakes like configuring the system in ways it wasn't meant to be. The system needs to provide users with sufficient feedback for them to evaluate the problem and correct it. Even better, create interfaces that make the state of the system continuously visible so users are able to diagnose potential problems before they even occur.
3. Make it easy to undo mistakes. Tog's guidelines on explorable interfaces, as well as Nielsen's heuristics and several other sources, emphasize the importance of creating an environment where users feel safe exploring by allowing them to back out of any operation they choose by accident. This is especially important in interfaces to support creativity, where almost every action the user takes is exploratory. In fact, many interfaces may benefit from a more sophisticated version of this feature than the ubiquitous "Undo" menu option. Programming IDEs have long offered integration with full-blown change management systems; perhaps its time for other applications to take a cue.
4. Allow for data sharing across applications. I brought this up in my earlier rant. Creative users may need to step outside the bounds of your application for some of their tasks, no matter how well designed your system is. Support copy & paste and import / export functionality.

This is, of course, just a first attempt. More thinking and research needs to be done to generate a list that can even resemble completeness.

It's worth noting that all of these features may impact the software architecture design of the system. See our U&SA technique for further information, especially the scenarios on Maintaining Device Independence, Supporting Undo, and Reusing Information.

[ Permanent Location ]

The Principles of Design Research

design, usability

November 08, 2003, 10:18 PM

Last Thursday, Dan posted a nice meta-summary of user research techniques.

The principles he describes are all embodied in HCI's most prominent user research method, Contextual Inquiry (CI). Basically, CIs are ethnographic observations combined with impromptu interviews, wherein the researcher watches the user work and asks questions to get at the purposes behind the behaviors, similar to the way an apprentice would work with a master craftsman. I've conducted a number of CIs since I came here, and I've found them to be incredibly valuable for data collection regardless of the overarching design methodology you plan to employ (i.e., you can benefit from doing CIs even if you aren't going through a full-blown Contextual Design process).

But back to Dan's post. I'll second the recommendation that it's best to work in pairs, especially if you are observing a task that might involve lots of interesting things happening at once, for instance, if multiple users are interacting with each other. I'll also second the prohibition on using video or audio recordings to capture the data (no matter who might tell you otherwise). Although it may seem like a good idea to have an exact recording of the inquiry on file for later reference, in practice it just unnecessarily complicates matters. If it isn't outright prohibited by the participant's organization due to security concerns, then at the very least it will make your participant feel less comfortable and thus may change their behavior, it may make it harder to recruit participants, and it may cause your participant to feel you are impinging on their hospitality, no matter how many promises you make that you won't share the data with anyone else. Chances are, you'll just pop the tape in a file cabinet after the interview and never take it out again anyway. Finally, note taking forces you to be more involved in the inquiry; you can't just sit back and zone out with the knowledge that the camera will catch everything (which is tempting because, let's face it, watching your participant spell check their document isn't exactly like seeing The Matrix Revolutions).

What Dan doesn't cover is preparing a set of focus questions before the inquiry. Running out to collect data without having a clear idea what you're looking for is likely to cause you to waste your time on the first few inquiries. Once you start to look, you'll be surprised how much data is out there in even the most routine tasks; you won't be able to capture the right data if you don't have some kind of filter in place. Deciding on that filter beforehand is critical if you want your research to effectively guide your design.

[ Permanent Location ]

OK/Cancel on Focus Groups

usability

October 31, 2003, 03:54 PM

Today's OK/Cancel comic has some nice discussion from Kevin and Tom about the benefits and problems with focus groups.

At first, I wasn't too fond of OK/Cancel. The art is nice but I rarely find the comics fall-down funny. But I'm really liking how they use the comic as a point of departure into interesting discussions about important usability issues. So, with that in mind, I'm ready to throw in my 0.02$ and recommend that you read their excellent site.

It'd be nice to see some other usability / interaction design / user-centered design oriented webcomics, though. I'm thinking something Dilbertesque that pokes fun at management and programmers (although maybe that would hurt my touchy-feely, lets-bring-all-the-disciplines-together movements :).

[ Permanent Location ]

The Open Source and Usability Problem

software development, usability

October 31, 2003, 01:56 PM

As regular readers are aware, for the past few months I've been working on the problem of bringing improved usability techniques and practices to open source software. Since the beginning of the project, my perspective on the problem has changed quite a bit, so I wanted to record where I stand now.

When I initially confronted this problem, I mostly considered it a problem of selling standard user-centered design practices to the open source community. But as I explored the problem more fully, I started trying to bring what's unique about open source (community involvement in the software development process) into user-centered design. That's opened up a whole new can of worms.

From my current perspective, here's the open problems that need to be addressed to fulfill the vision of an open source community that produces truly usable software:

1. Collecting data collectively - Both in the user research stage and the user testing stage, members of the community must go out into the world to examine user behavior. This research may feed into persona development, contextual design models, usability-related change requests, etc., which are then (hopefully) used to guide design. There are lots of issues relating to how to encourage people to contribute this data, how they will collect it with potentially no budget, how you know you can trust their data, etc.
2. Collaborative, remote design - Interface and interaction design require a high degree of collaboration and is an inherently highly visual process, unlike programming. These qualities make it difficult for effective design to occur over the text-based and often asynchronous remote communication mediums preferred by most open source projects. What changes need to be made to effectively support design, both in tools and processes? Moreover, effective design requires involving team members who have training in design and usability. What skills are required and how will these new skills be brought into the community?
3. Selling user-centered design to open-source communities - Many open source communities don't see the need for user-centered design. These communities need to be sold on the concept of UCD as well as the particular process and technology changes necessary to bring it about.
4. Selling open source to designers and usability professionals - This same sales pitch has to be made the opposite way. Very few designers and usability professionals participate in open source nowadays, for some good reasons. We must make people with these skill sets more aware of open source opportunities, convince them that its worth participating, and show them that their participation will be valued.
5. Connecting people with skills to projects that need those skills - This is a problem that exists in some open source projects even without the usability aspect. If a team finds they don't have the necessary skills to build a particular part of the system, how do they connect with people who have these skills? This calls for an open source "job market". I believe this has been tried before, but I can't remember where I saw it.
6. Incentives for usability professionals to contribute evaluations - With open source programming, there are direct incentives for participation, since you work has a direct impact on the quality of the project. With usability, benefits may be less tangible and thus more thought may be needed to provide the necessary incentives.
7. Incentives for developers to address usability issues - Developers often work on open source projects to enhance their skills with technologies and to add features they personally find useful ("scratch an itch"). But neither of these incentives would necessarily lead them to implement features that help improve learnability of the product for novices, for example. How can we convince developers to work on issues they don't personally care about?
8. Providing mechanisms for feedback to the team on usability problems - To encourage greater user involvement with the development process, open source projects may be able to leverage mechanisms for allowing users to report usability problems while using the software system. How can we provide for this interaction without burdening the users, and how can we convince developers that the data is useful and must guide design?

I've been toying with the idea of applying to a PhD program to work on this problem, since I'm realizing that tackling even one of these issues may take years. The jury is still out on that thought, however.

[ Permanent Location ]

Information on Personas and an Argument for Design

design, processes & methodologies, usability

October 25, 2003, 09:29 PM

There's a good, thorough overview of personas on Information Today. The article gives a great digest of Cooper's description of personas in Inmates, although once again I could ask for a little more guidance on how a researcher is supposed to collect the data to back up a good persona. There's talk of careful collection of qualitative data, but the process of translating those findings into personas is still pretty mysterious.

What I particularly liked, however, was this small story that illustrates the need for user-centered design:

One of the best arguments for using personas comes from some misguided design efforts at Microsoft. When the software giant geared up to redesign its well-known Microsoft Office Suite for a 1997 release, the research team soon discovered that many of the features users wanted already existed. In fact, four out of five of the features users requested for Office 97 came with Office 95. The outcome of Microsoft's design approach is just what Cooper warns against. In trying to support the diverse tasks of many conceivably different software users, Microsoft cobbled together a product that was bloated with capabilities and ended up satisfying few users.

Many people in marketing, development, and management will argue that piling features into a new product is far more important than designing usable interfaces and compelling interactions. And early on in the product's life cycle when the features are few and the interface relatively simple, this may be true, at least if your users are fairly computer-saavy. But once a few layers of those nifty new features have been slapped on to the product, you'll quickly start to find your users asking for capabilities that already exist. Bottom line is, you wasted your time with all those shiny new features, because your interface is so poorly designed that nobody even knows they're there.

[ Permanent Location ]

Is Usability Destroying Innovation?

design, processes & methodologies, usability

October 23, 2003, 11:45 PM

Dan has a post pointing to an article in the Guardian on innovation and user-centered design. The article's short so it's worth the quick read, but in brief it claims that user-centered design has become dominated by usability practitioners, that usability is a conservative approach and inhibits innovation, and thus it should be de-emphasized to encourage greater innovation in products (well, that last part is implicit but it's certainly there).

On the one hand, the core point of this article is quite accurate. It's important to produce innovative designs so that you aren't just incrementally improving the same old (bad) design concepts. And if you take a narrow view of usability (namely, that it only exists to find errors in the design of existing interfaces through analytical (heuristic evaluation, cognitive walkthrough) and empirical (usability testing) techniques) then this is the whole story.

But this is not the whole story. The user research techniques that form the core toolkit of any usability professional worth her salt include much more than testing. To be specific, we use lightweight ethnographic techniques like contextual inquiries and user interviews to study the way users work in the abstract so that appropriate design solutions can be found. This type of research can drive design, indeed it must drive design if the final product is to be usable, useful, or desirable.

The article also appears fairly ignorant of established usability practices. In particular, it states: "Too much user focus may be a barrier to innovation. Research is likely to tell us that users desire an improvement on something they already understand. Ask them if they would use a proposed innovation and they will say no - and then adopt it when they have seen its utility demonstrated." Very true, but this has been well known to usability professionals for over a decade. Studies by Nielsen (and others before him) firmly established that directly asking users what they want will yield bogus data. Any decent usability professional knows this and would not trust the answers to such questions. Design must be constrained by the users' observed needs, not the user's expressed ideas.

In summary, my main concern with this article is not that I disagree with its premise but that I'm unsure what the takeaway message is. If the takeaway is that user-centered design must begin earlier in the process through user research and interaction design, then I'm firmly in agreement. But if the takeaway is that designers must be left to design without being constrained by user needs that are well understood through sound research, then I'm worried, because then we're dropping the whole "user-centered" part of "user-centered design". The user is not like me; I don't trust my own or anyone else's "educated guesses" about what someone else will want in a product or interface design.

Finally, I'm concerned at the distinction between "usability" and "design" that is drawn in this article as well as in the industry as a whole. I don't really believe that the two are cleanly separable, any more than I believe that software architecture design is cleanly separate from programming, for example. In my not-so-humble opinion, user-centered designers must, at least to an extent, be good at both user research and interaction design, and specialists in both fields must work closely together to be effective. We of all people should know that we ought to be putting the user first in all that we do. And in this instance, the user doesn't care what fancy titles we pin on ourselves. The user cares about getting a great product that makes his life better. And we need all these skills (mixed with sound engineering too, I might add) if we hope to give it to him.

[ Permanent Location ]

A Compendium of Usability Resources

processes & methodologies, usability

October 21, 2003, 01:16 PM

Usability Net is a great reference site for all kinds of usability techniques and processes. They have a table of usability methods along with some "filters" that show which methods are appropriate in different circumstances (I'm not sure how much I believe these recommendations but they're some good starting ideas). They also have a list of guidelines for web, mobile, and other platforms, a list of all the usability methods and their detailed descriptions (U&SA isn't listed; big surprise), some businesses cases and cost justifications for doing usability, and lots more good stuff. Check it out.

I think I found this via Chad a long time ago. He also points to Usability.gov, the National Cancer Institute's web usability center. Rich was working on making these guidelines better, although I'm not sure if his work ever went live or not.

[ Permanent Location ]

The Design and Development Disconnects

design, software development, usability

October 14, 2003, 10:14 PM

I've been thinking a lot recently about the communication between software developers and usability/design people, for my usability and software architecture work, my open-source and usability work, and for an SSS topic I proposed. Here's a preliminary list I came up with of development & usability problems that might inhibit good design.

First off, one central point to make up front is that in most cases, everyone on the team; usability professionals, designers, and developers; wants the final product to be usable. Very few people set out to create a system that is useless to their target users.

That said, the system might wind up less-than-usable because of one or more of the following reasons:

1. A design may call for solving impossible problems. There are a few classes of problems in Computer Science that are genuinely unsolvable for all practical purposes; "Traveling Salesman"-style problems with exponential time complexities are examples, as well as incomputable problems like the Halting Problem. These are pretty rare in everyday system design, however, so chances are a design will be at least theoretically implementable.
2. A design may call for implementations that are possible, but infeasible. These types of designs may be theoretically implementable, but they would take far too long to implement and/or too many resources given the current state of the art. This changes with time, of course; once-upon-a-time building a WIMP-style interface would have been infeasible for most projects; with modern toolkits it is fairly trivial.
3. Implementing certain design features may necessitate sacrificing others. Some design features take longer than others. Given that the project must be completed in a fixed time frame, in order to get these features in as designed, other features may need to be sacrificed. XP's Planning Game provides a mechanism for communicating these tradeoffs based on experience-based estimates and encourages informed decision-making on the part of management and design.
4. Implementing certain design features may necessitate sacrificing other software quality attributes that are important to the project. Examples of quality attributes include performance, modifiability, and reliability. Sometimes, certain design features may require an implementation that performs slower than design alternatives, for example. Designers must be educated on how these quality attribute tradeoffs influence their designs, and Software Engineers must understand how these quality attributes influence the user experience, which is ultimately the most important consideration.
5. Architectural decisions made early on in the development lifecycle may preclude the implementation of certain design features. Generally speaking, this is a time tradeoff that didn't need to be made had these design features been considered earlier in the process. The solution, of course, is to ensure these features get considered early enough for them to be feasible. Our U&SA technique attempts to tackle this issue.
6. Design and development may have different priorities. Design tends to focus on the features that are most necessary for their users, based on their research and testing. Development tends to focus on the features that are the riskiest to implement, generally preferring to tackle those first to make the project more manageable. These two groups must come to a consensus; the developers must understand that the most technically sophisticated features aren't always the most important to users, and design must understand that development's technical priorities must be respected if the project is to get completed on time and under budget.
7. Design and development may experience simple miscommunications. Design may think they are proposing a particular interface design solution, but development might hear something different. The way it works out, of course, is that development's version actually gets implemented in the final product. The solution to this problem might lie in more rigorous specification techniques or formal sign-off procedures, but I tend to lean towards Agile Software Development's philosophy of simply having Design and Development work together more closely, ideally in the same room, all the time.

[ Permanent Location ]

The U&SA Book Chapter

announcements, software development, usability, writing & communication

October 10, 2003, 08:10 PM

I realized I haven't been posting much recently about what I've been doing for my actual job. For those of you that I haven't yet told, I'm in the process of writing a book chapter for an upcoming book on bridging the gap between usability and software architecture. You can even check out our preliminary abstract.

The chapter is about our experiences applying our U&SA technique to the NASA MERBoard project. It looks like I'll be first author for the chapter too, which is totally awesome. The current status of the actual writing is "completed first draft". If all goes well, the final, published book will be available for purchase next September.

[ Permanent Location ]

Welcome to Wiki

internet, usability, writing & communication

October 08, 2003, 04:50 PM

I set up a Wiki last night for Dana and I to use for our CSCW project (sorry, no link to the actual Wiki; it's an invitation-only affair for now). I happened across PmWiki (which is open source) when searching for options on Wiki software (through a Google text ad, no less) and installed it to give it a shot.

So far I'm pretty happy with it, both with regards to the Wiki concept and Patrick's implementation. It's even pretty usable, all things considered, which was something I was worried about. In fact, I was browsing the documentation for PmWiki and found that Patrick had actually done some user analysis for his system. It's not quite personas, but he's definitely on the right track.

I'm hoping that the wiki will help Dana and I collaboratively distill operational design principles and heuristics from all this social science reading that we are (supposed to be) doing, thus fulfilling Bob and Paul's (and our) vision for the class. We'll see how it goes, but in any event, if you're looking to set up a wiki of your own, I'd recommend checking out Patrick's offering.

[ Permanent Location ]

Newsable: Ready to Rock!

information, internet, software development, usability

October 04, 2003, 01:14 PM

It is, after all, Rocktober.

I've just unveiled Newsable 0.8 beta to a breathlessly-awaiting public (this public, in case you were curious, consists of Kerry and Jordan), thus marking the first release of my web-based news aggregator / modest experiment in merging open source and usability. The low-down on the current status follows.

First off, go ahead and create an account to take Newsable for a test drive. No, I really mean it. Go do that now.

Ok, now that you're back, let me confess that a few things still need work in the current release:

1. The interface is ugly as all hell. Jordan is kindly lending me some of his awesome graphic design skills to help fix that. Additional help would of course be greatly appreciated.
2. There are still known bugs. Namely, the harvester isn't entirely behaving itself, and the RSS autodetection algorithm needs a bit of improvement to handle the real web. I'll try to get these cleared up ASAP, but the system should be usable in the mean time.
3. The help system isn't available yet. If you get confused about something, feel free to email me. This will help me know what needs documentation / usability improvements as well.
4. Some of the interaction design decisions are tentative. The "Archived Stories" tab exemplifies this best. I don't yet have enough user data to make guided design decisions about these features. The personas still need some refinement.
5. Some more advanced features aren't yet available. OPML import / export support is one of these. I hope to have these in place before 1.0

As you can see, there are many opportunities to get involved with improving Newsable even if you aren't interested in writing Perl code (unlike the majority of open source projects). Please email me if you have any ideas for improvements in the aforementioned areas. I may try to set up a mailing list soon to create a central place for discussion.

Happy reading!

[ Permanent Location ]

An Online Community for U&SA

internet, society & sociology, software development, usability

September 29, 2003, 09:02 PM

As many of you already know, I am currently employed here at Carnegie Mellon as a research assistant for the U&SA project. By day, I investigate the relationship between the usefulness, usability, and desirability of software systems (the human-centered view) and the architectural decisions that dominate the construction of these systems and ultimately determine what is easy to change and what is hard to change (the machine-centered view). Specifically, I work to improve our U&SA technique which involves general usability scenarios and considerations that impact software architecture design decisions and design recommendations for developing architectures that better implement these scenarios. (By night, I'm either an overworked graduate student or a hard-boozin' party animal, depending on the phase of the moon and the moods of my professors...)

When I leave here, one of the projects I hope to have time to start involves developing an online community for improving and refining the U&SA technique. Essentially, I believe we've got something good here but that it could benefit from more industry feedback and the infusion of best-practice wisdom. Hopefully CSCW will better prepare me for this task by the end of the semester, but in the meantime, here's a few thoughts on what such a community might look like:

• The scenarios could go in a wiki so that several people could propose and refine examples of architecturally-sensitive usability issues. This way, every time someone encountered an important usability concern that turned out to be difficult to add later and that wasn't covered by our material, they could add a scenario for it or modify an existing scenario to include it.
• Discussion boards on applying the scenarios to real-world projects and technologies. These boards may focus on discussing technology-specific patterns for the scenarios, such as implementing Cancel in a J2EE system, or on existing toolkit and framework support for the scenarios.
• Moderated essays on distilling out considerations and patterns from the scenarios and technology discussions. These essays would necessarily entail a lot of meta-analysis and require a higher degree of accuracy, so they should require more community consensus for publication. I'm thinking something like a Kuro5hin-style voting system for determining whether they're published or dumped.

So those are my initial, very rough ideas. If you're interested in helping me flesh them out some time, please do let me know.

[ Permanent Location ]

End-User Software Engineering

software development, usability

September 17, 2003, 12:57 AM

Last Wednesday, a woman named Margaret Burnett from Oregon State University gave a talk at the HCII Seminar Series which I attended on the topic of "End-User Software Engineering". Here's the problem: many applications require end users (read: people who never program and don't want to) to write high-level, fourth generation programs to access certain advanced features of the software. Some examples include writing small Javascripts for your web page, setting up rules in your email client for automated organization of all the email you get, and (this was Margaret's "test" problem) creating formulas in spreadsheets to perform complex calculations. Margaret wanted to help these types of users produce better programs by making some software engineering techniques more accessible.

Margaret unveiled the rather disturbing statistic that, apparently, around 90% of production spreadsheets (that is, spreadsheets that are used for real work) contain errors in their formulas (and thus may be producing incorrect calculations). Moreover, these users are overconfident in their results; they believe their formulas are correct when they are not (this phenomenon is common among professional programmers, as well...). Her work attempted to attack both these problems by helping users write better code while also giving them the appropriate level of confidence in the result (since no code is ever perfect).

For her purposes, Margaret defined a "test" as "a decision if some output is right for a given input", so her interface had to help users make this decision, i.e. to execute tests. There's also the problem of test adequacy, that is, how do you know when you've done enough testing? (No amount of testing can guarantee correctness, it can only increase your confidence.) She then discussed the "WYSIWYT" or "What You See Is What You Test" paradigm, an attempt to make the fairly invisible process of testing more visible to the user. Her hypothesis was that making testing more visible and providing better feedback would help tackle both the execution and adequacy problems.

Since she was working with spreadsheets, Margaret's research group created an interface that allowed users to provide sample input values and automatically determined how fully the provided input had tested the formulas (this is basically white-box testing), then provided feedback on whether the formula was "not tested", "partially tested", or "fully tested" by changing the color of the cell. They also implemented a "Help-Me-Test" feature that provided suggested test values (to varying degrees of accuracy). Through empirical studies, they learned that this WYSIWYT paradigm:

• Resulted in higher test coverage for user's programs.
• Helped user's gain a functional understanding of their programs.
• Gave users a more appropriate level of confidence in their programs.
• Made users faster at debugging.
• Reduced (but did not eliminate) overconfidence.

To help further improve the overconfidence problem, Margaret's group also implemented an assertion mechanism into their spreadsheet testbed, which allowed users to define appropriate bounds for cell values. Assertion failures caused the values to get a big red circle around them. Empirical user tests revealed that users seemed to interpret these appropriately; Margaret quoted one user as remarking "Oh, there must be something wrong with the formula!".

This seems like some interesting and important research, and I'd love to see how it might apply beyond spreadsheets. Andy remarked that Margaret was collaborating with Microsoft to potentially get some of this stuff implemented. Which would bring us one step closer to bringing the power of programming (and thus the "true" power of the computer) to ordinary, everyday people.

[ Permanent Location ]

Operating System Learnability and the Digital Divide

society & sociology, usability

September 07, 2003, 04:23 PM

Dan has a post about how Macs aren't any more intuitive to use than PCs, at least in his opinion.

I'd have to say, after a year of using a Mac and several years of using PCs, that I can't see where all the hoopla over Mac OS being more usable than Windows came from. Both have their strengths and weaknesses (Mac OSs tends to be better at Just Working™, whereas Windows tends to be better at giving you meaningful feedback when it doesn't; Mac OS has slighly more sensible metaphors with its emphasis on drag-and-drop and files-as-objects, whereas Windows has many more features to help out expert performance), but overall, I'd agree with Dan's assessment that both OSes are probably still pretty tough for the novice user to grok. Unless someone can show me some hard data from summative learnability evaluations comparing equivalent tasks in both Operating Systems, I'm going to remain unconvinced.

Dan said something else interesting, though. I quote:

Yes, we are raising a group of hyperwired kids (my daughter can click and drag with a mouse to play games on the computer at age 2), but there's probably a huge underserved market out there who could use some sort of operating system that was powerful enough to run what they need and yet be hassle free: the elderly, the very young, the uneducated. These people, however, are also often poor. Which is probably why this hasn't happened yet.

I've been wondering, recently, if learnability will eventually become "obsolete". As Dan points out, the younger generations are becoming much more familiar with technology at a much earlier age. Does this mean that, in the future, systems won't have to be as learnable, since they can count on a user population that is generally computer-saavy? (Note that usability will still be important, since it is so much more than ease-of-use.) And what about the poor and uneducated? Will they just slip further behind as the rich are elevated to digerati and high-tech products become more complex and esoteric?

Things to think about.

[ Permanent Location ]

A More User-Centered HTTP Log Analysis Tool?

internet, usability

September 06, 2003, 12:39 PM

I was pouring over my logs yesterday and I realized I'm getting more and more frustrated with the inadequacies of Webalizer, especially with regard to performing log analyses for the purposes of enhancing usability. Webalizer takes a very system-centric approach of basically compiling and graphing HTTP request data, rather than a user-centric approach of trying to generate visualizations of the log data that answer the questions of its users. Although there are a few promising features (tracking "visits" and extracting search query terms from referer URLs), in general the tool leaves much to be desired.

So I'm wondering if there are any better tools out there, ideally open-source but I'd be willing to purchase a proprietary product if the price is reasonable. Here's some of the kinds of features I'm looking for:

• Better capabilities for associating visitors with the URLs they viewed. It's less helpful for me to see the top 30 URLs by hits than it is to see what sites visited what URLs and how many times they visited them.
• Some tracking of the paths visitors take through the site. This can help give a sense of what content people are most interested in, what content is hard to find, etc.
• Association of search terms with the pages they linked the visitor to.
• Better graphing of hits, sites, and visits, such as the ability to view daily trends over multiple months.
• Just working better in general; for instance, allowing me to tell it to completely ignore requests from htdig in performing the analyses. I've supposedly told Webalizer to IgnoreSites from the Labs itself, but this only works for some of the calculations.

I realize that much of the data users really want to see (exactly who is reading their site, how many real people are reading, etc.) cannot be obtained from HTTP request headers. But all the features I've mentioned above are perfectly implementable. The designers of these tools just need to take a more user-centered perspective.

So, anyone know of any better alternatives? 'Cuz I'd sure like to.

[ Permanent Location ]

Dancing Bears

design, usability

August 29, 2003, 08:20 PM

Against my better judgement, I was flipping through the first half of "Inmates" a couple days ago. I came across an interesting concept Cooper describes as a "Dancing Bear":

In essence, a dancing bear is a product that is interesting and useful not because it works particularly well, but because it works at all. Cooper describes his car's keyless entry system as an example, which is horrendously difficult to use but still addictively useful because of the advantages of remote entry to his vehicle. In his words:

In spite of its weak and clumsy design, it is still a wonderful thing. It's like the fellow who leads a huge bear on a chain into the town square and, for a small donation, will make the bear dance. The townspeople gather to see the wonderous sight as the massive, lumbering beast shambles and shuffles from paw to paw. The bear is really a terrible dancer, and the wonder isn't that the bear dances well but that the bear dances at all.

...

The difficulty of devising a better interaction isn't what makes the problem so intractable. Instead, it is our almost universal willingness to accept bad interaction as an unavoidable cost.

There are many dancing bears on the technology market today, I'm afraid. Sometimes, after seeing a series of poorly designed products beat out well-designed alternatives just because they were "new and cool", it's easy for us user-centered designers to dispair; sometimes it seems like people just don't want usability! I think it can be encouraging to consider these products dancing bears; the problem is that right now information technology-enabled products are so new that many will buy them even if they are unusable simply because they don't know any better. As these products sink into society, however, the buying public will become more informed consumers. Then, our skills will really have a chance to prove their value.

[ Permanent Location ]

Damaged Merchandise?

usability

August 27, 2003, 11:24 AM

I skimmed a paper by Wayne Gray (recommended by Bonnie) last night entitled "Damaged Merchandise? A Review of Experiments that Compare Usability Evaluation Methods". This is apparently considered one of the more infamous papers in academic HCI, since it single-handedly halted most of the research on HCI methods. In it, Gray and Salzman cover a number of serious flaws in common HCI methods experiments, then go on to show how these flaws taint the conclusions of five highly influential papers, including ones by Nielsen, Atwood, Jeffries, and others.

Gray and Salzman argue that the experiments in question violate many well-known best practices of experimental design, citing a book on designing social science and psychology experiments from 1979 to back up their claims. Their arguments are convincing, and its disconcerting to think that a fair amount of evidence for some of our most critical techniques in HCI is flawed. Some of the mistakes are even pretty basic ones that I knew about (without having any real background in psychology, social science, or experiment design) like ignoring the effects of group psychology (e.g., counting a focus group's results as testing eight users instead of one) or asserting propositions in the conclusions that are either unsupported or even directly refuted by the research.

I think the lesson to take away from this paper if you aren't an academic is that you need to realize when someone is speaking with the backing of scientific validity and when they are not. For example, certain usability gurus are wont to frequently provide tons of specific advice to practitioners, and, if you respect the opinions of said gurus, you may be wont to receive it. But you should realize that this advice is offered without scientific backing, frequently even if the guru claims it comes from "numerous user studies". The advice may be the opinion of a well-respected, experienced individual, but it is still an opinion and must be considered as such.

And if you are an academic, the lesson to take away is to get your shit together and learn some research methods.

[ Permanent Location ]

Extreme Usability

software development, usability

August 22, 2003, 05:06 PM

Early last spring, I gave a talk on eXtreme Programming (XP) at the MHCI Student Seminar Series. Giving a talk on a programming methodology to a bunch of usability and design folks fit in with my interests in unifying software development and usability and hopefully helped them understand how to better work with programmers. Given the audience, after I gave the overview of XP itself I threw out some ideas on how to integrate our usability techniques into an XP environment.

XP, at least as described by Kent Beck, is unfortunately fairly naive about usability. Beck does not put down usability, in fact he emphasizes that the software must provide value to its users (or in his words, "the business") and thus representatives of the users must be given control over defining that value. But XP makes a few assumptions that we in the usability community know to be false. For instance, it assumes the customer representative or domain expert will be able to define the users' needs and come up with an effective interface design. However, in HCI it's well known that users are not designers, that they know their work but not necessarily how to improve their work; they may not be able to come up with a more usable interface than programmers. In fact XP doesn't say much about interface design at all; the assumption appears to be that an effective design will come out of the user stories.

On the other hand, XP is in many respects very compatible with usability techniques. XP emphasizes the importance of testing and iteration, which are also important concepts in classic usability engineering. It also emphasizes the importance of trying out many ideas and making improvements where they are needed, which are important concepts in interaction design. So at least in the abstract, it seems that usability would fit well into XP. But how can we make this concrete?

During the talk, we had a discussion about exactly how usability techniques might fit into the process. This list contains the thoughts that came out of that discussion, as well as my own ideas and those I've found through my readings.

• Field workers and interaction designers play the part of "Business" in the Planning Game.
• If this isn't possible, Participatory Design techniques like PICTIVE could help customers develop more effective designs.
• Put usability people on the team, co-located with the developers while they work.
• Use the RITE method for functional tests.
• Have the Tracker record usability metrics and test results in addition to stories completed, time until release, etc.
• Have the Coach look at the UI "big picture" as well as the implementation big picture.
• Develop a Metaphor that is part of both the UI design and the code structure.

The most obvious way to partition the usability / development responsibility in XP is to have field workers and interaction designers (using a technique like Contextual Design or Goal-Directed Design) play the part of "Business" in the Planning Game instead of customer representatives or marketing. This means the designers are responsible for defining the "stories" (which are reasonably similar to scenarios; Scenario-Based Design is a component of Goal-Directed Design as well as a useful technique in its own right) as well as responsible for prioritizing those stories with respect to how essential they are to creating a usable system. The developers are responsible for estimating cost and time of implementing each story. Essentially, I'd claim the Planning Game already provides an adequate framework in which usability and development can work; we just need to change who's involved in the process.

This isn't a brand new idea; Chris Neuwirth, a professor in the English department here at CMU, worked with researchers at Xerox PARC to implement such a process. They even found that "their first model was unsuccessful even 'with a user-centred approach, largely because fieldwork inspired engineering repeatedly led to innovations that delivered no significant end-user advantages over what was already supported' by available systems.... But, for their second exploration, the team 'adopted XP as an approach to keep design closely tied to fieldwork findings and usage experience, with fieldworkers playing the part of customer representatives'. This approach, they conclude, was far more successful, giving a usable prototype in a fraction of the time of the first." In this case, XP improved the usability process just as much as the usability process improved XP; eschewing the large, up-front design time allowed the team to test real, working systems rather than simple mockups or prototypes, improving the quality of the feedback they received and the resulting designs they produced. Kent Beck and Alan Cooper publically argued over this very point; I think Beck came out on top, but you can see if you agree.

However, it may not always be feasible to hire a team of usability professionals and interaction designers to "play customer". In this case, Participatory Design techniques such as PICTIVE may prove useful in helping the customers, facilitated by a trained designer, design better UIs on their own. Participatory Design is apparently common practice in Europe, but sadly hasn't caught on as much in the States.

The next most important change involves getting the usability people into the same room as the programmers and ensuring they work together closely throughout the process. XP and other agile methodologies rely on close communication between all team participants; if the usability people are off in their own room, separate from the programmers, then their designs won't get implemented the way they intended. To ensure these two cultures have things to talk about and interact over, they may need effective boundry objects to help them communicate such as our very own U&SA technique. Ideally, the usability people should participate in pair programming when user interface issues are getting implemented in code. At minimum, one usability person should play the part of the On-Site Customer.

When it comes time to run functional tests, a quick, iterative testing methodology such as RITE should be employed by the usability team to get fast feedback on the usability of the system. RITE is a good fit for XP because it sacrifices slow, scientific precision for quick feedback on design glitches and ensures those glitches are fixed by more quick, iterative tests. The results of these tests can get fed back into design changes to the interface, which will get converted to stories that are placed into the next (or current) iteration as their criticality warrents (or may just get implemented on the spot; because everyone is communicating effectively the easy-to-change things can get fixed without having to go through the formal process).

To ensure that the usability goals of the system are getting met (whaddya mean you haven't defined usability goals?! That should have happened in the Planning Game!) the Tracker should chart the results of the usability tests and map those against the goals of the project. These results should be posted along with the normal functional test scores, completed stories, etc. that help the team realize when the schedule is starting to slip.

At the same time, the XP Coach should be keeping an eye on the "big picture" of the user interface design as well as the big picture of the internal software design. This requires the coach to be both a skilled interaction designer and software architect, however, so it may prove necessary to hire another "UI Coach" to perform this role. Someone has to be keeping an eye on the overall information flow and interaction architecture of the system's design, however, to prevent a series of sure-made-sense-at-the-time fixes from ruining the overall user experience.

Finally, XP calls for the creation of a "Metaphor", or a high-level description and terminology for the system's internals. Usability people also talk about metaphors, but these refer to metaphors in the interface design that draw on users' experience with real-world objects. On the face of it, I figured this was a situation where the same word has two different meanings in different cultures, but now I'm not so sure. If the metaphor for the system design and the metaphor for the interface design are the same, then this should greatly improve the communication between the customers, the interface designers, and the developers, since they'd all be speaking the same language. Supposedly, this was touted as one of the benefits of object-oriented programming: users and developers could both understand the software design if it is written in terms of real-world objects instead of functions and data structures. But so far this hasn't panned out; perhaps using the XP Metaphor in this fashion will finally make this "feature" of the OOP a reality.

And there you are: a usability-friendly XP, or "eXtreme Usability", if you will. Now all we have to do is try it out on a real-life project. Any volunteers?

[ Permanent Location ]

Usability Impacts Architecture (But Don't Take My Word For It...)

software development, usability

August 19, 2003, 04:34 PM

As many of you already know, one of the areas I'm interested in is understanding and improving the connection between software development and usability in general and software architecture and interface design in particular (in fact, I'm currently gainfully employed to work on this problem). Most of the time, when I describe what I do to my fellow HCI people (especially HCI people with some "in the trenches" experience), they immediately see the benefit of our work; often they have a "We can't change THAT!" story of their own to relate to me. Sometimes, however, I get blank stares or a casual shrug; some usability professionals assume that architecture design is the software developers' job and doesn't concern them or that the relationship between architecture and usability is a solved problem. For those people, as well as for my own records, I'm starting a list of other HCI/usability and software people who agree that architectural impacts on usability are an important problem. Feel free to post a comment if you know of anyone who made this important point that I left out.

So don't just take my word for it...

• Scott Burken, a long-time designer and usability evangelist at Microsoft, has an essay on the "long list of reasons why ease of use doesn't happen" on his website. One of those reasons is "the unseen impact on ease of use on system/code architecture". He states:

  The biggest danger with the code architecture view of a project is that if the UI is designed last, many assumptions can be built into the design of the architecture that unknowingly effect the UI design. For example, a database architecture for a website might require that information be typed in by the customer in a certain format (which turns out to be confusing to customers), or that certain kinds of widgets have to be displayed together (which turns out to be confusing to customers). By the time the team gets around to working on the UI layer, so many constraints may be placed onto the ui designer, that it's impossible to actually create a system that's easy for customers to use (so surprise: the system turns out to be confusing to customers).

• Ben Shneiderman, the well-known usability researcher, recognizes the architectural implications of usability in his influential book "Designing the User Interface" by recommending that programmers get involved early in the process for identifying architectural issues. I quote: "The prototypes are usually developed by a programmer who is part of the software engineering team. One of this programmer's responsibilities is to identify interface issues that have implications for the technical architecture of the product." (p. 106)
• The Software Engineering Institute (SEI) published a book on software architecture design entitled "Software Architecture In Practice" (SAP). The first edition of SAP equated architectural usability issues with modifiability issues, and deemed it a problem solved by separation patterns like Model-View-Controller or Presentation-Abstraction-Control. The first addition asserted: "Making a system's user interface clear and easy to use is primarily a matter of getting the details of a user's interaction correct...but these details are not architectural." The second edition (recently released) admitted this claim was an error and corrected it: "Many usability issues are architectural. In fact, the usability features that are the most difficult to achieve (and, in particular, the features that are most difficult to add on after the system has been built) turn out to be precisely those that are architectural."
• The Usability and Software Architecture (U&SA) Project, a joint venture between Carnegie Mellon's HCII and SEI of which I am a part, has made this argument since its inception. We've published a technical report on the topic; more current information is in later publications. We argue: "Separation of the user interface has been quite effective, and is commonly used in practice, but it has problems. First, there are many aspects of usability that require architectural support other than separation, and, second, the later changes are made to the system, the more expensive they are to achieve."
• The STATUS Project, funded by the European Union, is U&SA's counterpart overseas. In the overview of their research, they assert "The traditional software quality attributes are reliability, efficiency, maintainability,?. However, there is no doubt (and many authors defend this stand) that usability is another software quality attribute. However, the relationships between software usability and software architecture have not yet been studied."
• Extreme Programming (XP) does not explicitly argue for usability impacts on software architecture, but it does argue for a software design that is created and constantly refactored to fit customer needs. XP's concept of a "metaphor" instead of a formal architecture blends usability and software design: "We need to emphasize the goal of architecture, which is to give everyone a coherent story within which to work, a story that can easily be shared by the business and technical folks." (Extreme Programming Explained, p. 57) If you replace "business" with "users", which is usually what Beck means, you have support for the argument.

That's what I have so far. Again, if you know of any other good pointers on this topic, please do comment below or send me email.

[ Permanent Location ]

Know Your Designer: Scott Berkun

people, usability

August 14, 2003, 10:58 PM

Scott Berkun is an amazing man with some amazing ideas whose done some amazing stuff (he worked on the IE and Windows interfaces). I met him at CHI 2003 last April and he seems like a cool guy to boot. I'd strongly recommend perusing his essays if you are at all interested in practical usability and interface design issues. So far, everything I've read there is golden. If I can one day get my own writings section up to half that level of quality, then my work here will be done.

[ Permanent Location ]

Bringing Usability Professionals to Open Source Software

software development, usability

August 12, 2003, 09:38 AM

I happened across an old article on Open Source and Usability over on Lyle Kantrovich's weblog the other day. Lyle read the paper on the problems facing open source software usability that I mentioned in my first post on the topic and clearly articulates the interaction designer / usability professional's perspective on why they may not want to get involved with a "classic" open source software project.

I'm hoping, of course, that using personas can help bring a much needed focus on a clearly defined user to open source software projects, which will solve the major stumbling block as far as open source usability is concerned. In another post, Lyle lends credence to this hunch by asserting that Linux needs focus (warning to OS developers: vitriolicism levels are at medium-high); you can't expect a tool to be useful and usable for a target user population if you don't even understand who that target population is. If Lyle said "personas", he would have practically beat me to all my ideas :).

If we accept that having usability professionals of all stripes (whatever they all might want to call themselves) participate in open source software development projects can only be a good thing for improving the usefulness and usability of said projects, then open source software has to start adopting some of their techniques, respecting some of their ideas, and using some of their terms to meet them halfway. Otherwise, Lyle's assessment of the current situation will remain the norm for years to come.

[ Permanent Location ]

The Invisible Designer

design, society & sociology, usability

August 06, 2003, 08:24 PM

Since I first became interested in interface design, one of my favorite sayings has been "The most irritating thing about being an aspiring usability professional is that there is so much need for us out there in the world and so little demand!" Today I found out that Alan Cooper elaborated on this thought better than I have been:

Sometimes being an interaction designer can be so frustrating! If, as a designer, you do something really, fundamentally, blockbuster correct, everybody looks at it and says "Of course! What other way would there be?" This is true even if the client has been staring, empty-handed and idea-free, at the problem for months or even years without a clue about solving the problem. It's also true even if our solution generates millions of dollars for the company. Most really breakthrough conceptual advances are opaque in foresight and transparent in hindsight. It is incredibly hard to see breakthroughs in design. You can be trained and prepared, spend hours studying the problem, and still not see the answer. Then someone else comes along and points out a key insight, and the vision clicks into place with the natural obviousness of the wheel. If you shout the solution from the rooftops, others will say "Of course the wheel is round, what other shape could it possibly be?" This makes it frustratingly hard to show off good design work.

Cooper attributes the problem to this inherent property of design: the "right" design seems obvious once you've arrived at it, but boy is it hell to get there. And I think this is a large part of the problem. Our brains are funny things; neural impulse patterns have to click just right with other neural impulse patterns to produce those breakthrough design ideas, and sometimes it requires intense immersion in data or research, high-energy brainstorming sessions, mapping mental associations to physical objects using a process like affinity diagramming, or a host of other mental exercises to arrive at this "click". But once you have the idea and can describe it to others, they don't have to go through all this strenuous mental activity and thus have the "Duh; it took you that long to come up with this?!" reaction.

But I think this isn't the only barrier. Another big problem usability professionals seem to face is the "everyone's an expert" symptom when it comes to interface design, so it's hard for the designer to sell his skills when non-designers are convinced you don't need special skills to create great results. Interface design is one of those fields that is easy to do badly but hard to do well, and unlike programming, it isn't immediately obvious when a design is bad to those who designed it. This is especially if you don't run user tests, and since this problem crops up most in organizations that are fairly immature with respect to usability, this is likely the case.

But even if the organization is fairly mature, usability still often falls by the wayside because, as Cooper argues, its benefits are often invisible. A well-designed interface may not have immediate, measurable impacts on product sales, may not be demanded by clients (well, they may want the software to be "easy to use", but most don't understand what this entails), and thus may not hit the company in its pocket book. And even if it is impacting sales, this effect is likely to occur only after some delay, as users get frustrated with the product and switch to competitor's offerings. And as we know from systems thinking, we humans are bad at connecting effects to causes when there is a significant delay between them.

So, what's an enterprising young designer to do if he wants to help tackle this problem?

[ Permanent Location ]

Power and Language in Interaction Design

language, usability

August 04, 2003, 10:59 PM

I found an Ask Tog column today via Dan's weblog proposing "Interaction Architect" as a new name... er... "brand" for interface designers as well as the development of a corresponding professional association to elevate this discipline to a more respected status in the computing industry. I always thought interaction designers / user experience professionals / whatever they want to be called fit rather nicely with the user testing people under the umbrella of the Usability Professionals' Association, but no matter. Tog's proposal is interesting in its own right, but it lead me down an entirely separate avenue of thought.

In the past, I've argued that terms like "architect" speak towards similar patterns that appear in many disciplines. Tog's post, however, provides some indirect contradictory evidence for this assertion; he wants to use the term "architect" primarily because it's a "power word", i.e. its a term that is associated with a certain amount of respect in the political climate of software development teams. I could see someone arguing that this implies there is no such common meaning; terms like "architect" just get co-opted by certain groups of people to further their own political purposes. (To be clear, I'm not claiming Tog is taking this position.)

There is some merit to this argument; language gets used for many purposes by us naked apes. One of those purposes is, in fact, political posturing; politicians have to master this method of using language if they want to be effective at their jobs (just read some Machiavelli to see what I mean). But this argument doesn't tell the whole story. Tog uses the words "architect" and "designer" not only because they further his cause, but because they make sense in the context. Tog explains why he sees "Interaction Architects" as leading the development of software systems in a similar fashion to the way building architects lead the construction of houses. If there weren't underlying commonalities in the meanings of the terms, no one would be willing to accept "interaction architect" as a moniker. After all, Tog didn't choose "Interaction Executive" as his name of choice, even though "executive" is certainly a powerful title. "Interaction Executive" is laughable in this context precisely because it makes no sense.

Words are powerful to humans, and get put to a wide variety of ends. But ultimately they must follow the rules and norms of human languages; they must be grounded in common concepts and must make sense to those who hear them.

[ Permanent Location ]

A Foray into the Asylum

software development, usability

August 01, 2003, 12:41 AM

I've finally gotten around to reading Alan Cooper's "The Inmates are Running the Asylum", something I felt I should do since I'm pushing personas and Goal-Directed Design as a key component of open-source usability (in my defense, I have read several articles about personas while researching this project, just never one of Cooper's books). I've skipped the first half of the book since I hear its just a long rant about how programmers don't know how to design software systems for users. I picked up where he starts getting into the real meat of his techniques.

First off, I'm in the process of revising Newsable's personas to make them a little more specific, a little more human, and to give them explicit goals. Mathilde made most of these suggestions when she read the personas; I waffled on changing them for a variety of reasons but now I think she was right and I was wrong, as is often the case ("Correct as usual, your majesty"). Those changes should be up in a day or two.

I liked Cooper's arguments in favor of focusing on a single, primary persona for your design, on the need to get away from the "elastic user" whose personality, goals, and desires change from situation to situation depending on the current needs of the developers and managers. Cooper's description bolsters my hope that this technique will help bring more coherence to usability discussions about open source software. Ideally, it will end the "creeping features compromise" that plagues so many projects, where new features are added because someone felt like coding them, not because most of the users want them (which also leads to confusingly crowded preferences dialog boxes). On the other hand, I'm concerned about how this narrow focus will play with potential project contributors who don't fit the persona. Will they be turned off by the lack of attention to their needs, thus losing valuable help? Would it be sufficient to capture them as secondary personas? These are some of the questions I hope to discover the answers to.

Just to throw out a couple more thoughts while I'm on the topic: Cooper argues vehemently for "polite" software; he quotes Clifford Nass's work which demonstrates that people interact with computers in a very similar fashion to the way they interact with humans, even people who "know better", i.e., Stanford computer science undergraduates (as an aside, Cliff Nass gave a talk here at CMU last fall, which was excellent and drove home this very same point in the area of speech interfaces). So he proceeds to list a series of characteristics which are intended to make software more polite. Which is all well and good, but on the surface many of these heuristics are pretty obvious. The hard part comes in trading them off against one another as well as other usability concerns, something Cooper hasn't given much guidance on yet. For instance, Cooper asserts that "polite software is taciturn about its personal problems"; he complains "I don't need to hear the modem whistling or see information about the computer's data transfer rates..." Yet I often find this information essential for troubleshooting when things go wrong (if I don't hear the modem whistle, maybe the wire isn't plugged in?) Granted, a perfect interface would deduce the problem itself and tell me how to fix it, thus bypassing the need for such arcane techniques, but given that the interface does not do this (probably because it would be too time-consuming to implement), I'd like to keep my modem whistle, thank-you-very-much.

Which leads to my second point; Cooper is either quite naive about many implementation issues or he is purposefully ignoring them. Many of the problems he decries (such as how all his personal preferences aren't shared among all his computer's software applications) are the result of interoperability problems which spring from social and market failures; different companies have little incentive to make their products share such information. And some of what he recommends is just outright tough to implement; polite software may anticipate our needs, but predicting the future is often hard and may require sophisticated predictive models of human behavior. Bonnie considers this an interesting research question, but even she will probably admit we aren't there yet in terms of mass commercialization.

Ordinarily I wouldn't harp on these points at such length, but I strongly believe they are important for interaction designers and usability professionals to understand. If all of them adopt Cooper's attitude (basically, "damn the implementation; usability is the only consideration!"), then I fear for the fate of interdisciplinary collaboration, which is so essential for producing usable, "sane" software.

[ Permanent Location ]

Expressive Type, Visual Structure, and Human-Centered Design

aesthetics, design, usability

July 21, 2003, 07:59 PM

This past week in CDF we studied Expressive Typography with Dan Boyarski, head of the School of Design and typographer extraordinaire. Our running assignment was to arrange a quote on a page in a way that expresses its meaning. For the first couple iterations we were constrained to just positioning the text, for the rest we could also change the font size and stroke weight.

First off, I was amazed at how much emotion and meaning you can communicate using only type (no color, no images, and no shapes other than letter-forms). I'd seen typographic presentations before, both static and kinetic (kinetic typography, or videos of moving type, is one of Dan's research interests), but viewing someone else's piece doesn't quite drive home the point as much as actually sitting down and doing one yourself. The sheer number of variables and choices was daunting, to say the least, when the position, size, and weight of every word (or even every letter) necessarily conveys meaning across even small visual differences. And this is just type. When I imagine trying to put together a piece using the full range of colors, shapes, drawings, pictures, time (if the piece is a video), etc. my head starts to swim. Yet from the design perspective, interaction design (what we do to design software systems and other interactive experiences) is all these and more; its like kinetic design minus the strict sense of control the linear flow of a video gives its creator, like a movie where the viewer rather than the director gets to choose the next set of frames to play every few seconds from potentially hundreds of options.

This "design perspective" is a fundamentally different way of thinking about the world. The visual designer learns to give their full attention to how something looks, to its visual representation. Programmers, philosophers, writers, and (usually) interface designers generally think quite differently; to them the underlying ideas are more important than the visual presentation of those ideas. Yet perhaps the visual designers have something here; after all, we know that the appearance of a web site is the most influential factor in determining whether users trust the content. And that's not all it affects.

We humans are visual creatures; sight is our primary sense and thus it is integral to the way we think. In many cases, people can suck in and process a lot more information when they can see it all laid out before them than they can if they have to view bits and pieces in sequence or, even worse, read a linguistic description of it. This jives with some of the studies I've done for Newsable; I found that all of my users were quite adept at visually scanning for interesting content, but they were much worse at describing what they were looking for. Likewise, Dan preached about the need for large walls in project rooms so designers can post their work up for easy comparisons. Visual comparisons of design artifacts are much easier when you can see all the artifacts you're comparing at once.

Another interesting lesson, both for this expressive type week and the more mundane hierarchy week, was how much the visual presentation dictates the structure people ascribe to the content. This is pretty clear when you compare my final version of the first assignment with the raw text; this shouldn't be new to anyone. But it was interesting to see that visual presentation isn't just important for sound information design; it even proved critical for the "fuzzier" quotes. Take a look at my classmate Jennifer's piece and you'll see what I mean; the quiet meditation of her quote comes out clearly in the small size of the text, its position in the bottom corner of the vast whitespace of the page, the natural grouping of the words through line breaks and indentation, etc. (The papyrus piece is © 2003 Jennifer Anderson. All rights reserved. Used with permission.)

This approach to visual design as a communication medium that enhances (or even completely changes) the meaning of the content is very much in line with the philosophies of information design and technical writing that I'm familiar with. So far, the professors we've had have talked a lot about communication and what message our designs are sending to the viewers. I've been impressed by how much attention these expert designers pay to the viewer (dare I say the user?) of the artifacts they produce (or are asking us to produce), because it means their basic values are very much in line with ours over in HCI. Dan even outright said "We believe in human-centered design". Insofar as this holds in practice for both camps, we'll always have a common base we can stand on to collaborate. Most designers don't do user tests, and most usability practitioners don't do critiques, but these are just differences in approach, and both approaches are valuable in their own ways. There seem to be many opportunities for synergy between these two cultures and bringing them together, as we're ostensibly doing here at CMU, is a tremendously good thing.

[ Permanent Location ]

Usable Architectures and Designing for Change

software development, usability

July 16, 2003, 08:32 PM

Recently, I've been thinking a lot about architectures for software systems as well as systems in general, specifically about those aspects of system design that make future changes easy or hard. A while back I wrote up an outline for a post on this subject, so I wanted to flesh it out to frame the discussion.

First, a brief discussion of what I mean by architecture. For software systems, the architecture is the part of the system's internal design that determines what aspects of the system are easy and quick to change and what aspects are difficult and time-consuming (and therefore costly and unlikely to happen). They are commonly compared to the support walls in a building. If you want to knock out a wall to make a room larger, for instance, this may be relatively easy if the wall in question isn't supporting the roof. If it is, however, you can't get rid of it without bringing down a good portion of the house. Software also has support walls in the form of the major components, patterns, and idioms that crop up all over the code. Changing these components generally requires rewriting (and retesting, reintegrating, redeploying, etc) large portions of the system. Since most industry software development teams are under a tight schedule, this generally means that modifications to the system (new features, more usable interfaces, faster performing algorithms, etc) that require making these changes don't get implemented in most circumstances.

Ultimately, the goal of any software system is to be usable (in the broad, "useful to humans" sense). Since the architecture is hard to change later, we'd ideally like it to instantiate the high-level usage of the system and related system concerns. In other words, it should support the users' main tasks and provide a framework into which newly discovered subtasks can fit in. For example, for Newsable, the architecture should support the main gathering, sorting and cataloging, and skimming and reading tasks that the user studies identified as the most critical user tasks. This would be a usable architecture.

The problem with all this is the use of any system, even at its highest levels, evolves over time. Users' needs change, business processes change, the environment in which the system must operate changes, even the introduction of the system changes the work processes of its users in ways that may in turn suggest changes to the system itself. This means that the architectures of these systems must change as well. Even the SEI, the big proponents of "get it right the first time" software design, recognize this fact. In Software Architecture in Practice, Bass et al describe the Architecture Business Cycle (ABC), which describes how system architectures evolve through changing conditions. But if architectures necessarily define what is hard to change, how can we expect this evolution to take place?

We must embrace a simple truth: we will never be able to design architectures that are entirely timeless.

Extreme Programming (XP) doesn't talk about architectures; XP practitioners use the term metaphor instead. They confound metaphors of usage (designing systems based on metaphors to real-world concepts is a common best practice in usability, for example, most appointment managers' interfaces are based on paper planners and calendars) with metaphors of system structure (the core patterns and idioms; basically, the software architecture). But this may not be an entirely bad thing. The high-level usage metaphors (the desktop metaphor of most personal computer operating systems, the conversational metaphor, direct manipulation, etc) need to be well supported by the architecture of the system. Although these metaphors do change, they tend to change less frequently, and thus they fit the rigid parts of the software structure (the support walls) better. The architecture, then, should ideally be the system view of the primary usage metaphor.

The other point XP makes about the software design is that it must change over time (remember that XP's motto is "Embrace Change"). XP emphasizes refactoring all portions of the software design (architectural or not) as new needs become available. Given that the architecture must evolve along with the system usage, this constant attention to refactoring begins to make sense. Sure, the architecture is likely to change slowly, but at least we recognize up front that this change will happen and we must be constantly steering it in the proper direction. It's much easier to make many small changes over a long period of time than to make huge sweeping changes all at once (during which you'll be hard pressed to get anything else done, like adding new functionality).

But it's rather unsatisfying to think that the best answer we can come up with to the software change problem is Well, you just gotta constantly keep your eyes open and do the best you can to adapt the system to new needs. We'd prefer to find more perfect metaphors that change less often, so we can spend less time refactoring and more time adding new features, improving the interface, etc. Is this even possible? I would answer yes, but only through close examination of the very thing that defines the software architecture to begin with; the ways users work with the system, especially the ways this work changes over time. Getting at this data is difficult, but once found, the metaphors and architectures developed through these studies can be reused through product lines. Organizations that achieve this capability will possess a huge competitive advantage.

Unfortunately, few organizations (if any) currently possess such capabilities. In general, hoping for timeless architectures is unrealistic. To a Taoist, everything that lives must change, and those that refuse to change, die. Architectures that are healthy, that are being used by real people doing real work, will need to adapt to remain healthy and continue to do good work. Those that stay static, like everything else in this world, will soon die.

[ Permanent Location ]

Patterns and U&SA

patterns, software development, usability

July 05, 2003, 06:00 PM

At the command of my bosses, I'm working on researching the relationships between our U&SA scenarios and the software patterns found in Design Patterns and Pattern-Oriented Software Architecture Volumes 1 and 2. So far, I've gone through Design Patterns and compared the "Applicability" sections where the Gang of Four discuss the conditions under which you may want to use that pattern to the "Responsibilities" we've identified for each scenario and thought out whether the two seem to connect in any meaningful way.

I've found surprisingly many potential relationships between the Gang of Four's patterns and our responsibilities. However, it's difficult to say much that's concrete about these relationships without dictating a particular implementation. The responsibilities alone could be fulfilled by many systems (that's their whole point); the patterns may apply to some of these systems, but others they may just succeed in complicating unnecessarily. This was certainly a problem I encountered with EE; the first architecture I came up with, taken from a book on developing Websphere applications, was far more complex than it had to be for EE's requirements. Even after several refactorings, EE's architecture is probably more complicated than it should be. For this reason, I'm wondering if its possible to give meaningful implementation advice without at least assuming a particular domain, such as Windows desktop applications, web-based applications using J2EE, etc. And even then you run the risk of suggesting solutions for problems the team doesn't have. Since we have neither the expertise nor the desire to pronounce on optimal design strategies in all these possible domains for all possible requirements, maybe its just not possible for U&SA to say much at this level of detail.

On the other hand, it may be useful for developers to have a sense of which patterns may help implement the scenarios and how. This could have several benefits, including:

• Helping the team determine how to implement the scenarios without impacting other important quality attributes, such as modifiability or performance. A fear that the scenarios will negatively impact these other attributes is one of the three main reasons a team may not implement these scenarios (the other two being that they were deemed too expensive or time-consuming to implement and that the team simply did not consider them), so this could be a major boon.
• Saving the developers time in investigating solution options. We've already collected several best-practice solutions in the patterns literature for them, they just need to make these concrete for their system.
• Getting the interface designers' input on what changes, if any, need to be made to the patterns. For instance, some of the patterns don't support returning feedback on the results of an operation (Observer comes to mind). This may be a problem if user feedback on the results of the operation is required to implement the interface design.

All in all, I think this has been a useful (if tedious) investigation so far and is helping us move forward with U&SA. We are planning to publish a paper on the U&SA framework soon, hopefully in CHI and/or ICSE, which is something I've been advocating for awhile since I think we really have come up with something with potential here. Definitely something to look out for, and it'll have Rob's name on it, so you know it's got to be good :).

[ Permanent Location ]

How User Research Feeds Future Usability Activities

usability

July 02, 2003, 11:01 PM

Andy commented on my Personas and Open User Research post over on his Blozom News weblog yesterday. He suggests the use of task modeling, which appears to be a technique for expanding the users tasks into specifications for the system responses. Basically he wants to make the line between the Personas and Scenarios that I wove my hands at more explicit.

I think this is a great idea, and it points to the power of having a user research technique such as personas in place. The information on the users' goals and tasks a project collects through this user research feeds in to many "downstream" usability activities that I didn't discuss in my manifesto, such as getting to an actual interface design, testing that design, etc. For example, many usability techniques assume you know what the users' tasks are:

• Think-Aloud Usability Testing, as well as its many less formal variants, usually requires the experimenter to give the participants a standard set of tasks they must attempt to perform with the interface. The experimenter then looks for any problems his users consistently run into and redesigns the interface to prevent them. But if the tasks he's testing aren't tasks his real users want to perform, then usability testing is worthless.
• Cognitive Walkthroughs involve an in-depth analytical test of how well the interface supports certain tasks, down to the level of a series of questions that the analyst must answer about each step in fulfilling the task. But again, this analysis is wasted if the tasks tested aren't tasks real users perform. Moreover, the answers to many questions in cognitive walkthroughs depend on assumptions made about the user which are likely to be incorrect if not backed up by solid empirical research.
• GOMS is another analytical technique that estimates how long it will take an expert user to perform given tasks (and thus tests for efficiency of use rather than learnability or ease of use). But again, its no good to any expert if she can perform tasks she never actually needs to perform blazingly fast.

"Know the user" techniques like persona development form a solid foundation for almost all future usability activities. With personas in place, techniques like task modeling, cognitive walkthroughs, usability testing, etc. provide valuable insights into how the interface should be designed and whether this design is usable for the target audience. Without them, these future activities are compromised since they may rely on faulty assumptions about the users' needs.

[ Permanent Location ]

Personas for End Users

usability, writing & communication

June 29, 2003, 07:59 PM

While plugging away at Newsable's architecture yesterday, I came up with a possible side benefit of the open-source personas strategy that I am attempting to apply to this project.

One problem I frequently encounter while searching for a new piece of software to meet some new need I have is that, although I know what task I need it for, I can't necessarily tell how the features and screenshots shown on a potential application's website will help me accomplish that task. I usually only download or buy software if I know someone who had the same need and is satisfied with how a particular application fulfilled it (this is how I learned of OmniGraffle; my friend Matt was a satisfied customer who talked me into trying it). Otherwise, I frequently have to download the application, install it, and use it for a few days to see if it appears to do everything I want it to. This is a pain in the ass and takes a lot of my time, so I don't generally download a whole lot of new software.

If, on the other hand, the applications' websites had accurate personas online describing the people and tasks they designed the application to support, I might be able to assess how well the application will fit my needs before I download it. If the fictional people described by the application seem to be similar to me with respect to their needs and tasks, I can assume the application has been designed to support my tasks and thus is worth spending the time to check out. This isn't too far from the idea of testimonials or "day in the life" scenarios, which marketing departments have used for ages to try to get consumers to purchase their products.

I have no idea if the average user would be willing to read a persona when making a buying decision, but it doesn't seem much different than reading a product review or a brochure. So perhaps this technique has potential for end users as well as interface designers.

[ Permanent Location ]

Personas and Open User Research

processes & methodologies, usability

June 25, 2003, 10:16 AM

As I described on Newsable's About page, the goal of all this user research and personae development is to provide a technique for open source project teams to design and test usable interfaces. However, thus far I haven't made this technique explicit. This post is a first attempt at doing that.

In The Cathedral and the Bazaar, Eric Raymond presents open source software development as a highly decentralized process of many collaborating minds spread out over a potentially large geographic distance. However, Raymond glosses over the fact that most open source software projects are almost entirely managed and worked on by one individual (the "benevolent dictator" approach, as seen on Linus's Linux kernel project) or by a small team of core project members (the "meritocracy" approach, as seen on the Apache HTTP server project). What makes open source unique is that there are an order of magnitude more people who contribute in small ways to the project by finding bugs, suggesting changes, and possibly even sending code patches to the core team. So in essence, the typical open source project is run like this:

This is not a bad thing. We want to have one or a small number of team members in charge of developing our software systems when at all possible; this is one of the core tenents of Agile software development and was advocated by Fred Brooks in his classic text The Mythical Man Month when he asserted that the design of a software system should come from one mind.

Good user interface design is, in this respect, not so different from good software design, and thus we can expect the basic open source method to apply equally well. A small number of people should be responsible for the design of the interface, but these people need to be informed by the experiences and findings of a large body of users of the software. But when we're designing an open source interface, what takes the place of the "source repository" in the picture?

The obvious answer is the user interface source code itself. But nonprogrammer "normal" users can't understand user interface code, much less make suggestions on how to change it. Using the interface code as our artifact would alienate a large and very important part of the user population, which contributes to the usability problem open source software is experiencing now.

So perhaps we can use the interface design itself as our artifact. We could produce screenshots and mockups and share these with the users to get their feedback on usability problems with the design (the equivalent of "bugs" in program code).

Unfortunately, it's well known in usability circles that users aren't very good at providing accurate feedback on interfaces they can't use. Moreover, when users make suggestions about interface designs they generally come up with localized solutions that solve a particular problem they're experiencing but may make the interface as a whole less usable. Finally, people with the same user interface problems often have different opinions of how those problems should be solved. If they are spending all their time bickering over the solution, they may not even realize what the underlying common problem is.

What I propose, then, is that open source projects produce a set of personas, or archetypal users, and scenarios, or stories of the personas using the system, that encode their understanding of who they are designing the software for. One or more members of the core team can take responsibility for maintaining these documents. Outside contributors can provide additional ethnographic study data, user test data, cognitive walkthrough results, etc. that may alter the project team's understanding of who their users are. Users can contribute by pointing out differences between their usage of the system and the usage described by the personas; if the personas don't behave like real people, the users will be able to quickly spot the error based on their own understanding of how they work. The project team can then use these personas to agree on an interface design based on a clearly articulated understanding of user needs. So now, we have a process that looks like this:

Discussions of interface design decisions should all center around the personas. Instead of arguing over personal preferences ("I like the buttons on the top!" "Well, I like them on the side!") or about the mythical "users" ("I think users will find the buttons easier if they're on top." "Well, I disagree.") the project team can talk about the concrete user archetypes they've maintained and reason about how these users would behave if the new interface design suggestions are implemented.

In essence, this technique aims to be an "open user research" technique that closely parallels open source software development techniques. By encoding the team's understanding of their users into personas and making this understanding available to all, the team can leverage the unique benefits of working in an open environment without losing the conceptual integrity that comes with having a small number of people making the actual interface design decisions.

[ Permanent Location ]

The Ways We Read

internet, psychology, usability

June 25, 2003, 12:22 AM

Earlier this month, I ran an investigation of how people browse periodically updated websites (such as news sites, weblogs, or any page whose content changes and people keep coming back to view these changes). I ran these studies to inform the design of Newsable. They are directly intended to feed into the development of Newsable's user personas, but I promised several people I'd post the results when they became available, a promise I've been remiss in fulfilling thus far. So, at long last, here's the full writeup of my findings.

Experimental Method

To collect the data, I followed the basic Contextual Inquiry technique as described in Contextual Design. In essence, Contextual Inquiry is a quick-and-dirty form of ethnography. Like more rigorous ethnographic techniques, Contextual Inquiry relies on observing study participants in their "natural environment", i.e. while they are performing the tasks you are interested in as they would normally perform them. Unlike other techniques, Contextual Inquiry calls for lots of interaction between the experimenter and the participant; the experimenter constantly interrupts the participant's work to ask questions, clarify assumptions, explore areas of particular interest in greater depth, etc. A good contextual inquiry, then, keeps the participant "in context", or requires them to show the experimenter how they do their work rather than asking them to give vague summaries that won't contain many important details, while at the same time getting at the behavior the experimenter is interested in without having to spend months as a "fly on the wall" while dispassionately observing the participant.

For me, this translated into sitting down with the participants as they read their daily websites and asking them about what they were looking at, what they were thinking, why they chose to click this link and not that one, etc. In total, I observed eight users for about an hour to an hour and a half each. People who are trying to be rigorous about their contextual inquiries often videotape the entire session for later analysis. I just took notes.

Note that the nature of the data Contextual Inquiry is designed to collect is not statistically generalizable. Ethnography emphasizes deep investigation of the full range of rich behaviors for a small number of participants, not narrowly focusing on one tiny aspect of the behavior for a large number of participants that will yield a p of less than .05. This means that the findings I'm about to present are (hopefully) sufficient to guide the design of a news aggregator but not sufficient to make timeless generalizations about human behavior.

Results and Findings

Of the eight users I observed, two already used a news aggregator (NetNewsWire and Radio Userland). There was a clear correlation between the number of sites participants read and whether they used a news aggregator. The six who did not use an aggregator read around three or four sites every day. The two who did read upwards of fifty different sites on a daily basis. I think this speaks to an interesting quality of news aggregators. At first blush they don't appear to be very useful programs, after all, all they do is save you the minimal effort of clicking on a few entries in your bookmarks sidebar to hop from site to site. But by collecting the news from various sites into one easily-scannable location, news aggregators actually greatly increase the amount of content the reader is willing and capable of skimming regularly. This has the potential to expose people to a much wider variety of news sources and their corresponding perspectives.

All the participants I observed spent a lot of their time skimming content to see if it looked interesting to them. They all liked to visually scan for content that looked interesting; they didn't approach the news-reading task with many preconceived notions of what kind of content they were looking for, but instead glanced over what they found and made a judgement call on how interesting it was on a story-by-story basis. They generally preferred having both a title and a summary or short excerpt for each story rather than just the title; more information was more better (these observations explain why Google News decided to eschew Google's characteristic minimalist design in favor of a much denser information display).

For the most part, all participants appeared to prefer to see only the items that were "new to them", i.e. that had been added or modified since the last time they visited the site. Participants expressed a preference for sites that listed news stories in reverse-chronological order so they could read until they hit a story they recognized. A couple participants, however, actually preferred to read through the listing in chronological order and would skim down to a story they recognized then read backwards up to the top of the list. Finally, one participant remarked that she actually preferred to see the old stories as well as the new ones since the old stories helped her remember the context in which the new stories took place (if they were continuing coverage of some running event, for example).

Another common theme was the importance of the reputations of the authors of the stories and the publications they were writing from. As I've already mentioned, the participants depended heavily on their knowledge of the reputation of the authors of news commentary to determine whether they were worth reading or not. Participants would frequently read their favorite authors even if the topic of the essay didn't particularly interest them. Moreover, several participants had developed an understanding of the personality and interests of the people they read, and used this knowledge to help determine whether an article by that person was worth reading. Of all the criteria people used to determine what to read, the reputation of the author was consistently the most powerful.

Almost all of the participants would occasionally send stories they found interesting to friends via email or instant messanger. One participant remarked that she viewed reading the news as a very social activity and relied on her friends to keep her informed about sites she was interested in but not sufficiently to read on a daily basis. Another mentioned that he liked it when his friends read different websites than he did so they would have something to talk about later; he liked to be able to trade interesting news tidbits with his buddies.

After running two or three of these observations I become interested in how the participants reading habits changed over time. Although I wasn't able to observe this directly, participants generally described their reading habits as changing gradually, especially those without news aggregators who only read a few sites. Frequently, they would discover new sites by following links on their current news sites, and if the new site continued to come up with interesting content, they would sometimes add it to their daily "rotation list" of websites to check. Participants reading patterns also changed with their current interests; one participant was taking a photography course and thus was starting to read more weblogs on photography than she had before. Additionally, many would learn of new sites through friends who emailed them interesting stories, and sometimes these sites would be interesting enough to check daily. Some of the users without aggregators remarked that they knew of many sites that were interesting but not interesting enough to check every day, and thus they tended to check them very infrequently since doing so wasn't part of their daily habits. Participants with news aggregators tended not to have this problem.

Finally, one common theme was that participants tended not to remove sites from their daily rotation lists very frequently; one user (who did use a news aggregator) remarked that the cost of leaving a somewhat-uninteresting site in his list was so low that it wasn't worth the couple of clicks it would take to remove it, he only removed sites if they were both uninteresting and posted a lot of content that got in his way. Of course, this participant was also confronted by upwards of 300 stories on a daily basis...

Conclusions

The observations presented here have been distilled into the user personas of Newsable, the usable news aggregator (coming soon). If you're interested in this research, you may wish to read over the personas to get an idea of how all this comes together into a coherent picture of the user that website authors can design for. The personas are the "cannonical version" of this research and will be updated as new findings become available.

Thanks to everyone who put up with my obnoxious questions about their news reading habits to participate in my studies (without any compensation, no less). My understanding of what people need in a news aggregator has changed dramatically as a result of talking to you all.

If you have any questions or comments relating to these findings, please feel free to email me or leave them below. I plan to continue these studies as an "open source user research" project; more on that shortly.

[ Permanent Location ]

Case Studies in Open Source Usability: KDE and GNOME

processes & methodologies, usability

June 20, 2003, 10:16 PM

I joined the GNOME Usability and KDE Usability mailing lists this week, mostly to lurk and watch the discussion unfold to hopefully gain some insight into what the state of the practice in open source usability is like. Here's my (very) preliminary thoughts. Don't take these comments the wrong way; on the whole I think both projects seem to have an impressive emphasis on usability and are doing great work.

The Good

• There do appear to be people conducting actual user tests on the desktop environments for both projects, and these people are posting their findings to the list. Moreover, the developers appear to listen to them and follow up on their questions and recommendations.
• Both projects have a list of guidelines (GNOME calls them the "HIG", which I assume stands for "Human Interface Guidelines"), and these guidelines actually get mentioned in discussions. There's a discussion going on right now on the GNOME mailing list where the HIG was brought up to reason about why a suggested design decision may not be the best choice (a decision about using Alt keys in a keyboard shortcut, to be specific).

The Not-So-Good

• There seem to be a lot of "let's do it this way because I like it better" type of discussions, where list members are just arguing over personal preferences. So far there haven't been any serious flamefests, but there hasn't been a call for "let's resolve this by collecting some user test data" yet either.
• The people who are doing the user tests come to the list with problems and occasionally suggested solutions, but so far no one has come back with a story like "I user tested this part of KDE, and I found these problems. So I prototyped these suggested fixes and ran my tests again, and the problems disappeared. So I suggest we implement my fixes." HCI practitioners have found that software developers don't want laundry lists of problems, they want to know what exactly they need to implement to make their system more usable. So the HCI person needs to come back not just with problems and a few half-baked ideas, but with solid, tested solutions for the developers to pick up and run with.
• Neither project seems to have a consistent picture of who they are designing for (user profiles) and what these people want to do with their software (task lists). So far, its unclear to me how the user testing people decide what tasks to have their test users perform.

Watching these projects discuss their usability issues is a great experience, and I'd recommend signing up for these lists if you're interested in open source usability. I'll post again after I get a bit more insight from reading the list to see if any of these thoughts change.

[ Permanent Location ]

Natural Programming and Expanding Open Source

design, software development, usability

June 15, 2003, 11:48 AM

Sometimes when I come across something I want to write a weblog entry about but I don't have time to put together the full writeup (these things can take over an hour) I instead jot down a quasi-outline of the post so I can come back to it later. I have a couple of these kicking around right now and I wanted to get them off the table. This one comes from the same discussion with Jim about open source and usability that the efficient interfaces post originated from.

Jim brought up "natural programming" (programming environments designed to be so easy to use that ordinary users can program, if only in a limited domain) as a holy grail of open source usability. If everyday users can program, the reasoning goes, then users can design their own interfaces and feed those designs back into the development base, just as open-source developers do now. This comes from Eric von Hippel’s argument in "The Sources of Innovation" that most innovations come from end-users, not manufacturers. I balked at this, however. In HCI, its commonly accepted that its unwise for an interface designer to take all user suggestions at face value. Users are not designers; they are good at finding problems with the interface, but they tend to come up with point solutions and workarounds that fix these problems in the small but which, overall, contribute to a bloated, unusable, inefficient interface. This is why providing highly customizable interfaces doesn’t solve the usability problem. In "Usability Engineering", Nielsen quotes a study that compared user performance with two interfaces: one was designed by an experienced interface designer, the second the user was allowed to customize in any way he wanted. The study found that users were much more efficient at their tasks with the interface designed by the expert than the one they customized.

On the other hand, as I discussed in my efficient interfaces post, when you have a situation like in open source where the users (the programmers) really are the designers, sometimes superior designs do come out of users’ heads, at least with respect to certain usability goals like expert efficiency. This is certainly in line with the "eat your own dog food" maxim of software development. On the other other hand, many complaints about open source software center around the observation that most of the popular open source interfaces are copied and altered versions of existing commercial tools, rather than truly innovative designs (KDE and Gnome look a lot like Windows, Gaim is pretty similar to AIM, Linux is basically a reimplementation of AT&T’s Unix). Right now, open source interfaces are good at copying commercial interfaces, making them free, and streamlining them (to a certain extent), but not much else as far as usability is concerned.

So would natural programming help to bring users into open source and produce better interfaces, or do expert designers have to participate more to bring their unique knowledge and experience to the table? For my open source and usability project, I’m moving more towards the expert designer route, although I’m still very interested in investigating what the best way to bring average users into the community is. After all, it’s worth remembering that the study Nielsen refers to only measured how productive users were with the interfaces they designed. Historically, usability has been applied to the design of productivity applications almost exclusively. But not all software is for productivity, many software products aim to provide the user with an enjoyable or desirable experience, as Don Norman discussed at CHI. Perhaps interfaces of this variety would benefit from natural programming and high user customizability. After all, if users alter the interface to their tastes, perhaps we can assume they like it better that way. Hopefully the emotion and design people will do some studies looking for correlations between the amount of control a user can exercise over the interface design and the interface's desirability for the user.

In the short term, however, I'm not certain that natural programming is the way to go to solve the open source and usability problem. More on the approach I do plan to take should be coming soon.

[ Permanent Location ]

Fury, By the Book

processes & methodologies, usability

June 10, 2003, 01:38 PM

Kevin has embarked on a project to redesign Fury, his popular personal website / weblog. When most people get bored with their old website design, they hide away for some period of time and work on the new site in secret, showing early "drafts" of the interface to a few friends if they show it to anyone. But Kevin's decided to try a "by the book" redesign of Fury, complete with task analysis, low-fi prototyping, cognitive walkthroughs, usability tests, and log analysis of the current site. He's already identified some portraits of common users which he created by analyzing Fury 3.2's log files and put up a wireframe mockup for user feedback.

I'm particularly interested in seeing how this project goes since it ties in so nicely with my current Usability and Open Source project. Although Fury is not, to my knowledge, open source, Kevin is working in a similar environment as many lone open source developers, e.g., he's working in his spare time, has a 0$ budget, has a strong personal attachment to the interface design, etc. He's also a much more skilled and experienced interaction designer than I am, so it'll be neat to see what he comes up with.

I hope Kevin continues to keep us informed on how the project is going and how he's adapted his process and techniques to the job (which, given his voluminous posting history, I'm not too worried about). I also hope he doesn't get bored of the project before its completion (which, by his own admission, is a real possibility :).

[ Permanent Location ]

U&SA Website Is Live

announcements, software development, usability

June 09, 2003, 03:20 PM

The Usability and Software Architecture project finally has a website. The front page has a nice short description of what the research is all about which may be of interest. All of our publications that we're allowed to put online or link to are available as well in case you'd like to learn more.

Feel free to send me any comments about the site or requests for additional information you may have. I can almost guarantee that any insightful feedback anyone sends us about the site will be brought up as a serious item at our next meeting, so don't hesitate out of fear that your email will only serve to further clutter Rob's already horrendously disorganized Inbox.

[ Permanent Location ]

Innovation and Shifting Web Standards

internet, software development, usability

June 08, 2003, 02:24 PM

I wanted to add a quick (and slightly belated) blurb to the recent buzz over the release of Mozilla Firebird, Microsoft's comments on how they plan to stop releasing standalone versions of IE, and the Microsoft-AOL deal to include IE in future versions of AOL-Win instead of Gecko.

Some have suggested that Microsoft's sins with IE have involved stifling innovation in web browsers. The argument is that since Microsoft has now clearly "won" the browser wars, they no longer have any incentive to innovate as is clearly evidenced by the fact that no significant improvements have been made to HTML/CSS in the past four years.

I mostly agree with these sentiments, although I'm not so certain that changes in the core standards are appropriate at this time. Marc's post above implies that browsers have failed to add new features in recent years, they've only been playing catchup with old features (like CSS) by providing better (or more pessimistically, less broken) implementations of the standards, and this is a bad thing.

I agree that new features are good, but I disagree that freezing the standards and taking the time to get solid implementations of them out on the market is a bad thing. I like the comment on the need for decent CSS authoring tools, however. CSS2 is a pretty nice standard already as far as they go, but most website authors don't make full use of it since its so hard to create good CSSified web pages with current tools. As long as authors have to muck around with editing text languages to make websites that take advantage of the content-presentation separation that CSS offers, only the technically sophisticated few will benefit. What's needed is a good single-sourcing normative interface like I've called for before.

Innovation occurs at many levels in computing. Sometimes innovation must slow down at the lower levels (the core standards, such as HTTP, HTML, CSS, etc.) so that effective innovation can occur at the higher levels (such as web browsers, WYSIWYG web publishing tools, content management tools, etc.) The weblogging community is going through this phase now with RSS, an important standard for both publishing tools and readers. It's hard to create effective tools when the standards they are based on keep shifting underneath you.

On the other hand, there are some instances where a new feature in a tool requires a change to the core standards. For example, back in the '90s Netscape wanted to allow website authors to embed aribitrary media content in their pages, so they created the <EMBED> tag. IE had the same idea, but since there was no standard defining how to specify this behavior they made up the <OBJECT> tag, which of course was completely foreign to Netscape's browser. For a long time, authors had to kludge together both tags to consistently include media elements in their pages.

The basic problem is that the usability of any software system is not screen deep, the central point in our U&SA research. Many functions that users want reach far deeper into the software system that just the surface interface, and may require changes in the lower-level protocols and data formats themselves. If these formats are standardized, then the standards may need to be modified, or violated with vendor-specific extensions like <EMBED> and <OBJECT> which are rarely compatabile and cause large amounts of pain for users and developers.

So how do we ensure that the majority of these functions get written into the standards themselves, thereby preventing this problem? Perhaps standards need to be developed in a more user-centric fashion; they need to be informed from the beginning by tool development teams that investigate how the needs of users are going to affect the lower-level formats. Or perhaps this can't be done; the needs can never be determined except through painful, long iterations involving standardization, new needs uncovered, incompatable vendor-specific extensions, arguments and flamewars, and finally consensus (or at least cease-fires) and restandardization. Wash, rinse, and repeat.

[ Permanent Location ]

Consolidating Bookmarks

internet, usability

June 06, 2003, 11:52 PM

For my ongoing effort to construct a usable open-source RSS news aggregator, I've been doing some modified contextual inquiries of several of my compatriots in the HCII and elsewhere here at CMU. So far, I've observed five participants as they checked various web sites that update on a periodic basis and that they return to frequently to locate fresh content. I plan to run three more users to refine my personas which will feed into the interface design of my new aggregator.

As an unexpected side effect of the experiment, I've run across a real need for a centralized bookmark system in web browsers. The basic problem I continue to encounter is that all the participants I've observed have multiple computers they use on a regular basis (usually one at home and one at work, sometimes a separate laptop to boot). This creates a problem with bookmarks, since if they bookmark a page on one computer then want to access it from the other, they're out of luck. Many people didn't bother using bookmarks for this reason, and instead just remembered a few sites that they typed into the location bar manually. Some people even use Google to find sites they return to on a daily basis!

This is a problem I've confronted myself since at least 1998 (I now have a work laptop, a home desktop, the home server that handed you this page, and a home "backup" desktop, all of which I still use at varying degrees of frequency), and it's nice to see that I'm not alone. Granted, since all of my participants are from CMU, it is possible that they are unusually computer-oriented and thus are abnormally more likely to have multiple machines they use regularly. But I doubt this is the case; many white-collar workers nowadays have one or more computers at work as well as owning a home computer for their personal use. Even people who own laptops often like to keep their professional and personal computer usage physically separate.

To solve this problem for myself, I was thinking of installing a server-side bookmarks application on the Labs, but this is an inelegant solution that's only appealing because there are no better alternatives. Mozilla should provide built-in support for a centralized bookmarks repository (and possibly history and settings as well) to eliminate this problem, or possibly look into other design ideas for helping users to keep track of sites they wish to return to (since bookmarks have a tendency to get cluttered, similar to file folder hierarchies). I know Netscape used to have a concept of "roaming profiles", and IE supports a similar feature on LANs, but these tend to only work on intranets, if I recall correctly. If there are any available solutions, then they are obviously too difficult to set up, since no one I observed was using them (or even aware of them).

So enough ranting for now. I intend to run three more users, then pull together the observations to refine my personas and scenario analyses. Some people have expressed an interest in seeing the results of the study itself, so I'll probably try to put together some small report about it to share with you all. Stay tuned!

[ Permanent Location ]

Open Source and Efficient Interfaces

software development, usability

June 04, 2003, 01:03 PM

I met with Jim today about my open source and usability project. We were chatting about the current state of usability in open source software projects, and I observed that although open source products tend to have inferior usability with respect to novice to intermediate users of the type of system in question, they tend to have superior usability for experts. With commercial software, I mused, it is frequently the other way around, even if the project has a strong usability focus. Most of the commonly used usability techniques such as heuristic evaluation, cognitive walkthrough, usability testing, etc. are good at discovering the problems of first-time users and improving learnability and reduction of errors. They are not so good at improving expert user efficiency or satisfaction, since these attributes of usability generally only appear over long usage and thus are hard to detect in small experiments and heuristic methods.

This factoid has held true throughout my idle reflections on my experiences, and on a certain level it makes sense: open source developers and contributors tend to be power users who are highly concerned about efficiency and advanced functionality, and they tend to design for themselves. So when the user is, in fact, like them, open source interface design roughly works well. This is why many programmers eschew fancy commercial IDEs in favor of Vim or Emacs. The designers of Vim and Emacs know what programmers need, because they are programmers themselves.

I wondered, however, if anyone had done any research to provide more strength to this assumption than my idle hunch. What would be even cooler would be if someone investigated how various open source development social setups (e.g., Linus's "benevolent dictator" approach, Apache's meritocracy, Mozilla's more formal, commercialish approach) correlated to expert efficiency of the resulting interface. Jim couldn't think of any offhand and directed me to MIT's Open Source Research website.

I'll check it out, but I imagine there's probably a research gap in this area. And I don't think it would be too difficult to fill; someone could use Bonnie's GOMS technique to analyze expert efficiency for a number of common tasks with two similar products, one closed-source and the other open-source (e.g., Trillian and Gaim). They could also try to correlate the open-source development social setup with the efficiency reported by GOMS. I'd imagine this study could have interesting results both for the HCI and open source communities. Unfortunately I have no interest in actually running it. Abby, are you out there? :)

[ Permanent Location ]

The Users We'll Never See

design, information, society & sociology, usability

June 03, 2003, 05:10 PM

About an hour ago, I felt a need to feed my Chipwich addiction, so I invited Kerry to go to Entropy with me. On the way back, she needed to stop by the Hub to get some registration and class scheduling issues worked out. Since I had nothing imminently pressing to do, I waited for her.

While sitting in the Hub waiting area, surrounded by brochures, magazines, important-looking forms, computers, and the various other trappings of a functioning administrative center, I started to reflect on how much information was out there in the world, and how much of it had to be dealt with. That's the thing about information; it always seems to need processing and organizing and analyzing and basically loads and loads of attention. Information can be rather childish in that sense.

I was reminded of Herb Simon's quote: "In the future, the scarce resource will be human attention". I reflected for a bit about how right he was.

But wait a minute, a little voice in my head remarked. Exactly how right was he? Sure the information-overload problem is a big deal for people at CMU, and probably is for information workers everywhere. But is everyone like us?

My father owns a house in rural West Virginia (Monroe County, for those of you familiar with the area). Life's quite different out there. People aren't so busy, for one thing; they don't appear to be so bothered by all this information. The information is there; there are construction projects and farm vehicles and weather patterns and plant DNA (scads of information lies in the DNA of even a simple organism) and who knows what else. But this information doesn't seem to be so concerned with getting your attention. It could care less about being processed.

My point is not that this "simple" life is ideal or perfect; it isn't. My point is not even that it's better; "better" is a matter of opinion and generally doesn't mean much in any absolute sense. Hell, I happen to like city life; I wouldn't move to the country if I was given a million bucks to do it. Rather, my point is that old Herb was not thinking about these people when he made up that quote. And how could he? He was here at CMU, where everyone is information-overloaded. Of course he saw the future in those terms; he was extrapolating from the present, just as any rational person would.

As human beings, our conceptions of the world and its problems are necessarily a product of where we are situated in society, both physically and status-wise, and who we choose to associate with. Dan Sieworek, Scott Hudson, and many other researchers here at CMU, are taking Herb's quote seriously with projects like Aura and Situationally-Appropriate Interaction to try to solve this information-attention problem of the future. But whose problem is this, really? It's their own problem, ultimately, and the problem of the people they associate with and the people who fund them.

As user-centered designers, we can only design based on our conceptions of the world. We can only perceive and design for the slice of reality we find ourselves in, and for most of us, this slice is quite limited.

Around this point, I was interrupted from my meditations by the secretaries loud discussion about whether picture files are supposed to end in ".jpg" or ".jpeg". They spent a good five minutes trying to figure out the correct answer (of course, both extensions will work fine for most modern programs). I smiled. File extensions are one of those known usability problems that no one can get seem to rid of. Usability advocates routinely complain about how design decisions such as this one are made by programmers who don't understand the mentality of the users they are designing for. But in the large scale, in the "redesign society" sense that researchers and the upper echelons of industry and government confront daily, does anyone really know who they are designing for? Does even the most skilled user-centered designer have any meaningful grasp on who the users are when the user base includes the population of entire countries (or the entire planet)? In this scenario, even we user-centered designers may be no better than the archetypal isolated programmer in a cubical we so frequently revile.

Sometimes I wonder if at least a subset of the designers of the world need to step out of their bounds sometimes. Some of us should go out into the world to observe the people whose lifestyles and values clash with the ones we are familiar with. Perhaps this would help us get a larger view of the world we are so eager to change, essential if we are to do more good than harm.

This idea has some background in design research; last semester we read a paper from DIS 2000 that advocated designing for "extreme characters" as an exercise to help break out of the mould of the existing interface design frameworks. But this is just a thought experiment; the extreme characters described in the paper are (by admission) caricatures of real people. A designer can't truly know what the elements of society outside his spheres of experience are like without seeing them for himself.

Now if we really were able to see the people of the world, the "end-users" of our world-changing visions, as who they are, and not just who we assume them to be, well, how mind-blowing would that be?

[ Permanent Location ]

Comparing Complexities: Interface Designs and Code

design, software development, usability

May 30, 2003, 11:21 AM

I was reading Jacob Nielsen's Alertbox yesterday, and noticed he made an analogy I've frequently used myself; he compared interface design to software development to demonstrate the folly in assuming that good designers shouldn't have to user test their interfaces. The argument goes that you wouldn't make the claim that "good software developers should be able to write perfect code on the first try, so why should they have to waste time writing and running tests", would you? Interface design is just as complicated as software development, so why would you assume good interface designers should be able to pull perfect interfaces out of their butts without testing and iterating on the design?

This got me thinking about whether it would be possible to make this argument stronger by semi-objectively comparing interface design complexity with code complexity. Obviously the standard code-complexity metrics (lines of code, function points, etc.) don't apply to interfaces. But what we really want to measure here is "how many things could you potentially screw up?", i.e., could fail in testing. I'll call these things "complexity points" for lack of a better term.

For code, an example of complexity points in a function might be "null pointer passed in and dereferenced" or "array accessed out of bounds", basically, any state that would violate the preconditions, post conditions, or invariants if you were using Design by Contract. For interface design, example complexity points might be violations of Nielsen's heuristics, failures in a Cognitive Walkthrough (interface elements that aren't visible, labels that don't describe the function, etc.) or potential violations of a Think-Aloud critical incident. Obviously no objective procedure could hope to turn up all the complexity points either in source code or interface designs (it'd be great for humanity if we could, although all the software developers and interface designers in the world would get replaced by computers), but perhaps there is some way of approximating the complexity point count for interfaces and source code for the sake of making a rough argument.

As a side note, I'd read Jacob's column more often if he had an RSS feed...

[ Permanent Location ]

Prescriptive Interfaces Explained

design, processes & methodologies, usability

May 27, 2003, 02:16 PM

A couple days ago I mentioned that I was interested in "normative interfaces" as part of the single-sourcing paradigm idea. I've thought it through some more and decided I liked "Prescriptive Interfaces" better as a monkier (best practices frequently aren't very normative since they're often not the norm...) and that I'd like to expand on some of the points I made in that post.

In a way, the term "prescriptive interface" is a tautology; all interfaces are technically prescriptive since they necessarily predefine some method of interaction with the system. The interface defines what commands the system understands and what form it can present information in, and thus any interface dictates some method or range of methods the user must use to perform their tasks and accomplish their goals. Techniques like CI/CD help us design interfaces that prescribe methods of interaction the user is already familiar with through their existing work, while improving on the existing methods by fixing the breakdowns identified by the design team.

The interesting question, however, is how do we design interfaces that encourage users to work in the smartest way possible, as defined by the techniques developed by long-time experts in the field who have applied them successfully in many different situations. In the software engineering world, these are known as "best practices". One best practice in software development is unit testing your code. One best practice in technical writing is writing an outline before starting the document, as I mentioned before. One best practice in HCI is low-fi prototyping your interfaces to elicit real user test data as early as possible.

A prescriptive interface, then, is an interface designed to encourage the use of best practices, to make it easy (or at least easier) to solve the problem in the "right" way. One example of an interface that did this well is the original Apple GUI toolkit. At the time, there were no high-level GUI toolkits available so programmers wrote the code to implement all their own widgets, thus the buttons, text boxes, menus, etc. in every application looked different, which proved confusingly inconsistent to the user. Apple's genius was in providing a toolkit that not only enforced a consistent look and feel on their platform, but actually made the application programmer's job easier as well. This encouraged programmers to use Apple's toolkit instead of rolling their own, and thus Apple wound up with a remarkably consistent-looking and usable platform for the day. An example of an interface design that does this poorly is Microsoft Word's styles support. Writing a document using styles is hard; the user must first know that styles exist and what they are for, then must go into the styles dialog to edit these styles to make them look the way they want, consistently apply them where they are appropriate, etc. The interface makes it much easier to just hit the "Bold" button up in the toolbar to define a heading than to try to do it with styles.

Ideally, prescriptive interfaces should make it easier to solve problems the right way than it is to solve them the wrong way. Failing that, they should try to make the right way as convenient as possible, and may even consciously make the wrong way just a little bit more difficult. If done right, this approach has potential benefits for both novice and expert users. The interface helps novices learn the best practice approaches to solving their problems in the context of doing their work, which is much more effective than formal training. The interface helps experts by making the best practices they already know much easier to follow; even though I know styles are an overall better approach I frequently don't use them in Word simply because it's such a pain in the butt.

So now that you're convinced that prescriptive interfaces are a good idea (I hope), the question becomes, "How do we design these things?" Here's my thoughts on a rough process:

1. Identify the best practice approach to solving the problem. This may require reviewing relevant academic or trade literature in the domain, but I would guess that the most important practice would be to ensure you have one or more domain experts on the design team who have experience applying the best practices and can advise your design accordingly.
2. Research how users currently solve the problem in their day to day work by using a requirements collection method like Contextual Inquiry. Merely understanding how users ought to work is useless if you don't understand how they really do work and why.
3. Design your interface to define clear transitions and adaptations between how users currently work and the best practice solutions. The interface may need to have a sizable teaching component; collaboration with the learning technologies people may help with this part. Unfortunately I don't have much more guidance to provide here; this is definitely the hard and unexplored part of this technique.
4. Allow users to circumvent your best practice processes, but discourage this as much as possible. Be mindful that your ideal solutions may not fit all situations, but make sure the users don't decide to deviate from the best practices idly.
5. As always, do lots of user testing to make sure it works! Testing is particularly important when you're trying to impose a new way of working. Your assumptions will probably be wrong at first; they'll need to be tested and refined even more than normal interface designs before the concepts are nailed down. Low fidelity testing is probably particularly important since drastic changes in the design may be necessary to fix the breakdowns.

So those are my thoughts as they stand. In conclusion, the goal of prescriptive interface design is to produce products that afford best practices within their domain just as they afford proper use of themselves. A good prescriptive interface builds the experience of experts into the system itself so that even novices can experience that benefits of expert-level productivity "out of the box".

[ Permanent Location ]

Single-Sourcing, Outliners, and Normative Interfaces

design, processes & methodologies, usability

May 25, 2003, 09:18 PM

I've posted before about the single source paradigm and the separation of presentation from semantic content. I've been working with Word's outline mode a lot recently and have started to see how using an inherently structural view like an outline to edit content could be the key to providing a usable presentation-agnostic content management system like I described before.

When you create an outline for a document, a presentation, etc. you are basically considering only the structural and semantic aspects of the work. Your purpose is just to lay out the content and get a sense of how your thoughts decompose into supporting points, how your arguments fit together, etc. After you've finalized the outline, you'll worry about making sure the document flows well and looks good by defining the appearance of headings, putting in figures and messing around with how the text flows around them, changing the pagination, etc.

The important part about this process is that the entire time you're focused only on the structure of the work, deferring the definition of the presentation until later. And this is an example of a natural human workflow that people engaged in long before they had computers, so it makes a nice metaphor for the computer-supported single-content paradigm. So the question becomes, can we design interfaces that build on this metaphor to encourage semantic definition of documents, rather than only providing a WYSIWYG style interface that encourages tying content to presentation? And can this metaphor be extended even to such vastly different presentation decisions like whether to publish a printable document, a web page, or a slideshow presentation?

When I brought this up to Micah, he mentioned that Dave Winer, the creator of Radio Userland, has been interested in outliners and the single-source paradigm for years. I'll have to set aside some time to read Scripting.com to see what his thoughts on the subject are.

This whole concept ties into another area I'm interested in; how do we as software designers develop interfaces that encourage our users to follow predefined best practices? Basically, the idea is that certain workflows can be inherently more productive than others, but users may be following a less-than-optimal work flow in their current practice. For example, most technical writers agree that creating outlines before working on a final document produces better-structured prose than just sitting down and writing (I even try to sketch out "mini-outlines" when I toss together these weblog entries). But many users of Word don't do this; they just sit down and start typing. How can we encourage people to follow the superior practice by making it easier than the alternative? In essence, how can we design "normative interfaces"?

Most HCI techniques help us get at the ways our users think and determine what would be intuitive for them, but don't help us design interfaces that help teach users a better way of working. Solving this problem may prove to be an important component of getting people to think structurally so the full benefits of the content-presentation separation paradigm can be realized.

[ Permanent Location ]

Markets for Usability

society & sociology, usability

May 19, 2003, 10:06 PM

Since many of my friends here at CMU are nearing the end of their one-year program and are actively hunting for jobs, I've been thinking a lot recently about the market for usability in the software industry. Here's the basic problem I've been pondering: although there seems to be a great deal of need for usability professionals in the IT industry, there appears to be distressingly little demand for us.

My assumption that we are needed isn't just based on wishful thinking. Anecdotally, people complain all the time about how frustrating they find computers and technology in general; they can't program their VCR, figure out how to format tables in Microsoft Word, etc. It's a truism in the software field that 90% of the users only use 10% of any reasonably large application's functionality. Why? I believe it's because they don't even know the other 90% is there, or can't figure out how to work it if they do. Many companies that develop software spend way more on running their technical support centers than they do on writing the software itself. The list goes on.

And yet many talented usability professionals with a graduate degree from one of the top academic institutions in the world in the computing field are having serious problems finding a job. Why?

I believe the key problem is not that usability professionals don't provide significant value to their market, but that they don't provide value that can easily be assessed. Since their value is invisible to those making the hiring decisions, they aren't seen as necessary.

Here's my logic: for software developers, the value they provide is tangible. They write code that does stuff; they have a tangible output that could not be produced were it not for them. Companies need software developers if they want to develop custom IT systems or even if they want to integrate commercially purchased systems. I know this is true because I am a software developer and have lived it myself. Product designers (the people who make sure a company's products look nice and are desirable) also provide tangible value since their work translates directly into increased sales. A more desirable product will be purchased by more people, which brings more money to the company, which shows the managers bottom-line results for the hard work of the product designers.

But usability professionals aren't in either of these camps. They don't produce any direct work artifacts that are (a) essential to the company and (b) can't be produced without them like the software developers do. User interfaces are essential, but they can be produced without usability professionals, they just tend to be badly done. Yet unlike ugly or undesirable products, the unusable interface often doesn't directly impact the sales of the product. People frequently buy products with badly designed user interfaces only to realize their error later. And although this may give the product a bad reputation which impacts sales, its difficult to tie this back to raw dollar figures. So the usability professional doesn't have the direct impact on the bottom-line that the product designer enjoys.

The one market that seems to be an exception to this rule is the web design industry. Usability sells better to web design firms than just about anywhere else, I've found, and I think the reason is that unlike conventional products, a web site's marketability is badly damaged by poor user interface design. Users who can't figure out how to use a web site leave and don't come back, which directly impacts the bottom line of the company. Correspondingly, the professional usability scene has had an almost unhealthy fixation on web design for quite some time now.

If we want to make our skills as usability professionals marketable, I think there are a few changes that need to occur:

1. Usability professionals have to become better at showing bottom-line business value, and this means in terms of real dollars. There need to be techniques for getting at least a rough estimate of how much money producing a usable interface as opposed to an unusable one is saving the company. I know Deborah Mayhew has done some work on this.
2. Markets must change to favor usable products over unusable ones, perhaps by allowing users to try products out before committing their hard-earned dollars to them. This has become easy to accomplish on the web, and will hopefully become more prevalent if the industry moves more towards a software-as-a-service model where you rent your applications rather than buying them.

In the mean time, if anyone out there is pulling their hair out trying to design a usable interface, shoot an email my way. I know many amazingly talented people who would love to help you out (for the right price).

[ Permanent Location ]

End-User Participation in Open Source Development

processes & methodologies, software development, usability

May 16, 2003, 03:57 PM

I talked to Jim Herbsleb today about my idea for doing an independent study on Open Source and Usability. He was interested in my ideas and provided some great feedback, so I think the project is a go.

I talked about the ideas I've already mentioned in my earlier post and added Andy's comments about usability logging and feedback. Jim mentioned a system he'd heard of at Bell Labs that was similar to the usability talkback idea Andy posted about awhile back. He thought this would fit well into an open source development process since it would involve the users more in the development of the application. He suggested I consider usability techniques that would take advantage of the unique qualities of open source development, such as open development processes and user participation. Specifically he wondered if there was a way to involve nontechnical users in the development of the software system so they could feel that they were involved with the project and contributing to its advancement.

I think this is a great idea and something I hadn't thought of, but it's also a difficult challenge. The main obstacle to getting real "end-user" participation in the open source development process is that developers tend to have all the power in an open source process. As Scott says, "He who writes the code gets the last word". Developers tend to fix their own problems, and since end users can't write code, their needs may wind up at a much lower priority. However, there may be something to learn from the participatory design tradition which has always made user involvement an integral part of the usability process. Maybe some of the incentives present there will also apply to open source.

These are just a few thoughts; I'm flying blind on this issue right now. Any comments or feedback that could help lead me in the right direction would be much appreciated.

[ Permanent Location ]

A Note on XML and Single Sourcing

design, information, software development, usability

May 09, 2003, 01:29 AM

In reference to my post a few days ago on a single sourcing paradigm for research content management, Dave asked why I didn't mention XML as a solution to the semantic content / presentation separation problem. That's a good question and raises a point I want to clarify.

XML provides a technical solution to the single sourcing problem by allowing people to define markup languages specifically for semantic content and markup languages for presentation specifications (XHTML, CSS, SVG, etc.) and a means of translating between them given some content-to-presentation mapping rules (XSLT). This is a fairly neat answer for the technical problem, but isn't very intrinsically interesting since there have been other technologies that have purported to solve this same problem before. XML is really only interesting because it has broad industry support and thus has a chance of catching on and lasting.

The interesting and difficult problem that I discussed in my post was the interface problem: the "how do you build an interface that lets the user define semantic content and presentation of that content separately, yet still provides a metaphor that makes sense to the average user?" problem. The difficulty is that we don't think in terms of semantics, we think in terms of how things look. Headings in documents are headings because they are in big bold sans-serif fonts. So when we are starting a new section and want to make a heading for it, our first impulse is to type some text, then change it to a big, bold, sans-serif font. If the program instead requires us to highlight the text and click on "make this a heading 3" then go into a bunch of dialog boxes to say "oh and by the way, heading 3s in this document will be in a big, bold, sans-serif font" then I guarantee people won't use it. This is why so few people use styles in Word, despite their many benefits; they break the WYSIWYG paradigm that fits so well with their tasks. And what if the user underlines the third heading in the document? Should the style change, hence underlining all headings in the document? Should just that heading change, creating a new, local style? And so on. I found an excellent article awhile back that expands on the ambiguity of styling nicely.

It's a hard problem, but one that I think could be solved much more neatly than any interface I've seen (Word's styles and Dreamweaver's CSS support both spring to mind as bad examples). And that's what XML doesn't do for us, and what I was driving at in my post.

[ Permanent Location ]

Research and Content Management

information, software development, usability

May 03, 2003, 12:00 PM

I've recently become more and more aware of the need for a "single source paradigm" in productivity applications. To gloss over a complicated topic, single sourcing calls for the separation of the content you create (the text, images, raw data, etc.) from the medium you view or present it in. Here's a diagram of such a system:

Imagine this weblog post is the semantic content. I'd use my entry editing tool (Movable Type in this case) to create the semantic post; I'd write this text and the diagram above and break the post down into its semantic parts (such as the "main points", "examples", etc.) Once I had that, I'd be able to tell the system to generate not only the HTML pages that normally make up the post, but a printable PDF, a Powerpoint presentation to show to my bosses, etc. based on stylesheets for each presentation medium I had previously defined. Having a system like this would bring us closer to the view of application-agnostic computer use that I've mentioned before, since we could use many types of editors to create the appropriate view of the content we've already defined.

Single sourcing has long been a promise of computing but has yet to be delivered in any satisfactory form. So I thought I'd describe the problem we're having where single sourcing might apply to help reason about what an adequate single source solution would need to do.

For my work on Usability and Software Architecture (U&SA) with Bonnie and Len, we have a number of "scenario packages" we are developing that contain, among other things, a description of a system usage scenario with architectural implications, a list of responsibilities the system must fulfill to implement this scenario, benefits to the user of including the scenario in the system, etc. These scenario packages are the cornerstones of our work.

There are a number of properties these packages have that would make them amenible to a single source solution. Here's a list of them:

1. We produce a number of publications about these scenarios. These publications span data formats (we produce Word documents and PDFs for academic journals, Powerpoint presentations for giving tutorials on our scenarios, I'm now trying to put together a U&SA website, etc.) as well as presentation formats (even publications that all accept Word document submissions all have different ideas of how the paper should be organized, what the font and spacing should be, etc.). We waste a lot of time fooling around with getting the scenario text, graphics, etc. out of one presentation format and into another.
2. The scenario packages change frequently. Since this is still an area of active research, we're still developing the notion of what makes up an architecturally-sensitive usability scenario package. And when we change our minds, we want all our future publications to include the updated scenarios and not the old stuff. We've also wasted a lot of time fooling around with doing manual "diffs" of old material to make sure its up to date with respect to our new understandings of the problem.
3. We don't want to be limited in what we can express by the particular presentation-oriented tool we're using. For example, most of the diagrams we're developing are currently done in Powerpoint, but Powerpoint is not a diagramming tool and it places limits on the ease with which we can create and modify these graphics. I've proposed using Omnigraffle instead, but Omnigraffle and Powerpoint don't always play well together. Plus, when I create a diagram in Omnigraffle it may be more detailed than what I want to show on a Powerpoint slide; I may want to be able to generate a stylized version of the diagram for presentation purposes.
4. We use different machines running different platforms (mostly Windows and Mac OS) to develop this material. This isn't directly solved by the single source paradigm, but by centralizing the content in a single location, we could more easily share our work, track versions, and attribute changes to who made them. This would help solve a frequently occuring problem where Bonnie changes her copy of our slides, I change mine, and then I have a messy and error-prone manual merge task to perform.

Micah is hopeful that Office 2003 will help solve some of these problems. I'm skeptical, but willing to reserve judgement. At this point I'm willing to see anyone take the next step forward, since this idea has been in gestation far too long without much real progress to show for it.

[ Permanent Location ]

Show Me All My Options

design, usability

April 27, 2003, 01:17 AM

Many interfaces give their users a great deal of control over the appearance and behavior of the application. Unfortunately, the means for exercising this control are usually hidden away in obscure preferences dialogs or even worse (horror among horrors!) configuration files written using some obscure, usually nonstandard syntax. To help fix this, I'd like to propose a new user-interface design principle for consideration. Here it is: "Keep configuration options close to the things they configure."

A classic example of this principle in action is the ubiquitous (and often overused) "Don't show me this again" dialog box. Here's one from IE/Mac:

This design is good because if the user doesn't want to be bothered by messages such as this one again, he can simply click the check box that is right there in front of him. He doesn't have to go figure out where in IE's voluminous configuration dialogs the relevant preference to disable these messages appears.

Here's another example from Movable Type's author's interface:

Note the option to change the default welcome message is clearly displayed right below the message itself; this makes it easy for the weblog author to 1) realize that the message can be changed and 2) click the link to go straight to the configuration screen where he can make the change.

You may object to this principle on the grounds that putting all the configuration options relevant to a given interface object right near the object would often clutter the screen horribly and likely confuse users who are new to the system and aren't ready to start reconfiguring it yet; they just want it to work. I have a couple of responses to that. First off, you may be able to design your configuration option so that it doesn't require any screen space; this is frequently possible with direct manipulation. For instance, I can reconfigure the order of my programs in the Mac OS X dock just by dragging them around:

If your configuration option is difficult to represent with direct manipulation, then we need another metaphor. One possibility is to extend the GUI to support the concept of standard "tools" which are always available regardless of the application the user is in. Think of an extension of the contextual help system in Windows, where you can click on a "?" button to turn the cursor into a question mark, then click on any interface element (theoretically) to get a nice little tooltip explaining what it does. That would be how a "help tool" would work. A "configuration tool" would instead provide you with the screen of configuration options when you clicked on that interface element to let you change its appearance or behavior.

These interfaces help solve the configuration problem because they provide the option right when the user needs it; when she has hit the point in her task where she wants the appearance or behavior of the application to be different than what it currently is. This integrates the "reconfigure the interface" task in with the user's primary task, rather than forcing it to become a secondary task that will likely be forgotten since it isn't a part of the user's main workflow, especially if it's difficult to figure out. If you're a cognitive walkthrough fan, you'll notice this helps to both ensure the user trys to achieve the "reconfigure the interface" affect and to ensure the user notices the relevant reconfiguration option is available.

[ Permanent Location ]

Bringing Usability to Open Source Software

processes & methodologies, software development, usability

April 23, 2003, 12:07 AM

I'm thinking of doing an independent study during the summer on the topic of "Improving the Usability of Open Source / Free Software". It is widely realized among open-source software users and the more level-headed developers that the usability of most open source software products is abysmal. Unfortunately, the response of many open-source advocates is to stick their heads in the ground, but even those who are seriously looking into the problem have yet to propose a sound human-centered open source development process that has a chance of actually getting used by J. Random Hacker (at least, no one has that I've heard of).

Basically, I'd like to write a "Usability HOWTO" that's based on some actual experience running a human-centered open source project. I have a few ideas to start with:

1. Developing and using scenarios and user personas as a way of encoding a shared perspective of the target user population among contributers to the project.
2. Using inspection methods like Heuristic Evaluations and Cognitive Walkthroughs periodically on the user interface and filing usability "bugs" in the project bug tracking tools.
3. PUI-style user testing where contributors go out into the world and test their interfaces on people who belong to the target user population of the project, then bring the results back to the other contributors to advise the interface design.

To flesh out and validate these ideas, I want to run a small open source project of my own. I was thinking of doing a server-side RSS News Aggregator since there don't seem to be many good ones in the open source world. But anything would do as long as usability is critical to the project.

I haven't decided for sure whether I'm going to do it or not; I'm not positive I have the right expertise. But I do think it's an important problem that needs to be solved, so I may start casting around for a faculty sponsor.

If anyone out there has any thoughts or suggestions, they'd be much appreciated as always.

[ Permanent Location ]

Quality Attributes and Human-Centered Design

software development, usability

April 22, 2003, 05:50 PM

In Software Engineering, we analyze and design systems based on known "software quality attributes", or general properties of software that define what makes a "quality" system. Some examples of these attributes include performance, reliability, security, maintainability, etc. The Software Engineering Institute has a technical report on the topic.

Usability is sometimes included as a quality attribute (though it's all-too-often omitted), but it is only one among many. Software engineers are busy people and when they have performance and modifiability and all those other attributes to worry about, usability often understandably falls by the wayside. So no one ever thinks to call in the HCI people.

However, I suspect that if we carefully examine all of these quality attributes, we'd find that human-centered design must underlie all of these concerns. After all, software systems are tools to help provide for human needs, right? It stands to reason, then, that we can't design quality systems without understanding what those needs are. So no discussion of quality attributes can truly begin until we uncover the human needs the system must support.

To help support this argument, I'd like to consolidate some real findings (either hard research results or best practices and industry folklore) for each quality attribute proving how it ultimately relates to human-centered design. For example:

• Performance - Anyone who knows anything about tuning systems for performance knows that having a quick response is much more important for some parts of the system than others. So what determines how fast a system operation has to be? It's how fast the user of the system expects that operation to be, of course. If the user expects an operation to take a long time, it doesn't matter if you can shave off a couple of minutes of execution time; the user won't notice. This is most apparent in problems like computer displays; humans can't perceive the difference between display refresh rates faster than 50 times a second. So if you're trying to tweak your algorithm so it can throw pixels on the screen faster than that, you're wasting your time.
• Security - Security people spend a lot of time coming up with newer, harder-to-break cryptography, more powerful firewall systems, etc. Yet the vast majority of systems are compromised because of "user error"; perhaps someone used a weak password, or a cracker used "social engineering" to fool an authorized user into giving them access to a secure system. Bruce Scheier is known for making this point.
• Reliability - When most software engineers think of reliability, they think of redundant components or extensively tested systems. Yet Jim Gray has found that the largest single cause of failures is operator error. No matter how many well-functioning, redundant components you have, if the guys who are supposed to plug them all together screw up, you're hosed.

If anyone has any other thoughts or leads, I'd love to hear of them. Some day I'd like to write an essay or a paper on the topic.

[ Permanent Location ]

Dreaming Up Mantras

usability

April 20, 2003, 10:08 PM

Kevin recently posted an interesting comment about usability mantras in the U.S. government. He specifically refers to the "three clicks or less" mantra of the OMB's government information consolidation project. His slant is that this mantra is meaningless from a usability perspective since finding information has much more to do with search interaction and results presentation than it does with the number of clicks, so "three clicks or less" is really more of a non-goal and may well lead the project down the wrong directions. And I think he's right.

The problem here as I see it is that the usability goals of the project aren't made clear (at least in the PR article). Why three clicks? Is the point that people should be able to find government information quickly? If so, then perhaps that should be the stated goal, and the designers should be given sufficient leeway in how to realize it. All too often I've seen people get bogged down by an oversimplified solution to a complex problem. This fixates their thinking into a framework that hides the true solutions from them, potentially causing huge wastes of time and money.

What's even worse from my perspective, however, is that the article doesn't mention how they came up with this consolidation idea in the first place. Did anyone do any studies of how people use government information services? What sorts of information do they look for? At what times and from where? What sorts of people look for what types of information? Simply consolidating the government's databases may not be the whole solution, or even part of it. The point is you just can't know until you go out and see what your users need. And it is shameful how rarely that gets done. I'm sure even a project with a "paltry" five million dollar budget could afford to run a few such studies. And I'm really becoming convinced that even a few can be much better than nothing.

[ Permanent Location ]

Stories as Pictures of the Users

processes & methodologies, usability

April 18, 2003, 01:45 PM

For my work on Usability and Software Architecture, Bonnie and Len want me to design and build a website to help disseminate our technique to professionals. In our meeting last Tuesday, Bonnie asked if I knew who the users are. I said, truthfully, that I didn't really. She replied, "Then you can't design a website."

She suggested I try writing up some personas instead of the more time intensive Contextual Inquiry process she'd taught us in Methods. So I asked Micah, who has read "The Inmates are Running the Asylum" to tell me a little bit about them. I decided Alan Cooper's full process was probably too heavyweight for this task, so I sat down to write up some portraits of our expected users based on my own understanding.

As I've thought about the high-level design of the site, I've expanded these profiles into slightly more extensive "stories of use" that describe the problems my stereotypical users will experience and what they're going to need to solve them. Of course, all I was really doing was recording and making explicit my own understanding of our users (although I've found this in and of itself is valuable); I had no evidence that my assumptions about their needs and problems were correct. So as a start, I've been asking a few of my colleagues who more or less fit the profiles to read over my scenarios and give me feedback on how realistic they seem based on their experiences. I've realized that I'm starting to follow my own Scenario-Based Design process, which makes me want to read Rosson & Carroll's "Usability Engineering" that's been sitting on my shelf since last September.

What's even more interesting about this exercise is that I find it's not just feeding into a website design, but it's giving me a better understanding of the needs of the people who will (hopefully) be using the U&SA technique. If I can sell this idea to Bonnie and Len, then I believe these profiles could be very helpful to us in evolving our understanding of our users and keeping us grounded in reality. If we want to make changes to our technique and one of us says "Wait a minute, Bob isn't going to like that change because..." then I think our discussions could become much more practical and user-focused.

I've really become sold on the power of stories as a means of driving design (and not just design of interfaces). When I first encountered scenario-based design, I'll admit, I thought it was a silly idea ("You expect people to take me seriously when I'm making up stories about people using our system??"). But since then I've seen the pragmatic value and unparalleled communicative power stories possess; people can share your vision of the system when you tell them a story in ways they can't when you just list functional requirements (or even show them screen mockups). Stories keep your thoughts concrete even as you envision that which does not yet exist. I'm beginning to believe that no design team can be effective without them.

[ Permanent Location ]

Tell Me Why

usability

April 16, 2003, 11:42 AM

I have a quick complaint to register about every desktop interface framework that I've ever used (which includes Windows, Mac OS X, Java's cross-platform Swing, and Linux/KDE).

I was recently reading a Word document created by one of my bosses, Len Bass. In this document, he uses a word that is specific to a particular domain (software security) and hence doesn't appear in Word's spelling dictionary. So I wanted to add it. I got the following screen:

Apparently I cannot add this word to the dictionary. I can tell this because the "Add" menu option is grayed out and when I mouse over the menu option, it does not highlight like all the other ones do. So far so good; the system is providing feedback to the user (me) that he cannot perform this action and is preventing him from selecting the erroneous option, which prevents slips and mistakes.

But why can't I add this word? Now there may be a very good reason; maybe I haven't selected an appropriate dictionary, maybe I don't have write access to the dictionary, maybe I'm in the wrong edit mode, or maybe Word has decided that no one would possibly want to add "nonrepudiation" so it has helpfully overridden my judgment to prevent this grevious user error. But here's the problem: I don't know what Word's reason is.

Because I don't know why I can't add this word, I have no way of knowing how I can correct my error. And this is a problem that runs through most every option deselection mechanism I've encountered; there is no toolkit support for the developer to provide this important feedback to the user. The developer can say "sorry, you can't do this" but he can't say why.

Toolkits should provide this support, and developers should use it. Providing a simple tooltip that gave this feedback would be enough, and would save users like me a lot of headaches.

[ Permanent Location ]

CHI Report, An Emotional Closing

aesthetics, design, usability

April 12, 2003, 06:53 PM

The final activity of the conference was the Closing Plenary, which was delivered by Don Norman on Emotion & Design. Don's speech was definitely one of the examples of a good use of the presentation format, which I briefly discussed earlier.

Don's plenary had three main points:

• Emotions are an important part of the user experience, inseparable from cognition, and are all too often ignored by usability professionals to the detriment of the profession.
• Science has begun to unveil the inner workings of emotion. We can confidently separate emotion into three main levels: the visceral level where we experience instinctive emotional reactions, the behavioral level where we act on our emotions, and the reflective level where our values, rationalizations, and emotional influences reside.
• Usability professionals should learn more about emotion and its relation to product design. If we continue to ignore emotions, we will continue to become irrelevant insofar as we are not directly affecting the desirability and emotional attractiveness of products that is so important in customer buying decisions.

Don illustrated these points with several examples of good product design. He admitted that people have different emotional reactions to these products and that designing for emotions are subjective. His response was to "deal with it". There is no reason to believe that products can be designed for everyone; often the necessary response is to design solely for a given target market.

The speech served to deliver several excellent points and an encouraging call to action. I've begun to see the seeds sown here at CMU for a more thorough understanding of this important new area; even within the MHCI program there are some who are interested in emotion and design. Kerry, for instance, is interested in fashion design for wearable computers, which definitely ties in to the emotion and self-image themes Don discussed. And I already mentioned several promising examples of work I heard of in the earlier panel session.

I'm looking forward to hearing more from this field, and maybe even having the chance to contribute.

[ Permanent Location ]

CHI Report, Recommender Systems

design, information, internet, society & sociology, usability

April 12, 2003, 04:40 PM

After lunch, I went with Mathilde to a short talk session on Recommender Systems. Three out of the four talks were interesting, so I considered it a pretty successful session.

The first talk was on how the ratings in systems like Amazon.com's customer-supplied product rating system can influence future customers ratings and lead to inaccurately high or low ratings for some products. The speaker studied a system called MovieLens, which looks similiar to an Amazon.com that only does theater movies. He found that around 8% of the time people would raise or lower their ratings based on the rating that was already present. Thus if users would rate a movie as 2 stars, but see a rating of 4 stars on MovieLens, they might adjust their rating, consciously or unconsciously, up to 3 stars. Though these individual effects may be slight, the inaccurracies may spread.

Another interesting finding he reported was that users could tell if a system had wildly inaccurate ratings, and that this would effect the users' opinions of the system. So if users found good movies rated low or bad movies rated high consistently, they reported a much lower opinion of the system as a whole. Unfortunately, he did only study a fairly extreme case of inaccuracy so it is difficult to say where the relevant threshhold value is.

The next speaker, David McDonald from the University of Washington School of Information, discussed a recommender system for group-ware tasks such as finding the relevant expert in a company who may be able to answer a given question. What made his talk particularly interesting was that he was using social networks to build such a system for a medical software company. Here are my main takeaway points:

• David built his social network through ethnographic observations. He discouraged the use of automated tools, such as tools that examine email sending patterns, since he felt these tools do not accurately approximate the real social network but instead are byproducts of the medium (a secretary may send lots of announcement emails to people she is not socially close to).
• He ran into a couple problems where one person was socially close to another even though few people knew it; they were buddies outside of work. Thus their connection didn't come out through his initial observations. Many users felt the tool should reflect their own personal social network rather than the socially perceived network of the entire workplace.
• Social networks are dynamic beasts, so the network had to adapt to changing social environments over time.
• The interface needed to provide a way to "escalate" the results, or expand and contract the number of recommended experts shown. Users reported that they often preferred many choices rather than a single choice, even if the single choice was in fact the best person to answer their question. They also wanted a good deal of control over the direction of the escalation.

The final interesting speaker was Thomas Erickson from IBM T.J. Watson Research Center. He discussed visualizations of social activity, such as those produced by the Sociable Media Group at the MIT Media Lab (which I applied to, by the way, and never heard back from...). He had a few interestiing things to say:

• There are two purposes of social visualizations: one is for an individual's information, the second is to portray the social state of a group back to the whole group. The two should not be confounded, and frequently shouldn't be combined in one interface.
• A completely accurate visualization of a social situation may be bad; many situations depend on face saving, "first impressions", etc. The little white lies that keep our social lives going...
• Ambiguity is often useful in these visualizations depending on the user's information needs. Large differences can be exaggerated, whereas runs of similar-looking data may be compressed. This is probably true of most visualizations

One side comment I thought of while watching these presentations: talks such as these are useful for making a few small, well-argued points that are interesting enough to encourage your audience to look into the area in more depth (or just take away your main points). The speakers who tried to go into great detail and make complicated arguments quickly lost the audience, whereas those who had simple points that powerfully portrayed their message were more effective. More on this when I get to Norman's closing plenary.

[ Permanent Location ]

CHI Report, The Everlasting Twilight of the Idols

personal, usability

April 11, 2003, 08:02 PM

This entry is private. Forgot the password? Ask the Keymaster!

[ Permanent Location ]

CHI Report, Emotion and Design

aesthetics, design, usability

April 11, 2003, 08:01 PM

When I got in I went to a panel on Emotions and Design chaired by Jodi Forlizzi, a HCI and Design professor from CMU. Don Norman also showed up, since his new interest is in emotions and design. He opened the panel by discussing how emotions are tightly coupled to cognitive mental activities; he brought up a study that products that are more aesthetically pleasing are actually easier for people to use, even when all other factors are constant. More on Don later.

A number of interesting topics came up that I'd like to pursue in more detail. Here's a few of them:

• Gibson's theory of affordances which is intended to unify all aspects of the user, including cognitive and emotional responses.
• Jodi mentioned how emotions can both heighten the experience of using a product and get in the way of its effective use. How do we ensure we get the former and not the latter?
• Sometimes different emotions are coupled in ways you might not expect; for instance, disgust is frequently coupled with fascination, which explains why many people will linger so long over a disgusting scene, and talk about it at length afterwards.
• HCI as a field has a much higher percentage of women than many other technology disciplines. Someone asked if this accounted for the current attention to emotion. The panelists seemed to feel this was quite possible although emphasized that we shouldn't overgeneralize that "men are bad at emoting, women are good at it".
• I also recall a discussion at the Weblogs SIG about expressing emotions in weblog posts. I feel like these two areas relate, although I'm not sure how just yet.
• We have a tendency to anthropomorphize our products. One person wondered if we should design for this, the conclusion was that it could be good if it was done right. Little guidance was offered on what "right" meant though.
• Kees recommended educating designers in understanding and "getting in touch" with their emotions. I wondered, however, how easy this would be, since western culture in general tends to discourage display of many emotions and considers them "irrational". I was pleasantly surprised to find that Norman agrees.

[ Permanent Location ]

CHI Report, Awakening

personal, usability

April 11, 2003, 08:00 PM

My fourth day in Fort Lauderdale marked the last day of CHI 2003. I overslept and missed the first session, so I went to breakfast with Andrew instead. He is one of the winning Interactionary stars and quite possibly one of the greatest guys I've ever met. We talked about guy stuff: job hunting, relationships, etc. One of the things I like best about Andrew is that I find him so easy to talk to; he listens well and affords discussion even about subjects I wouldn't bring up with anyone else.

After breakfast I headed over to the conference center. On the bus I talked to a guy from Sprint's usability research team about his experiences as an HCI researcher in the industry as well as a little bit about U&SA (turns out he works with a guy who used to work with Bonnie and Len back when they first published the technical report). Hearing about his experiences reconfirmed my belief that constantly showing real business value (by contributing to a productive team, quantifiably saving the company money, etc.) is the most important activity for an industry usability team. He also mentioned that Sprint was working on a new process (called PACE) which was partially intended to help bring the usability people into the development process so they could have more real influence. Sprint, he said, was currently committed to delivering a quality product since they had recently released two market failures, so usability is prioritized above time to market. When time to market is most important, which is all too often, the usability people are often seen as actually holding back the release and are frequently dropped.

[ Permanent Location ]

CHI Report, The Interactionary

design, personal, usability

April 10, 2003, 11:45 PM

Finally, I went to the Interactionary competition. Six people from the MHCI tribe were competing in a CMU team: Kelli, Andrew, Abby, Neema, Kevin Lee, and Kevin Fox. I swear I almost had a heart attack while sitting in the audience waiting for them to come up. After they'd finished Kenneth asked me the time; when I showed him my watch he said "my God, Rob, you're shaking!" I was incredibly nervous for them, and I didn't even have to go on stage.

But it all turned out for the best, since they won! We're all very proud of them; Bonnie even took us all out and bought the Interactionary team drinks at Hops. A great and well-deserved ending to a journey that took a lot of time and effort on all of their parts.

After we got back from Hops, we checked email at the conference center, then left for the CMU reception at the Marriott. Turned out there was no free beer at the CMU reception, so we crashed the Microsoft reception across the hall. I talked to several people at both receptions, which got exhausting for an introvert like me after awhile. Abby was getting tired too, so she and Neema and Matt and I left the party and went back to the hotel. We then went out for a night swim in the ocean. There was a storm going on far away; lightning flashed above the dark waters in the distance. It was a surreal and beautiful experience.

[ Permanent Location ]

CHI Report, Lunch and Games

life & times, usability

April 10, 2003, 11:44 PM

For the next session, I tried attending a session on e-Learning, but the points being made came across as fairly unsophisticated, quite a bit of rhetoric to back up very little insight. If the field is really at this level of maturity, then I can see why current e-Learning systems are not very good. Besides, I was starving since it was past lunch and I hadn't had breakfast.

After an overpriced sub from the Subway cart I went to an informal SIG on video games put on by a couple of guys from Microsoft. It was interesting although I didn't have much to contribute since I know so little about the field. One guy mentioned he was working on developing design patterns for video games, which sounds like an interesting idea. One of the organizers mentioned there was going to be a special games track at CHI 2004; I look forward to checking it out since there seems to be lots of interesting HCI research going on in this field.

[ Permanent Location ]

CHI Report, Magic Numbers and Cranky Usability Consultants

processes & methodologies, usability

April 10, 2003, 11:43 PM

The past two days were packed, hence the reason this weblog entry is appearing a day late. Yesterday I got up early enough to make the morning session, so I went to a panel on "The Magic Number 5: Is It Enough for Web Testing?". The basic premise of the panel rested on Nielsen's Alert Box assertion a few years back that 5 users is enough to test a single interface; after testing five users you should iterate and test again or you are just wasting your time since you'll start finding mostly the same problems over and over again. Jared Spool, Gilbert Cockburn, and Rolf Molich (Nielsen was supposed to attend, but he was ill and did not come to CHI this year) disagreed with this claim on the grounds that the number of users you need to test to find a significant number of critical errors varies with the complexity of the interface and the goals of your study and is usually much higher than five. Carol Barnum agreed with the number if the discount usability engineering method is followed, and Dennis Wixon from Microsoft was the black sheep who argued that the entire preoccupation with finding "all the errors" was stupid and everyone should use the RITE method.

The debate was spirited and enjoyable to watch, but I got the feeling afterward that the panelists were all shouting loudly at each other in agreement. Everyone seemed to believe that you needed to do highly iterative development to ensure problems found actually got fixed and retested (although Rolf called for a more robust discipline that could get the interface close to right the first time so we wouldn't have to do much user testing; I think Jared's sarcastic comment that "Yes, we can solve this problem right now by always building usable interfaces!" summed up the problems with this approach fairly neatly). Everyone seemed to believe that ultimately you should run as many users as you can to ensure many iterations and a more usable product.

Ultimately, I came away from the talk believing all the more strongly in iterative development while still holding out hope that we can design interfaces to be as initially usable as possible (through CI/CD, etc). I'm also even more convinced of the potential for RITE.

[ Permanent Location ]

CHI Report, Service Learning in HCI

charity, teaching & learning, usability

April 09, 2003, 06:49 PM

After lunch, I attended an Informal SIG (Special Interest Group) on HCI Service Learning. There were only three other people there (their names were Jonathan, Carol, and Anne), so it was a nice, small discussion group. We talked about experiences in teaching service learning; I discussed some of our findings in our TCinC studies. I would have liked to give a full presentation on our results and redesign ideas so the other members in the group could comment, but I had no suitable presentation prepared and didn't want to dominate the discussion. One interesting problem everyone else was running into that we don't seem to have is that students were having difficulty implementing the project designs they came up with. Our problem has been the opposite; students have no problems understanding the technology but are lost when it comes to understanding the organization.

Jonathan did ask how we could spread the concept of service learning to other academic institutions. I mentioned we were working on reusable lesson plans that should help integrate service learning into the curriculum, but that we were more concerned with solving our own problem at the moment and hadn't yet given a thought to everyone else's. We exchanged business cards at the end; I plan to bring them back to Joe and Matt to hopefully start a dialog on how to spread this stuff.

[ Permanent Location ]

CHI Report, Privacy, Security, and Trust

design, internet, usability

April 09, 2003, 06:48 PM

This marks the end of my second day here at CHI 2003. So many things happened; a thick swarm of ideas buzzed around my head, flew into my ears, nose, and mouth, got into my eyes and made them all watery, and generally became very annoying until I realized the analogy had gone too far and cut it off.

The first session I went to was on Privacy, Security, and Trust. Most of the talks didn't particularly interest me. One person had done a study of an online community in a small Massachusetts town and concluded that when you ensure people are using their real names, it tends to raise trust and civility while lowering people's willingness to disagree about controversial topics. Another woman had done a study on people's level of distraction while using cell phones, she found people tend to pay less attention to traffic lights while talking on their cell phones. Fairly common-sense stuff.

The one presentation I really liked was one on "Prominence-Interpretation Online". The doctoral student giving the talk, B.J. Fogg, argued that the credibility of a website was a function of two properties of the experience: the prominence of information (she must notice the information) and interpretation (she must make a judgment about the information). What's more, he did a study that found that the top factor people used to interpret information related to the design look and feel of the site, which was almost doubly more important than information design and structure. Which is disconcerting for those who expect that consumers in a market will make informed, researched decisions when assessing credibility. When asked whether we can design our websites to encourage a more robust credibility assessment, Fogg said he thought the only way to improve this situation was through educating people on their limitations and, in the meantime, make sure you have a nice site design.

[ Permanent Location ]

CHI Report, Day 1

processes & methodologies, software development, teaching & learning, usability

April 08, 2003, 09:58 AM

Yesterday was the first day of the Computer-Human Interaction conference at Fort Lauderdale, Florida, which is where I'm currently located. Not a whole lot of academics yesterday; after I got in I mostly hung out on the beach and in the pool with the gang. We all remarked that so far this was the best conference any of us had ever been to :).

In the evening hours I went to the U&SA tutorial that Bonnie and Len and I put together. It was interesting to see the reaction of professionals to our material; so far I had only seen the tutorial presented to students. All in all I think it went well, although Len was concerned that people didn't seem as excited about the material as they were last year. The tutorial attendee's comments also confirmed some thoughts I'd had on the major problems with the tutorial:

1. There was some confusion on what the distinction was between the scenarios and responsibilities (which are supposed to apply to all systems) and the sample architectural pattern (which may or may not apply to any given system).
2. Before Bonnie gave an in-depth explanation about our Benefits/Tactics matrix, some people didn't understand what process we were advocating for applying these scenarios (should every system implement every scenario, or do you need to decide which ones make sense for your system?)
3. One person raised an issue about reusable code support for the scenarios, which bears out my thought that people may like pointers to toolkits and frameworks that help implement the scenarios.
4. Someone mentioned that we should look at existing patterns in the Gang of Four's book, etc. to see if there are more general solutions to our patterns. I'm kind of kicking myself here, because I've actually already done some preliminary investigations in that area and neglected to mention it. I had a chance to look smart in front of people in my field and blew it. But it is good to have confirmation that this is a good path to go down, although I haven't found many patterns that directly apply to our scenarios and am beginning to think the right way to do this would be to investigate existing solutions in working architectures and generalize from them.
5. Someone was concerned with how we come up with the scenarios and how we distinguish between an "architecturally-sensitive usability scenario" and a "functional requirement". I have a simple criteria that we could use: has the scenario caused a "We can't change THAT!!!" situation?
6. Someone also raised the issue of how we could test to ensure these scenarios are implemented in an architecture. This I actually hadn't thought of, but certainly appears important.

In other news, I went to the networking event after the tutorial. Talked to several people, many of whom I already knew, but some new faces too. Then several of us went out to eat around midnight. I'm having a lot of fun; it's great to be here in Florida with beautiful weather at a top-notch conference in my field. And the best part is I'm here with some of the best people I've ever known. What could be finer?

[ Permanent Location ]

Interoperability and a Level Playing Field

design, software development, usability

April 02, 2003, 07:25 PM

I went to the HCII Seminar Series talk today given by Roger Dannenberg, a CMU professor who is a musician and computer scientist. He talked about a real-time audio processing system he developed to assist musicians in producing new electronic audio devices, and he also spent a little time at the beginning discussing the problems and challenges with building systems to assist artists.

One point he made was that artists like to plug together things that weren't originally designed to work with each other (he gave an example of a robot/cello that played itself and a flying car/airplane). This can be a problem for software designers, because this means that individual frameworks and applications are unlikely to be able to serve all of the needs of artists. Applications are usually designed to solve the needs of their users by anticipating what sorts of tasks the user will perform while using the application (in usability engineering, getting the user tasks right is one of the most critical stages of the process). But for artists, you can't anticipate what tasks they're going to want to perform with the application; after all that is part of the artistry. What's a designer to do?

To me, this just reinforces the need for more componentized applications that can interoperate with one another, or "play nice together". I should be able to take my document out of Word and bring it into Photoshop to add some decorations to the pages, then schlurp it into an audio program to add voice notes on why I chose the wording I did. Or even better, get rid of these clunky applications and let me just work with my document, letting all that switching programs stuff go on behind the scenes. I want to be able to record my thoughts in documents, not Word .doc files. I don't want what I create to be limited by the one program I happened to first use to create it.

Others have said this many times before, and usually one of two answers are given:

1. It's too hard.
2. People don't want to use multiple applications; they just want one program that does what they want.

As for point 1, I'll agree that it's hard. But the hard problems are the fun ones, right? We shouldn't shirk away from something just because it's hard; not at least if we all agree that it is nonetheless worth doing.

As for point 2, I don't buy this at all. Sure people don't want to have to hassle with converting their data from one format to another (and dealing with the ensuing problems that causes); that's a program-centric problem and has nothing to do with their goals. But people do get frustrated when they want to do something and they find their application can't do it (or they can't figure out how to make it do it); we UI designers see this all the time. People use one application because it's the path of least resistance. It's the easiest way to get their job done, even if it won't always produce the best results. And they do this because we as technologists have failed, and we've failed because we can't figure out how to make our tools work together so that we don't have to guess at our user's needs.

Roger talked about how much many artists have to struggle with technology to get it to do what they envision. If you aren't a computer expert, the learning curve is generally too high to allow you to try out new things with technology, new directions that computing can go in. And I wonder how many great ideas we're losing each day because of this.

[ Permanent Location ]

© 2003-2010 Rob Adams ()

Last Modified: Wednesday September 17, 2008

Unless otherwise indicated, all original work appearing on this web page is licensed under the Attribution / No Derivitive Works Creative Commons license. Please before using any page content in a manner not covered by this license.