processes & methodologies

About

Syndication

Categories

Archives

Scheduling Tasks Collaboratively

processes & methodologies

May 29, 2004, 09:16 PM

Our capstone project has entered its second phase, where we're doing iterative testing and redesign work. The biggest change is that we're all working on the project full-time now, rather than around 12 hours a week as one class among many. The best part about this is that Bob got each project team it's own shared office space (ours has a window! Yay!). This constant proximity has proven extremely helpful for our team so far, and has allowed us to try out some collaboration techniques we weren't able to do before.

One problem we suffered from last semester was group task management. People would commit to tasks but wouldn't always follow through with them, not out of laziness but just because they'd fall by the wayside and nobody was really keeping track. People weren't always terribly aware of what everyone else was doing (including me, the project manager). This cost us a lot of time in meetings coordinating with each other. I kept a detailed schedule for a while, but gave up since it kept falling out of date and nobody ever really looked at it anyway.

Once we got our summer space, I had an idea for how to improve matters. I bought a roll of butcher's paper, some Post-It notes, and markers, drafted a much larger version of my linear schedule format, and posted it up on the wall. Here's some photos of the entire schedule and a close-up of the first couple of weeks:

Whenever someone takes on a task, they write it on a Post-It and stick it up on the date they think they'll complete it by. Project milestones and scheduled events such as client meetings and user tests are also put up on the board by the person who was in charge of scheduling them. During our morning check-in meeting, we go through the tasks that were scheduled for completion that day, check off those that actually were completed, and move those that weren't to a new deadline.

We use different colored Post-Its to represent milestones, events, and tasks assigned to different people. Some immutable events (such as team member absences) are written directly on the board itself.

This approach to group schedule management has several advantages:

1. Everyone can always see the schedule. Because it's up on the wall staring at the entire team, anyone can just glance up and see both what tasks they've currently committed to as well as what tasks everyone else is working on. This is a huge benefit when many of your tasks depend on deliverables other teammates are currently working on (an extremely common scenario in interactive design projects).
2. Team members must commit to deadlines. Because there is no space on the board that isn't situated in time, there are no "floating" tasks that someone is working on but hasn't committed to a deadline or deliverable. As soon as a task is defined and assigned, the assignee must make at least a rough estimate of when he or she expects to complete it.
3. Schedule slips and dropped tasks become explicit decisions. Post-Its can be moved or removed, but the team member responsible for the task must consciously make that decision, probably in front of his or her teammates. Though the mobility of Post-Its might encourage overly flexible deadlines, I believe this is necessary for the schedule to stay in sync with reality (and there are other mechanisms for preventing schedule slip, such as the next one).
4. The amount of time remaining before deadlines is immediately visually obvious. The project milestones and completion date (where the timeline itself ends) are plainly visible for all to see. Because days all take up the same amount of space, team members can quickly see how much time is left before the next approaching deadline and experience the appropriate emotional response.
5. The team can get a rough sense of who is overcomitted and who has some time to spare. This is possibly the least useful feature, but because the tasks are all visible, the team can get a rough idea of everyone's level of comittment, past and present. Simply counting Post-Its won't work, of course, since some tasks may eat up only a little bit of time now and then whereas others may suck up the entire day. But the board at least provides a starting point for a conversation.
6. Everyone owns the schedule. If only one person is charged with maintaining the schedule (usually this is the project manager's responsibility), then the rest of the team is encouraged to concern themselves only with their immediate tasks and not take a big picture view of the project. Tracking everyone's individual tasks can get quite tedious for the project manager while also giving everyone else the impression that he's micromanaging. Having the schedule accessible for viewing and updating by everyone encourages collaborative ownership of the long-term direction of the project (provided the board is truly owned by the team and not just a tool for the project manager to keep an eye on everyone). A schedule maintainer is still needed, of course, but now he or she must simply be there to make sure the board stays up-to-date and to direct the team's attention to long-term trends and future milestones.

The core benefit, of course, is much the same as that provided by affinity diagrams, that is it takes abstract concepts (time and tasks) and makes them visible and physical.

There are some drawbacks, as well as situations in which this approach might not work so well:

1. The board requires a fair amount of permanent wall space. Teams that don't have dedicated project space (ideally shared by all members in a radical colocation style setup) can't use this approach. Even teams that do might have trouble finding enough space, especially if the board is introduced into an existing project room. Fortunately, we were setting up our space at the time that I had the idea, so this wasn't an issue, but if the team already has posters or whiteboards or whatever covering the wall, there might be pushback.
2. The board may not scale to long-running projects. In the same vein as the previous point, long-running projects may require absurdly large boards. The one you see pictured above is for 12 weeks, and it wraps around to two walls. Projects that last a year or more can't put their whole timeline up, but perhaps it would be sufficient to stick up only the amount of time between project milestones, or enough for a three-month span. Longer-term planning can be done through other means; most teams are probably more concerned with what they're doing in the next three or four weeks than in the next four to six months anyway. However, this compromise might lose some of the long-term view of approaching deadlines that our board enjoys.
3. The board may not scale to large numbers of people. We have five people in our team. If we had many more, the board might get cluttered with Post-Its and it might get harder to read and manage. Then again, if you have a team with many more people things will get unwieldy no matter what you do. But I could reasonably see a team with around 12 people wanting to use this, and that may be more than it can support.

Maybe I'll write a paper on this topic. I'll call it "A Cost-Effective Large Display for Collaborative Scheduling and Ambient Task Awareness using a Tangible Interface". I think it could totally get into CHI.

[ Permanent Location ]

Life Lessons for Designers

design, processes & methodologies

March 26, 2004, 10:16 AM

Found via Dan, this article titled "The Top Ten Things They Never Taught Me in Design School" is interesting reading for those of us who are starting careers in one of the subdisciplines of design (or hoping to, anyway). Some of these ring true for me already, partially due to my (admittedly marginal) professional experience but also because of my experiences here at CMU (so some of them can be taught in design school, albeit indirectly). For example, I'll buy the "95% of any creative profession is shit work", especially after playing project manager for our capstone project. And this is important to come to terms with, because you probably won't be too happy or successful as a designer if you can't learn to take the bad (think convincing management that your design is important, writing up user testing reports, fighting with developers over technical compromises, etc) with the good (sitting down to do some creative design work).

Others I can certainly see as important, but I recognize that I need to get better at, like prioritizing and not over-thinking problems. And a few sound pretty familiar from my experience as a software developer, like "Start with what you know; then remove the unknowns".

It's good to remember, every now and again, that things work differently in the real world than they do in the ivory tower of academia.

[ Permanent Location ]

Ready, Fire, Aim: Parallelizing Design and User Research

design, processes & methodologies, usability

February 18, 2004, 11:48 PM

For our capstone project, we're reaching the point in our process where we agreed to begin thinking about design. As I mentioned earlier, we want to overlap our design and research efforts to avoid getting caught up in wasting lots of time doing up-front research that doesn't turn out to be useful for our design efforts.

Bob calls this approach "Ready, Fire, Aim". The basic idea is to spend some time preparing yourselves by exploring the domain, reviewing literature, talking to domain experts, etc., but then quickly pick a focus and start down a design direction. These early designs may turn out to be totally off-base, but at least they will serve to point out the holes in the team's understanding of the users and suggest fruitful directions for additional research.

I remember a similar concept appearing in programming practice. The Pragmatic Programmers recommend an approach to architecture design they call "tracer bullets". The idea is to attempt to construct a functional end-to-end skeleton system to serve as an initial prototype of the proposed architectural design. Although you may throw this away, you'll learn where the flaws in your understanding of the problem domain lie so that you can design a better architecture and fire your next tracer.

The challenge we'll face, I expect, lies in keeping a clear vision of the team's current understanding of the users and their needs. This is likely to change with the designs as we uncover holes in our knowledge and fill them in. Our plan at the moment is to encode this understanding in our personas and task lists, hopefully using these as living documents as the design process progresses. Perhaps this will be sufficient, perhaps not. It will be interesting to see what this process produces and whether it will be successful.

[ Permanent Location ]

The Art of Being Critical: How to Give Design Critiques

design, processes & methodologies

February 16, 2004, 08:13 PM

One skill that is very important for a designer of any stripe to have, but which I've found is rarely explicitly taught, is how to give and receive effective criticism in a design critique. As a result, I find when someone asks me for feedback on a design I often don't know quite what to say, and when I ask others for feedback sometimes that feedback isn't as helpful as it could be. This can be a big problem, since often this form of feedback is all a designer gets, especially on those many projects that aren't important enough to warrent user testing. So to start to address this problem, I'm outlining some thoughts on what should and should not go into an effective piece of design criticism, whether this criticism is given during a formal crit session or in response to a friend asking "whaddya think of my poster?" over lunch the day before an assignment is due.

First off, it's important to remember to check your ego at the door, to criticise the work and not the person, blah blah blah. We all know that already (even if we don't always live it). What else is there to say?

When offering criticism, I'd first ask "what is my initial reaction to the work?" How does it make me feel? Confident? Playful? Annoyed? Confused? Where does my eye go first? What information is most prominent? What do I think the designer was trying to accomplish with this work? Anyone, even those with no design experience, can give useful feedback along these lines. And sometimes its these first impressions that are most helpful.

Next, make sure you're clear on the designer's goals with respect to the work. Don't guess; ask him. It's easy to suggest changes that aren't appropriate not because they aren't great ideas for some purpose, but because they aren't so hot for the intended purpose. Design is an inherently normative act, and thus it is always performed in service to some set of goals. You might disagree with the author's goals, but then at least you're clear on what it is you need to argue about. Alternatively, the designer might not be clear on what his goals are. This is a problem. Maybe you can help him figure that out before proceeding.

Once you're clear on the goals, you can start to apply that design expertise by comparing and contrasting the design to a set of principles and best practices, as well as an understanding of the needs of the intended audience. It's best to describe both problems and good aspects, so that the designer understands not only what needs to change but also what he must avoid damaging with changes. However, don't fall into the trap of applying principles without proper justification. All principles have reasons behind them, and those reasons don't always apply. I once knew a guy who insisted that "you should never provide help unless the user asks for help". A good principle, probably born out of the Clippy fiasco, but he believed it so religiously that many of his interface designs would fail to provide sufficient feedback for users to evaluate their actions under the theory that it was "providing help the user didn't ask for" (some screens he left entirely blank, with no explanation of their lack of content). Novice designers often apply their shiny new design principles haphazardly, and as a result may do a worse job than they would have were they ignorant.

Try to understand design tradeoffs; make a note of parts that could be improved but also say whether you believe the problems are serious or just small refinements. All design must be signed off on at some point, and its useful for the designer to know whether you think his solution is pretty good as it is or needs much more work.

Finally, don't get upset if the designer challenges your suggestions. He might be too in love with his work to take criticism, yes, but he might also just be trying to better understand your concerns so he can decide how best to improve his work. Give him the benefit of the doubt and assume it's the latter.

When receiving criticism, it's best to be open but inquisitive. Listen to all design suggestions but don't always take them at face value; you may need to probe to get at the reasons for the concern. It can also be good to get a second independent opinion if you're not convinced; see if both your reviewers come up with the same issues independently but if they don't, ask them about it specifically. Try to avoid biasing them one way or the other.

Perhaps after waiting for initial impressions, make sure you've made your goals clear. Your reviewers can't give useful feedback if they don't understand what you're trying to do.

Finally, remember that critiques are analytical analysis methods, not empirical ones. You must judge critique comments, both as a designer and a reviewer, by the quality of their arguments, and to do so you must first understand the argument. Both parties must be able to divorce themselves from the context in order to do so, but it takes a present mind as well as an absent ego to actually affect useful design change. Remember that pushing pixels around on the page doesn't always equal progress.

Good critiques can transform bad designs into excellent ones, but they aren't trivial to run. It's a skill that takes practice, just like anything else.

Scott Berkun, a man who is probably far more qualified to pontificate on this subject than I, wrote an article about how to run a design critique that may be worth checking out as well.

[ Permanent Location ]

Camera Studies

design, processes & methodologies

February 05, 2004, 11:15 PM

Sonia Wendorf, the resident industrial designer on our capstone project team, has introduced us to the concept of "camera studies" and is currently designing one for our own data collection purposes.

A camera study is a user research technique where the research team gives participants in the target user group disposable cameras and asks them to photograph important events, objects, and places they encounter during their day, as well as possibly writing down quick notes describing what they're doing and why they took that particular photograph. Alternately, the researchers may just ask the participants to take one photograph about every hour to collect more time-based data. The researchers then develop the film and review the pictures to gain an understanding of what goes on during an average day in the life of their users. If necessary, they may do an artifact walkthrough with the users after viewing the photographs to gain a more in-depth understanding.

The purpose of the study is to gain a visual understanding of the user's activities and habits during their average day so that the design team can get a sense of their lifestyle. And because the researcher need not actually accompany the users during the data collection process, this technique can glean information from a broader range of users than, say, ethnographic observations can. One could see the results of such an inquiry feeding quite nicely into a set of realistic personas for a project.

The advantage of this approach is that it helps the design team see the world through the participant's eyes rather than just their own; the participant is filtering the information by choosing what to photograph. Furthermore, since the technique allows for running many users in parallel, the design team can get data on many more users which allows for richer data with greater variety. The downside is that the data collected won't be as richly detailed as the data an experienced ethnographer could collect during an extended observation session. So although the technique may be quite helpful for getting an overall impression of a participant's lifestyle, it's less useful for, say, understanding how that participant fits into the larger corporate organization or how the software they use impacts their job performance.

Anyway, it's an interesting technique to add to the toolbox. As an aside, Dan has a post up on a few more from Maya Design.

[ Permanent Location ]

Defining SMART Objectives

processes & methodologies

January 22, 2004, 10:18 PM

More wisdom from Matt on project management, this time on the topic of defining good objectives.

First off, Objectives need to be distinguished from Mission Statements or other expressions of general direction. When a team or organization expresses their mission, this is an important tool for maintaining a joint understanding of what the group is trying to accomplish by staying together and how every individual fits into the larger group purpose. Because the purpose of Mission Statements is to provide direction, there's not necessarily a requirement that the mission ever actually be accomplished.

Objectives are different. Objectives do need to be accomplished if a project is to be considered a success. The purpose of objectives is to provide a definition of that success and to provide it up front, so that all project team members understand what the group needs to do. Good objectives have five important properties. To remember them, PMs say that good objectives need to be SMART:

1. Specific—Objectives need to describe particular actions and results in quantifiable terms. For example, let's imagine a widget factory manager who is given the very vague objective from higher up to "increase production". To make this objective more specific, she might change it to "increase widget output by 3%".
2. Measurable—This means that each objective must have some sort of mechanism in place to check the extent to which it is getting achieved. If our factory manager doesn't already have something in place to count the number of widgets produced, then she'd better find something to do so.
3. Achievable—The objective needs to be possible to accomplish with a reasonable time and energy investment from all team members. This may seem obvious, but there are plenty of times that impossible objectives get set, often because the manager who set them didn't know enough to realize they were impossible and didn't consult with anyone who did. Our factory manager may wish to meet with workers and their supervisors to discover whether the 3% output increase is reasonable or not before she imposes it.
4. Relevant—The objective must be meaningful to all team members and it must be something they are able to influence. Presumably the factory workers under our manager are able to influence the widget output (although setting individual objectives, like 5 more widgets a day per person, might be even more relevant) but had our manager asked them to "increase profit margins" then she would have been greeted by blank stares. Relevance is, of course, determined by the people the objective is being set for. In a way, it speaks to objectives being user-centered.
5. Time-bound—The objective needs a deadline, or no one's going to ever get around to accomplishing it. Our factory manager should change her objective to "increase widget production by 3% in two months" to make it conform to this property.

This article describes each property in a little more detail for the curious.

Though these properties are simple in concept, writing effective objectives is harder than it appears. Make sure you understand the purpose of each property, and don't overlook any. After all, you don't want to wind up with objectives that are SMAT or SMRT.

[ Permanent Location ]

Setting the Agenda

processes & methodologies

January 19, 2004, 11:16 PM

Awhile back, my friend Matt taught me how to write a good meeting agenda. It's a simple skill to learn, but highly important for running effective meetings. Agendas are often skipped by novice facilitators, which all too often leads to wandering, ineffective meetings that waste everyone's time.

There are three important components to any agenda item:

1. The item description, which gives a one-line summary of what's going to happen.
2. The person responsible for the item, so that it's clear who will be facilitating that part of the meeting, and thus must come prepared to do so.
3. The type of the item, which is described in more detail below.

It may also help to allocate a time range for each agenda item. For formal meetings, these times are generally present and strictly adhered to. For less formal meetings, however, they can still be helpful as process checks to ensure an unimportant agenda item doesn't dominate the whole meeting, or cause it to run too far over.

The type of an agenda item is one of three options:

1. Information refers to an agenda item where one member of the group is going to report to all the other members about some topic. The other members are only expected to listen and ask questions, if necessary.
2. Discussion refers to an agenda item open to general discussion. All members may raise issues and voice their opinions, but no decision need be reached by the group.
3. Decision, on the other hand, refers to an agenda item where the group must reach a consensus. The item is not complete until the members have agreed on one course of action over the other available alternatives.

The type is an important component, since it makes clear to all members the purpose of each agenda item, allowing the group to deal with each accordingly. An item that must conclude with a decision probably needs to be handled differently than one that merely requires discussion, for instance.

A good agenda should give each group member a shared understanding of the meeting's purpose and tasks, at a glance. Without one, you'll likely face the Dilbertesque situation of series of pointless meetings that slow down the project rather than move it forward.

[ Permanent Location ]

An Obsession with Measurement

processes & methodologies

December 30, 2003, 12:33 AM

A policeman on the early morning beat sees a man, obviously drunk, standing under a street lamp and casting about on the ground as if he was looking for something. He parks his cruiser and asks the man "You there! What are you doing?"

"Looking for my keys, ocifer," slurs the drunk. "I lost them and I can't find them."

"I see. And you're certain you lost them under this street lamp?" the officer replies.

"Oh no, sir, I lost them over in that alley. But the light's better over here."

You've likely heard some version of this story before; it's an old one. The joke, of course, lies in the drunk's absurd belief that the better light will somehow help him find his keys. But its not always uncommon for beliefs like this one to get embedded into accepted practice, until no one realizes how absurd they are.

Tom DiMarco once said, "You can't manage if you can't measure." And without a doubt quantifiable measurements are invaluable tools; they provide objective proofs of progress and success, as well as early warning of impending failure. Processes that are founded on measurable objectives are almost always more efficient and effective than processes that are not. In a society founded on scientific values, it's unsurprising that measurability is often insisted upon by many managers.

The problem is that many valid and important objectives may be inherently difficult to measure. "Provide an excellent user experience" is one such objective, so is "improve programmer productivity", "increase the sense of community of our website", and "improve employee morale", for example. Yet rather than leave these important issues to softer evidence, often people will try to graft numbers onto the objective (efforts in software metrics, starting with counting lines of code, come to mind) or will just ignore the objective outright (especially if their incentives favor maximizing the quantifiable objectives). But now we're starting to look like that drunk under the street lamp; we're carefully measuring the wrong thing, just because we figure that somehow measuring something will be better than having no measurement at all.

That isn't necessarily the case. A programmer whose pay depends on how many lines of code he cranks out will write sloppy, inefficient, overly verbose code just to crank up the line count. A UI designer who will be fired if his interface doesn't start "passing" more in-lab usability tests will inevitably design for the novice users in the lab, probably at the expense of any other usability objectives. These examples of "metrics that miss" can be more of a curse than a cure for the project as a whole.

What's perhaps most frightening about all this is that any objectives that have to do with values or ethics are frequently notoriously unmeasurable. And they might be the ones that are most unsafe to ignore.

So what's the solution to all this? Only to think carefully about what exactly your metrics are measuring, and to not be afraid of techniques that yield softer evidence when quantifiable metrics just won't do. A quote attributed to Albert Einstein goes "Not everything that can be counted counts, and not everything that counts can be counted." Better light won't help you if you're looking in the wrong place.

[ Permanent Location ]

Divide and Conquer

processes & methodologies, writing & communication

December 16, 2003, 04:25 PM

Now, I'm thinking about team projects and efficient ways for several people to work on the same project.

In The Mythical Man Month, Fred Brooks confronts the problem of how to effectively divide up the work of large, complex software development efforts. He concludes that the project management notion that bringing more people on to the project will linearly increase the amount of work produced is a fiction, in fact new people will actually decrease the work output temporarily as they demand the current staff's time during ramp-up. Thus, the concept of a "man-month", which assumes men and months are interchangeable, is a fiction (hence the name of the essay). Even after ramp-up, adding more people to a project exponentially increases the communication overhead, since everyone must now potentially communicate with one more person to do their job.

The key issue here is that communication cost. When working in a group, you have to consider whether its possible to split up the work in such a way that the communication costs will be low. If everyone can work in parallel without talking, then you can just divide up the work into N pieces and if it isn't going fast enough, bring in more people to make N larger. Brooks' example of a task like this is ditch digging; if your ditch deadline is bumped forward, just hire more people, give them shovels, and dig in.

Unfortunately, most kinds of thought-work, which everyone in design, usability, research, and software development tends to be engaged in, isn't very much like ditch digging in this regard. Most of our projects are communication-intensive, and can't easily be split up in such a way as to make them parallelizable. Code has to work with other code, interface designs are too holistic to develop entirely separately, research must tell a coherent story, etc. What to do?

If possible, the best answer is "do it yourself". Many novice teams make the mistake of splitting up jobs that could be done way more efficiently by any one of them alone. Don't feel like you have to give everyone an equal amount of work at the expense of making everyone work harder than they'd otherwise have to.

If the task is too big for one person to accomplish, then your best bet is radical colocation. In Agile Software Development, Alister Cockburn recommends this technique for small teams that must engage in frequent communication, and he gives an extensive argument on why even moving people down the hall from one another can drastically limit communication opportunities. This generally leads to miscommunications, which leads to buggy software, unusable interface designs, misrepresented experiment results, etc.

So in summary: if possible, don't even try to split up thought-work; have it come from one mind. If that's not feasible, try to divide it up to minimize the amount of communication the workers will have to engage in to do their jobs. If the job is such that you can't reduce this to an acceptable level (which is more often the case than not, so don't be lulled into underestimating the amount of communication necessary), then radically collocate.

The nice thing about this advice is that it applies at all levels: from units as large as corporations outsourcing large initiatives to one another and as small as two people in a small student team.

[ Permanent Location ]

Maintenance Projects and U&SA

processes & methodologies, software development, usability

November 22, 2003, 07:06 PM

Turns out Dan wrote an article for Boxes and Arrows about a year ago titled "Tackling Maintenance Projects". The article is excellent and I recommend reading it. What I found most interesting, however, is how Dan's advice relates to the impact of software architecture on usability.

Dan draws a distinction between "Atomic Maintenance Projects" (more or less complete redesigns, of potentially both the interface and the underlying implementation) and "Neutron Maintenance Projects" (small changes that serve as point solutions but may not address the larger usability issues). The problem he observes is that often your client thinks they only need a Neutron fix, but after inspecting the interface you realize that an Atomic redesign is necessary to achieve adequate usability. In his words:

And there’s the rub. Often, IAs are left to wrestle with decisions that were made long before their involvement with the project; decisions that people in the company may not even like but are stuck with. Or decisions they have a vested stake in keeping intact. Or decisions various forces in the company (IT, Marketing, and Legal being typical stakeholders) insist must remain, for reasons the IA may or may not agree with.

Our hope with U&SA is that by bringing usability professionals (or designers or information architects or what have you) into the process at the system architecture design stage and giving them the intellectual tools they need to understand how to effectively contribute, we can reduce the need for Atomic projects so that more usability issues can be dealt with at the Neutron project level, which, as Dan correctly points out, is often insufficient given the current state of the practice.

[ Permanent Location ]

How the IT Industry Really Works

funny, processes & methodologies, software development

November 19, 2003, 04:29 PM

This cartoon is an oldie (I think I first saw it in "Developing User Interfaces" by Hartson and Hix), but it's worth repeating. It's a great 8-pane summary of the product development communication problems that I keep rambling about on this here forum.

[ Permanent Location ]

Discussing Disconnects

design, processes & methodologies, software development, usability

November 15, 2003, 07:08 PM

At the SSS two days ago, I led a discussion on how we can bridge the gap between interaction designers, usability professionals, and software engineers working together on real-world development teams (as I promised I would). I started out with a quick presentation describing the personalities and goals of both "players" (designers and developers) through some rough personas, then opened the session up for discussion on what techniques might help in getting these two players to work together effectively.

Here's a list of the possibilities that were thrown out for consideration. If anyone out there in readerland was at the discussion and notices something I missed, please do add it in a comment.

1. Emphasize more up-front design so design and engineering need not work in parallel. The idea is that this will reduce the amount that the two players need to communicate, thus reducing conflicts and making the process more efficient. It was also pointed out, however, that although spending time on up-front design is good, it doesn't solve all the problems. The final design, no matter how good, will still need to change quite a bit based on engineering's input and concerns, thus reintroducing the communication problem.
2. Design must bring engineering onboard. Those who came from an engineering background emphasized the need for design to justify and "sell" their design solutions to the engineering team. The idea is that although engineering trusts design to be good at their job, they will be more effective participants in the overall development process if they understand what the important usability issues are and why the related design decisions were made.
3. Design and development must "live" together. Basically, the two players should work closely throughout the entire process; there should be no "throw it over the wall" style information exchanges. I've suggested this before in the context of XP, going so far as to recommend that the two players work together in the same room.
4. Designers should contribute to development work. Many designers have enough technical abilities to contribute to parts of the user interface development process. In addition to speeding things along, this gives engineering a better sense that the designers are true members of the team. Cooper and Reimann suggest that one individual cannot fill the role of a designer and developer, due to the fundamentally different mindsets, but everyone in the group thought it was possible to strike a happy medium. In order to facilitate this technique, the software architecture must separate the user interface modules from the rest of the application using a separation pattern such as Model-View-Controller (which still won't handle all concerns, of course).
5. Designers should have engineers sit in on user tests. Watching users struggle with bad designs is often cited as a good way to motivate people who aren't willing to buy in to user-centered design. If this is the situation designers face, this is a technique they can try to bring engineers on board. Randy Pausch, ever famous for mistreating his engineering team, requires them to sit on their hands while watching people use their interface, and demands they keep silent unless they are willing to precede their comments with an apology for designing such a horrible interface. This might be a bit extremist for most industry teams to swallow, however...
6. Both players need case studies on how to effectively work together and how not to ineffectively work together. Design and engineering (and management) can learn from examples. Even experienced industry veterans may not associate usability failures with market failures, since these two events are often separated by a long period of time, and humans are bad at associating effects that occur long after their causes. Of course, someone must put together a collection of these case studies before designers and engineers can learn from them.

Perhaps the most interesting part of the conversation, however, was the discussion of the benefits of "creative tension", or those times when design and engineering don't get along yet wind up with a solution that is better than what they would have come up with if they had. Haven brought up the case of the Harley-Davidson VROD, a case study where there were several examples of the designers putting their foots down and demanding that engineering figure out a way to implement their design, as well as examples of the engineers demanding that the designers come up with workable designs. In the end, this resulted in a better product than Harley-Davidson would have come up with had the two sides come up with a compromise solution.

This raises the question of whether it is necessarily every desirable for design and engineering to get along smoothly, or whether a certain amount of tension and controlled chaos is sometimes necessary to produce a quality solution.

[ Permanent Location ]

Radical Colocation is a Good Thing

processes & methodologies, software development

November 15, 2003, 03:32 PM

Joel has an article up on the subject of adopting methodologies for software development. He makes a lot of good points about the state of the practice, but when discussing the subject of whether software development should occur in private offices or public collaboration spaces, he makes the following claim:

But we don't have the data. We don't have any data. You can give us anecdotes left and right about how methodology X worked or didn't work, but you can't prove that when it worked it wasn't just because of one really, really good programmer on the team, and you can't prove that when it failed is wasn't just because the company was in the process of going bankrupt and everybody was too demoralized to do anything at all, Aeron chairs notwithstanding.

This isn't quite true. Earlier this year I came across an article that came from Judy Olson's research group on this subject called "How does radical collocation help a team succeed?". It's an amazing piece of empirical software engineering research that gives some hard data to back up the claim that putting everyone in the same room together increases productivity in software teams. Due to copyright concerns I can't repost it here, but you can find it in the ACM digital library if you're a member. Here's the abstract:

Companies are experimenting with putting teams into warrooms, hoping for some productivity enhancement. We conducted a field study of six such teams, tracking their activity, attitudes, use of technology and productivity. Teams in these warrooms showed a doubling of productivity. Why? Among other things, teams had easy access to each other for both coordination of their work and for learning, and the work artifacts they posted on the walls remained visible to all. These results imply that if we are to truly support remote teams, we should provide constant awareness and easy transitions in and out of spontaneous meetings.

In short, the article found that, in the situation under study (small teams, fixed time frame), radical collocation drastically increased the productivity of the software teams, as measured by a function point count that they compared to the count for the company's standard cubicle setting as well as to a whole industry baseline. They also found that although many of the participants were reluctant to work in the collocated environment at first, they came around by the end of the study. Not surprisingly, they did find that certain tasks with high cognitive demands were hard to perform in the collocated environment due to constant distractions, so they recommend including private "hoteling" areas for such tasks as well as for placing personal phone calls and the like.

The actual article is worth a read if you can get to it, especially since they conclude with recommendations for developing effective collocated and remote work environments based on their findings. And if you're a researcher, the study is a great model of the direction the field of empirical software engineering should go in if they wish to provide the kind of hard data that Joel correctly points out is sadly lacking in the current state of the science.

[ Permanent Location ]

Storytime and Pretty Pictures: Data That Informs Design

design, processes & methodologies

November 11, 2003, 11:47 PM

One of the things I'm interested in is studying how to guide design through empirical research, "science-guided design" you might call it. The problem with classical research methods is that although they are effective at uncovering truth, they are too general and reductivist to guide design by themselves. To fill this void, methods have arisen in both usability engineering and interaction design that are intended to help development teams distill fieldwork findings into forms that help designers creatively develop interfaces that satisfy user needs as well as test their interface assumptions against their understanding of the state of the world. Although there are many techniques to fill this role, all of them fall into two general forms. Very roughly, these two forms are "telling stories" and "drawing diagrams".

The "telling stories" approach involves distilling interview and observation data into descriptions of the target users and scenarios of them performing their tasks with the system. The most well-known technique that employs this approach is Goal-Directed Design, developed by Alan Cooper. Cooper's technique involves developing personas, or detailed descriptions of "prototypical" users that incorporate their personalities, goals, skills, etc. Cooper emphasizes that the persona descriptions must be realistic and believable; the team must envision these imaginary people as if they were real users of the system whose needs must be satisfied. Other techniques, such as Scenario-Based Design, take a similar approach but focus more on developing stories of users interacting with the system. But all techniques that follow this approach focus on developing natural language stories about the people who must use (and therefore are a part of) the system.

There are several advantages to this approach:

• Stories make it easier to understand the real human costs and benefits for a solution. Abstract numbers are important for understanding aggregate effects, but they are notorious for hiding away the true events that effect the lives of the people behind those numbers. The "telling stories" approach helps provide this level of detail. This is especially essential for designing desirable products, since desirability is very much a function of individual users' emotional responses to the product.
• Detailed descriptions of users brings focus to design. It helps you define clearly "Here are the people we are designing for. We must always check our designs to ensure that we're convinced they'll make these people happy and meet their goals." You can't do this with vague notions of what "the users" want, and its very hard to do this with purely numerical data (how can we know if we're meeting the goals of all these numbers?).
• Stories are easier to understand. Describing a scenario to people outside the design team (such as management or clients) is generally a much more effective and compelling way to present a problem or a design solution than specifications, diagrams, or screenshots. Explaining and selling your designs is very important, so representing your research data in this fashion helps leverage this fact.
• Stories are "fuzzy". This may or may not be an advantage depending on who you are dealing with. Designers, artists, and end-users may appreciate this quality of these techniques. Engineers, scientists, and (some) managers may not. Know your audience.
• Scenarios can be translated to system responsibilities. If you distill your stories down into specific scenarios, then these scenarios can be distilled, in collaboration with engineering, into system responsibilities that guide implementation efforts. For instance, in the popular UML notation, system requirements are represented as use cases, which are natural derivatives of scenarios of system use.

The "drawing diagrams" approach involves representing data collected during interviews and observations as diagrams and structured prose that describes pictorially the events, interactions, environments, etc. that you encountered. The method that defines this approach best is Contextual Design, which calls for distilling inquiry data into the "five faces of work" — five diagramming notations that describe the work flow, culture, physical layout, artifact usage, and sequence of tasks for every user observed. Beyer and Holtzblatt, the developers of Contextual Design, emphasize that these documents capture the relevant information about the world to guide design, and provide an entire methodology for transforming these diagrams into an actual interface design. Unlike Goal-Directed Design, this process guides you through each step of the interface development life cycle.

There are more techniques that employ diagramming than Contextual Design, however. Matt introduced me to causal flow analysis and systems thinking last spring, two techniques that employ diagramming notations to map out complex reasoning paths and identify patterns and breakdowns in organizational systems, respectively. Although their purposes vary, all these techniques employ diagrams for the same purpose — to focus attention on the aspects of the data that the methodology designers decided was important and needed to be explicitly represented.

There are a number of advantages to using diagrams:

• Diagrams help you understand the whole system. Pictures like this give a more holistic view of the state of the world than stories do, since stories tend to focus on individual people. Natural language often falls apart when you try to use it to describe abstract concepts like organizational systems or the structure of communication in a work environment. A good diagramming notation is much more effective for communicating this information at a glance.
• Diagrams get at data and structures that aren't obvious from descriptive prose. Many important concepts are hidden in plain language, if they appear at all. For example, oftentimes an argument may sound convincing when you read it in prose, but when you draw a causal flow analysis or apply another argument mapping technique, you quickly see the holes. Diagrams help for cases like these by highlighting the relevant parts of the prose and de-emphasizing others.
• Diagrams help you design how your changes will fit into the whole system. It's important to understand that any new system must fit into the larger systems that compose our reality. Diagrams make it easy to guess the results of changes in the systems they represent, since this generally only requires altering the diagram to include the new system.
• Diagrams are perceived as "rigorous". Again, whether this is desirable or not depends on who you are communicating with, but diagramming techniques are generally perceived as more rigorous than storytelling techniques. This tends to go over well with engineers and scientists, and possibly management.

I believe that both of these approaches are important for understanding different aspects of a design problem. Thus, when faced with a choice of which data analysis methods to use, the appropriate question should not be which is better in the abstract, but rather which is more tuned to solving the particular problem the team is facing (often, in fact, the correct answer may be to mix the two and use different techniques for different aspects of the problem).

Finally, I firmly believe that this work has a broader applicability than just interface design. Many, many fields could benefit from decision-making processes that are informed by empirical work. Why, for example, does Congress not charter fieldwork teams to investigate the needs of low-income households before altering welfare laws? Why do we not turn these design processes to systems built of humans, as well as systems built of software or raw materials? The need is there, and the techniques are emerging. I hope the right time for this movement is coming soon.

[ Permanent Location ]

Information on Personas and an Argument for Design

design, processes & methodologies, usability

October 25, 2003, 09:29 PM

There's a good, thorough overview of personas on Information Today. The article gives a great digest of Cooper's description of personas in Inmates, although once again I could ask for a little more guidance on how a researcher is supposed to collect the data to back up a good persona. There's talk of careful collection of qualitative data, but the process of translating those findings into personas is still pretty mysterious.

What I particularly liked, however, was this small story that illustrates the need for user-centered design:

One of the best arguments for using personas comes from some misguided design efforts at Microsoft. When the software giant geared up to redesign its well-known Microsoft Office Suite for a 1997 release, the research team soon discovered that many of the features users wanted already existed. In fact, four out of five of the features users requested for Office 97 came with Office 95. The outcome of Microsoft's design approach is just what Cooper warns against. In trying to support the diverse tasks of many conceivably different software users, Microsoft cobbled together a product that was bloated with capabilities and ended up satisfying few users.

Many people in marketing, development, and management will argue that piling features into a new product is far more important than designing usable interfaces and compelling interactions. And early on in the product's life cycle when the features are few and the interface relatively simple, this may be true, at least if your users are fairly computer-saavy. But once a few layers of those nifty new features have been slapped on to the product, you'll quickly start to find your users asking for capabilities that already exist. Bottom line is, you wasted your time with all those shiny new features, because your interface is so poorly designed that nobody even knows they're there.

[ Permanent Location ]

Is Usability Destroying Innovation?

design, processes & methodologies, usability

October 23, 2003, 11:45 PM

Dan has a post pointing to an article in the Guardian on innovation and user-centered design. The article's short so it's worth the quick read, but in brief it claims that user-centered design has become dominated by usability practitioners, that usability is a conservative approach and inhibits innovation, and thus it should be de-emphasized to encourage greater innovation in products (well, that last part is implicit but it's certainly there).

On the one hand, the core point of this article is quite accurate. It's important to produce innovative designs so that you aren't just incrementally improving the same old (bad) design concepts. And if you take a narrow view of usability (namely, that it only exists to find errors in the design of existing interfaces through analytical (heuristic evaluation, cognitive walkthrough) and empirical (usability testing) techniques) then this is the whole story.

But this is not the whole story. The user research techniques that form the core toolkit of any usability professional worth her salt include much more than testing. To be specific, we use lightweight ethnographic techniques like contextual inquiries and user interviews to study the way users work in the abstract so that appropriate design solutions can be found. This type of research can drive design, indeed it must drive design if the final product is to be usable, useful, or desirable.

The article also appears fairly ignorant of established usability practices. In particular, it states: "Too much user focus may be a barrier to innovation. Research is likely to tell us that users desire an improvement on something they already understand. Ask them if they would use a proposed innovation and they will say no - and then adopt it when they have seen its utility demonstrated." Very true, but this has been well known to usability professionals for over a decade. Studies by Nielsen (and others before him) firmly established that directly asking users what they want will yield bogus data. Any decent usability professional knows this and would not trust the answers to such questions. Design must be constrained by the users' observed needs, not the user's expressed ideas.

In summary, my main concern with this article is not that I disagree with its premise but that I'm unsure what the takeaway message is. If the takeaway is that user-centered design must begin earlier in the process through user research and interaction design, then I'm firmly in agreement. But if the takeaway is that designers must be left to design without being constrained by user needs that are well understood through sound research, then I'm worried, because then we're dropping the whole "user-centered" part of "user-centered design". The user is not like me; I don't trust my own or anyone else's "educated guesses" about what someone else will want in a product or interface design.

Finally, I'm concerned at the distinction between "usability" and "design" that is drawn in this article as well as in the industry as a whole. I don't really believe that the two are cleanly separable, any more than I believe that software architecture design is cleanly separate from programming, for example. In my not-so-humble opinion, user-centered designers must, at least to an extent, be good at both user research and interaction design, and specialists in both fields must work closely together to be effective. We of all people should know that we ought to be putting the user first in all that we do. And in this instance, the user doesn't care what fancy titles we pin on ourselves. The user cares about getting a great product that makes his life better. And we need all these skills (mixed with sound engineering too, I might add) if we hope to give it to him.

[ Permanent Location ]

A Compendium of Usability Resources

processes & methodologies, usability

October 21, 2003, 01:16 PM

Usability Net is a great reference site for all kinds of usability techniques and processes. They have a table of usability methods along with some "filters" that show which methods are appropriate in different circumstances (I'm not sure how much I believe these recommendations but they're some good starting ideas). They also have a list of guidelines for web, mobile, and other platforms, a list of all the usability methods and their detailed descriptions (U&SA isn't listed; big surprise), some businesses cases and cost justifications for doing usability, and lots more good stuff. Check it out.

I think I found this via Chad a long time ago. He also points to Usability.gov, the National Cancer Institute's web usability center. Rich was working on making these guidelines better, although I'm not sure if his work ever went live or not.

[ Permanent Location ]

Using Research to Guide Design

design, processes & methodologies, society & sociology

October 13, 2003, 11:15 AM

As I've mentioned before, I'm taking CSCW: Designing Online Communities this semester and one of the stated goals of the class is to figure out a way to translate the knowledge gleaned through social science experiments into a set of principles that can guide the design of online communities. The way we've been approaching this so far has been to distill out a set of stylized social science facts and then brainstorm features that would conform to these principles (generally by altering the design of an existing system, MovieLens).

My preliminary thought on this approach, however, is that this is the wrong way to go about applying social science to design. The problem is that brainstorming features from social science facts doesn't take account of the holistic nature of design; the principles are very specific and reductionist and thus the feature ideas you tend to get may violate some principles while supporting others, or just be bad ideas for other reasons. Some of the stylized facts are even contradictory, at least at this abstracted phase (e.g., disclosing personal details about yourself makes you like the people you disclose them too, but spending too much time with people you have an intense relationship with makes you like them less). They also might not fit into the conceptual structure of the design, or the culture of the particular community.

This is not to say that the whole idea is fundamentally flawed, just that I believe we should take a slightly different approach. Instead of using social science to guide design explicitly, we should use our stylized social science facts and distilled design principles as heuristics that can be applied in a post-design analytical usability evaluation, similar to Heuristic Evaluations. Although these principles may influence community designers, just as Nielsen's heuristics influence user interface designers, they are primarily intended as a checklist. Nobody comes up with interface designs merely by pondering Nielsen's heuristics, so we shouldn't expect that of community design either.

[ Permanent Location ]

Small Groups for Design Critiques

design, processes & methodologies

September 25, 2003, 11:09 PM

In VIID today, we divided up into small groups to discuss our design sketches for the second project with members of the class who were not on our project teams. I wound up in a group with just Kerry, although Jodi and Bilge both dropped by during the session to offer their comments on both of our designs.

This turned out to be a pretty effective format for refining and improving our design ideas. My group got a lot of great feedback that has really helped us pull together the several disparate design solutions that we'd been kicking around before. We came out of that session with much more direction than we'd had going into it, which is something I can rarely honestly say about a class meeting.

Dana and I are discussing the possibility of attempting to bring this format into Newsable as another solution to the open source and usability problem. We hope to set up an appropriate online virtual environment for developers and usability professionals / designers (we're calling this group "usables", which I assure you was entirely Dana's idea) to brainstorm, test, and critique new suggestions and designs. We hope this will form the kind of small groups with small group size, high task valence, and high senses of group identity that previous social science research has found to greatly increase quality contribution.

It's an amazing experience to be working with Dana. So far she's brought much more focus to the project than I've had all summer as well as some great new ideas that I would never have come up with on my own.

[ Permanent Location ]

Generalizing Design

design, processes & methodologies

August 10, 2003, 09:37 AM

Six weeks ago, I posted about the design process that we learned in CDF. Now, CDF is over (exempting the process book, which I'm struggling to compile right now). Over the course of the session, I've been reflecting on whether there's any such thing as "design" as a general concept and, if so, what it might be.

I've had some preliminary thoughts on this topic and even voiced a definition of "design" before, but I still considered the word mostly an open category; there's little similarity between all the activities called design, software design and visual design being prime examples. However, after broadening my horizons a bit more, I'm starting to form a vision of general themes that run through all these activities to bring them together into a coherent class of activities.

The Design Process

I'll start with some general thoughts on the design process. From my original post, we know that the design process is broken down into three parts: familiarization, development, and refinement. So we have:

However, we also know that this ideal model never works in practice. So in reality, we have:

The important thing to note about these pictures is that the theoretical process strictly divides each step, whereas the actual process (the one that works) is much more messy since it calls for moving on to the next step before you've really completed the previous step. Thus you're likely to end up oscillating between two (or even all three) of the steps, running them in parallel, tossing out work from later steps to refine earlier steps based on what you've learned, etc.

In all the design processes I've encountered, there is considerable disagreement in their respective fields over how much overlap there should be between techniques that fall into each distinct step of the process. When you're in the familiarization-development part of the process, you can't finish familiarizing yourself with the problem unless you start developing, since this helps you learn what you need to familiarize yourself with. There is simply too much information in the world to make familiarizing yourself with everything you "need" practical in a limited time frame; starting development gives you focus. But if you develop too much without doing enough familiarizing, you may discover (usually too late) that you've been developing the wrong solution to the problem (or the solution to the wrong problem) all along. Likewise, in the development-refinement stage you need to begin refining to discover whether your development is working or not. But if you refine too much and then realize you need to change your development, you'll have to throw out all your refinement work, which may be unacceptable late in the game. Additionally, too much emphasis on refinement can distract you from large, looming development problems that are much more important in the long run than the piddly little refinement issues.

The Design Problems

This general design process is independent of what you're designing; it should hold as true for designing cars as it does for designing cities. Additionally, there are a few problems inherent in this process that always tend to crop up in one form or another:

• Early decisions dictate future decisions. This means early decisions are hard to change late in the process since doing so entails throwing out lots and lots of later decisions (which translates to throwing out work, which translates to throwing away money in our capitalist system). An instance of this is the software architecture problem, about which I've spilled so much digital ink in the past.
• Wide exploration of the problem is necessary for coming up with the best ideas. The wider your exploration, the more possible ideas you'll come up with, thus improving the probability that a select few of these ideas will be good ones worth pursuing. In the small, this isn't always true; sometimes you'll stumble on a great new idea right off the bat. But its more likely that this idea merely seems good, and upon considering several more you'll find better ones or at least realize why the first one wouldn't have worked after all. More experienced designers can sometimes get to these ideas with shorter, more directed explorations, but even they realize that they can't take the first thought that pops into their heads as the One True Way.
• Feedback on all activities is essential, with tight feedback loops. Without feedback, there's no way to distinguish the good ideas from the bad and no way to know which designs are working and which aren't. Without tight loops, you'll run afoul of the early decisions problem I mentioned earlier since you won't realize your approach is problematic until it's too late to economically change it. Both are absolutely essential to keep the whole process from falling apart around you.

Examples of the Process in Practice

The design process gets instantiated in many fields, all of which follow the same basic steps and struggle with the same basic problems. I've already said a few words about how I think it relates to the everyday task of redesigning my living space, but now I'll try to apply it to my more professional interests.

In software development, the familiarization stage involves understanding the technologies the system will be constructed on through research and prototyping. The development stage involves constructing the software architecture and related decisions. The refinement stage involves developing a detailed design and writing and testing actual code. As I mentioned before, the software architecture problem appears because early design decisions dictate future decisions; changing the architecture late in the process involves throwing out and rewriting lots and lots of code. Unit and functional testing provides feedback to the developers in the refinement stage on whether their detailed designs and their realization in code are working (although often this feedback occurs too late to fix problems from the development stage). Ideally, the team uses refactoring to explore a wide range of ideas, both at the detailed (refinement) level and the architectural (development) level, but sadly, wide exploration and early testing of architectural decisions is rarely done, to the detriment of the profession.

In interface design, the familiarization stage involves discovering the real needs of your user population through observations, interviews, stakeholder meetings, competitor analysis of similiar products, etc. Development involves brainstorming ideas for the actual design, constructing information architectures (there's that word again), doing screen flows and other structuring activities, and building lo-fi mockups to test all this. Refinement involves building hi-fi mockups of individual screens and running them through the testing methods (cognitive walkthroughs, think-aloud tests, etc.) for feedback on whether your solutions are usable. Again, once you've settled on a basic design its hard to change since it involves throwing out so much work. This is especially true when its committed to actual code and thus runs afoul of the architecture problem, and its even more true when its in actual use, and modifying the interface may confuse and anger existing users. Of course, any usability professional knows how important feedback (in the form of usability testing) is, and any interaction designer will tell you how important it is to brainstorm and try out many ideas if you want to find one that actually works.

In communication design (what we've mostly been doing in CDF), familiarization involves understanding the audience and the message as well as the constraints on the communication. We did some of this in the final week of CDF on information design, where we redesigned the post office's mail forwarding form and did some analysis of focus group results, the information structure of the form itself, and the technical constraints of the problem. Development involves coming up with lots of ideas for solutions and realizing these ideas in rough sketches or possibly more polished proposed solutions. Refinement involves the thousands of little painstaking tweaks to perfect the visual presentation, the tediousness of which will keep me from ever becoming a visual designer :). Feedback in this field usually occurs at all stages through critiques with other designers. Sometimes user testing occurs as well (Bob, our professor for information design, remarked that the next stage after designing our forms would be to user test them), and I'd argue it should occur more often.

All of these more specific design processes follow the basic pattern, as I hope I've shown. The difference between them lies in the techniques used at each stage, the pliancy of the respective mediums, and, of course, the end goals; each process is aimed at producing a different sort of product. Though these differences are significant (I'm not claiming that, when you get right down to it, all design processes are the same), these similarities run through them all to pull out a unified picture of what it means to "design".

And there you have it. Rob's manifesto on the nature of design.

[ Permanent Location ]

On the Value of Constraints

design, processes & methodologies

July 26, 2003, 11:33 PM

Over the course of this summer session, I've been working on a variety of projects, many of which involve skills I haven't yet developed and concepts I haven't yet fully grasped. During the course of all this, I've experienced a semiprofound revelation.

Despite what you might think, in the majority of cases constraints are a Good Thing™.

By constraints, I mean limitations on our actions, the finite quantities of resources we're given to solve problems, frameworks we are required to work within, etc. A common example is the time constraint; I often complain that I don't have enough time to properly complete an assignment or code, up some software component. I've generally conceived of such things as necessary evils that we must, sadly, deal with and take into account if we wish to get things done. In a perfect world, however, there would be no constraints. We'd have all the time in the world, all the people we need, all the raw materials we could wish for, to come up with the perfect solution.

I've changed my mind, for several reasons. In Communication Design Fundamentals, we've been given a number of different design problems, generally fairly open but with some constraints attached. For example, in the first week we developed a poster for a series of talks and used various typographic attributes (line spacing, stroke weight, etc.) to communicate the hierarchy of information. At first, we were limited to a single typographic "variable", later assignments opened it up to two or three. We quickly found that this freedom made things more difficult (although we certainly came up with better designs with two variables rather than one). Likewise, during Dan's week we started off arranging our quote using only positioning, then we got to play with stroke weights and size variation and whatnot, and sure enough it got harder (and some of the designs started to look "overdone"). Dan told us that in this class he was imposing the constraints on us, but in our work we would have to decide what constraints to impose on ourselves. An interesting proposition. I wonder if the qualities that make a designer (or an artist) great aren't so much "wild creativity" but a form of intelligently constrained creativity that knows where to direct its energy for maximum effectiveness.

This meme even gets reflected in the class itself; I've come to feel that I enjoy the course as much as I do because each professor has been forced to boil down their topic into something that can be taught in fifteen hours. This has lead to a nice mix of hands-on practice, minimal instruction, and a wide (but not stretched) view of the topic area. Great for an introductory course.

Likewise, I've been mucking around with Perl quite a bit lately, a language that is highly unconstrained and suffers as a result. There may be More Than One Way To Do It, but this intimidates novices and makes Perl code notoriously hard to read. As does everything, Perl has its advantages as well as its disadvantages, but a few more constraints may have resulted in a much cleaner design.

In the end, constraints help get things done. To return to the ever-present time constraint: if I was given the infinite time I desired, its likely I'd never finish anything (which happens all too often with personal projects for which I set no goals). In academia, a sector I've become all to familiar with recently, there are often few time constraints for day to day work. Many of the academics I know tend to wander aimlessly or squabble over relatively unimportant details. As a result, high-quality outcomes don't often come from university labs.

Another form of constraint comes in setting goals. In a way, goals are the self-imposed constraints Dan was talking about. Matt argues that every project needs to have a clear set of goals to guide it, otherwise no one will accomplish anything meaningful. Unless you commit to accomplishing measurable objectives within a given time frame, you'll wander aimlessly, wasting your money and your time.

[ Permanent Location ]

The Design Process

design, processes & methodologies, teaching & learning

June 30, 2003, 04:11 PM

Today was my first day of Communication Design Fundamentals (CDF), the prerequisite studio-based design course for the HCI program here at CMU. We started off the course with an exercise in the design process where the teacher, Karen, gave us a pile of semirandom household implements and asked us to arrange them into an ordering so that someone who walked in the door could understand just by looking at them how they all related to each other. Apparently this is a common exercise for introducing students to the design process.

The design process Karen described goes like this:

1. Familiarization, where you work on understanding the domain and what the general relationships between objects in that domain are. For us, this involved figuring out what each implement does (difficult, since there were some weird items. No one got the Fijian cannibal's fork...) and grouping them into some general functional categories.
2. Development, where you start to define the general structures and groupings. We divided the "kitchen items" and "art items" into separate areas, organized the kitchen items into a circle and the art items into a grid, and divided into two teams to manage the arrangement of each.
3. Refinement, where you start to focus on the nitty-gritty details of the design. This is where we separated out the various piles of items we made into groups separated by whitespace and aggregated by function or appearance.

It struck me as we were going through all this how similiar it was to other design processes I'm familiar with, like the HCI user interface design process and the software system design process. In HCI, you start by studying your users and trying to understand their needs, their desires, their tasks, and their gripes, which is part of familiarizing yourself with the domain. My persona development process for open source software is an example of a technique used at this stage. Next you design the high-level information architecture and rough screen designs for the application, which is equivalent to the development stage. Finally you build higher-fidelity mockups that you can actually run user tests and analytical checks on, the HCI version of the refinement stage. In software development, you frequently familiarize yourself with the technologies (although this stage is often skipped, to the detriment of many projects), develop a high-level architectural design, and then refine this design into detailed class hierarchies and object interactions and finally down to actual source code. And as Karen pointed out for the design process, at each step you frequently must move to the next step to see if your design ideas will work or not. A strict "waterfall" approach to the design problem rarely works.

When we got to organizing the structure of our designs, one group had put items into a basic circular structure, the other had used a grid. Karen pointed out how design decisions such as these can determine the structure of your later decisions. Her off-the-cuff analogy described this as similar to cutting a pie; the first cut you make determines the positioning of all the future cuts. So when you make those early decisions, you need to make sure are careful and have your audience's needs in mind. This is quite similiar to the classic software architecture problem; the architecture of a software system encompasses those decisions made early on that determine what is easy to change and what is difficult. Its interesting to reflect on how much these apparently different design processes have in common.

As we were progressing through this exercise, Matt remarked that this was an example of experiential learning; Karen gave us a specific problem to work on first, then helped us generalize our experience to all of design. She didn't lecture per se, instead she interjected comments and suggestions as they we needed them to help solve the example problem. It was a neat approach; I'm excited about teaching in this style with Matt for our TCinC summer course, which starts tomorrow.

This session is shaping up to be a lot of work, but it looks like it will be interesting work and thus bearable. It's also great to hang with my fellow part-time HCI students who are all in CDF with me (sans Mathilde). I even get to see Neema again regularly, which is a rare treat. Good times all around.

[ Permanent Location ]

Personas and Open User Research

processes & methodologies, usability

June 25, 2003, 10:16 AM

As I described on Newsable's About page, the goal of all this user research and personae development is to provide a technique for open source project teams to design and test usable interfaces. However, thus far I haven't made this technique explicit. This post is a first attempt at doing that.

In The Cathedral and the Bazaar, Eric Raymond presents open source software development as a highly decentralized process of many collaborating minds spread out over a potentially large geographic distance. However, Raymond glosses over the fact that most open source software projects are almost entirely managed and worked on by one individual (the "benevolent dictator" approach, as seen on Linus's Linux kernel project) or by a small team of core project members (the "meritocracy" approach, as seen on the Apache HTTP server project). What makes open source unique is that there are an order of magnitude more people who contribute in small ways to the project by finding bugs, suggesting changes, and possibly even sending code patches to the core team. So in essence, the typical open source project is run like this:

This is not a bad thing. We want to have one or a small number of team members in charge of developing our software systems when at all possible; this is one of the core tenents of Agile software development and was advocated by Fred Brooks in his classic text The Mythical Man Month when he asserted that the design of a software system should come from one mind.

Good user interface design is, in this respect, not so different from good software design, and thus we can expect the basic open source method to apply equally well. A small number of people should be responsible for the design of the interface, but these people need to be informed by the experiences and findings of a large body of users of the software. But when we're designing an open source interface, what takes the place of the "source repository" in the picture?

The obvious answer is the user interface source code itself. But nonprogrammer "normal" users can't understand user interface code, much less make suggestions on how to change it. Using the interface code as our artifact would alienate a large and very important part of the user population, which contributes to the usability problem open source software is experiencing now.

So perhaps we can use the interface design itself as our artifact. We could produce screenshots and mockups and share these with the users to get their feedback on usability problems with the design (the equivalent of "bugs" in program code).

Unfortunately, it's well known in usability circles that users aren't very good at providing accurate feedback on interfaces they can't use. Moreover, when users make suggestions about interface designs they generally come up with localized solutions that solve a particular problem they're experiencing but may make the interface as a whole less usable. Finally, people with the same user interface problems often have different opinions of how those problems should be solved. If they are spending all their time bickering over the solution, they may not even realize what the underlying common problem is.

What I propose, then, is that open source projects produce a set of personas, or archetypal users, and scenarios, or stories of the personas using the system, that encode their understanding of who they are designing the software for. One or more members of the core team can take responsibility for maintaining these documents. Outside contributors can provide additional ethnographic study data, user test data, cognitive walkthrough results, etc. that may alter the project team's understanding of who their users are. Users can contribute by pointing out differences between their usage of the system and the usage described by the personas; if the personas don't behave like real people, the users will be able to quickly spot the error based on their own understanding of how they work. The project team can then use these personas to agree on an interface design based on a clearly articulated understanding of user needs. So now, we have a process that looks like this:

Discussions of interface design decisions should all center around the personas. Instead of arguing over personal preferences ("I like the buttons on the top!" "Well, I like them on the side!") or about the mythical "users" ("I think users will find the buttons easier if they're on top." "Well, I disagree.") the project team can talk about the concrete user archetypes they've maintained and reason about how these users would behave if the new interface design suggestions are implemented.

In essence, this technique aims to be an "open user research" technique that closely parallels open source software development techniques. By encoding the team's understanding of their users into personas and making this understanding available to all, the team can leverage the unique benefits of working in an open environment without losing the conceptual integrity that comes with having a small number of people making the actual interface design decisions.

[ Permanent Location ]

Case Studies in Open Source Usability: KDE and GNOME

processes & methodologies, usability

June 20, 2003, 10:16 PM

I joined the GNOME Usability and KDE Usability mailing lists this week, mostly to lurk and watch the discussion unfold to hopefully gain some insight into what the state of the practice in open source usability is like. Here's my (very) preliminary thoughts. Don't take these comments the wrong way; on the whole I think both projects seem to have an impressive emphasis on usability and are doing great work.

The Good

• There do appear to be people conducting actual user tests on the desktop environments for both projects, and these people are posting their findings to the list. Moreover, the developers appear to listen to them and follow up on their questions and recommendations.
• Both projects have a list of guidelines (GNOME calls them the "HIG", which I assume stands for "Human Interface Guidelines"), and these guidelines actually get mentioned in discussions. There's a discussion going on right now on the GNOME mailing list where the HIG was brought up to reason about why a suggested design decision may not be the best choice (a decision about using Alt keys in a keyboard shortcut, to be specific).

The Not-So-Good

• There seem to be a lot of "let's do it this way because I like it better" type of discussions, where list members are just arguing over personal preferences. So far there haven't been any serious flamefests, but there hasn't been a call for "let's resolve this by collecting some user test data" yet either.
• The people who are doing the user tests come to the list with problems and occasionally suggested solutions, but so far no one has come back with a story like "I user tested this part of KDE, and I found these problems. So I prototyped these suggested fixes and ran my tests again, and the problems disappeared. So I suggest we implement my fixes." HCI practitioners have found that software developers don't want laundry lists of problems, they want to know what exactly they need to implement to make their system more usable. So the HCI person needs to come back not just with problems and a few half-baked ideas, but with solid, tested solutions for the developers to pick up and run with.
• Neither project seems to have a consistent picture of who they are designing for (user profiles) and what these people want to do with their software (task lists). So far, its unclear to me how the user testing people decide what tasks to have their test users perform.

Watching these projects discuss their usability issues is a great experience, and I'd recommend signing up for these lists if you're interested in open source usability. I'll post again after I get a bit more insight from reading the list to see if any of these thoughts change.

[ Permanent Location ]

Fury, By the Book

processes & methodologies, usability

June 10, 2003, 01:38 PM

Kevin has embarked on a project to redesign Fury, his popular personal website / weblog. When most people get bored with their old website design, they hide away for some period of time and work on the new site in secret, showing early "drafts" of the interface to a few friends if they show it to anyone. But Kevin's decided to try a "by the book" redesign of Fury, complete with task analysis, low-fi prototyping, cognitive walkthroughs, usability tests, and log analysis of the current site. He's already identified some portraits of common users which he created by analyzing Fury 3.2's log files and put up a wireframe mockup for user feedback.

I'm particularly interested in seeing how this project goes since it ties in so nicely with my current Usability and Open Source project. Although Fury is not, to my knowledge, open source, Kevin is working in a similar environment as many lone open source developers, e.g., he's working in his spare time, has a 0$ budget, has a strong personal attachment to the interface design, etc. He's also a much more skilled and experienced interaction designer than I am, so it'll be neat to see what he comes up with.

I hope Kevin continues to keep us informed on how the project is going and how he's adapted his process and techniques to the job (which, given his voluminous posting history, I'm not too worried about). I also hope he doesn't get bored of the project before its completion (which, by his own admission, is a real possibility :).

[ Permanent Location ]

Prescriptive Interfaces Explained

design, processes & methodologies, usability

May 27, 2003, 02:16 PM

A couple days ago I mentioned that I was interested in "normative interfaces" as part of the single-sourcing paradigm idea. I've thought it through some more and decided I liked "Prescriptive Interfaces" better as a monkier (best practices frequently aren't very normative since they're often not the norm...) and that I'd like to expand on some of the points I made in that post.

In a way, the term "prescriptive interface" is a tautology; all interfaces are technically prescriptive since they necessarily predefine some method of interaction with the system. The interface defines what commands the system understands and what form it can present information in, and thus any interface dictates some method or range of methods the user must use to perform their tasks and accomplish their goals. Techniques like CI/CD help us design interfaces that prescribe methods of interaction the user is already familiar with through their existing work, while improving on the existing methods by fixing the breakdowns identified by the design team.

The interesting question, however, is how do we design interfaces that encourage users to work in the smartest way possible, as defined by the techniques developed by long-time experts in the field who have applied them successfully in many different situations. In the software engineering world, these are known as "best practices". One best practice in software development is unit testing your code. One best practice in technical writing is writing an outline before starting the document, as I mentioned before. One best practice in HCI is low-fi prototyping your interfaces to elicit real user test data as early as possible.

A prescriptive interface, then, is an interface designed to encourage the use of best practices, to make it easy (or at least easier) to solve the problem in the "right" way. One example of an interface that did this well is the original Apple GUI toolkit. At the time, there were no high-level GUI toolkits available so programmers wrote the code to implement all their own widgets, thus the buttons, text boxes, menus, etc. in every application looked different, which proved confusingly inconsistent to the user. Apple's genius was in providing a toolkit that not only enforced a consistent look and feel on their platform, but actually made the application programmer's job easier as well. This encouraged programmers to use Apple's toolkit instead of rolling their own, and thus Apple wound up with a remarkably consistent-looking and usable platform for the day. An example of an interface design that does this poorly is Microsoft Word's styles support. Writing a document using styles is hard; the user must first know that styles exist and what they are for, then must go into the styles dialog to edit these styles to make them look the way they want, consistently apply them where they are appropriate, etc. The interface makes it much easier to just hit the "Bold" button up in the toolbar to define a heading than to try to do it with styles.

Ideally, prescriptive interfaces should make it easier to solve problems the right way than it is to solve them the wrong way. Failing that, they should try to make the right way as convenient as possible, and may even consciously make the wrong way just a little bit more difficult. If done right, this approach has potential benefits for both novice and expert users. The interface helps novices learn the best practice approaches to solving their problems in the context of doing their work, which is much more effective than formal training. The interface helps experts by making the best practices they already know much easier to follow; even though I know styles are an overall better approach I frequently don't use them in Word simply because it's such a pain in the butt.

So now that you're convinced that prescriptive interfaces are a good idea (I hope), the question becomes, "How do we design these things?" Here's my thoughts on a rough process:

1. Identify the best practice approach to solving the problem. This may require reviewing relevant academic or trade literature in the domain, but I would guess that the most important practice would be to ensure you have one or more domain experts on the design team who have experience applying the best practices and can advise your design accordingly.
2. Research how users currently solve the problem in their day to day work by using a requirements collection method like Contextual Inquiry. Merely understanding how users ought to work is useless if you don't understand how they really do work and why.
3. Design your interface to define clear transitions and adaptations between how users currently work and the best practice solutions. The interface may need to have a sizable teaching component; collaboration with the learning technologies people may help with this part. Unfortunately I don't have much more guidance to provide here; this is definitely the hard and unexplored part of this technique.
4. Allow users to circumvent your best practice processes, but discourage this as much as possible. Be mindful that your ideal solutions may not fit all situations, but make sure the users don't decide to deviate from the best practices idly.
5. As always, do lots of user testing to make sure it works! Testing is particularly important when you're trying to impose a new way of working. Your assumptions will probably be wrong at first; they'll need to be tested and refined even more than normal interface designs before the concepts are nailed down. Low fidelity testing is probably particularly important since drastic changes in the design may be necessary to fix the breakdowns.

So those are my thoughts as they stand. In conclusion, the goal of prescriptive interface design is to produce products that afford best practices within their domain just as they afford proper use of themselves. A good prescriptive interface builds the experience of experts into the system itself so that even novices can experience that benefits of expert-level productivity "out of the box".

[ Permanent Location ]

Single-Sourcing, Outliners, and Normative Interfaces

design, processes & methodologies, usability

May 25, 2003, 09:18 PM

I've posted before about the single source paradigm and the separation of presentation from semantic content. I've been working with Word's outline mode a lot recently and have started to see how using an inherently structural view like an outline to edit content could be the key to providing a usable presentation-agnostic content management system like I described before.

When you create an outline for a document, a presentation, etc. you are basically considering only the structural and semantic aspects of the work. Your purpose is just to lay out the content and get a sense of how your thoughts decompose into supporting points, how your arguments fit together, etc. After you've finalized the outline, you'll worry about making sure the document flows well and looks good by defining the appearance of headings, putting in figures and messing around with how the text flows around them, changing the pagination, etc.

The important part about this process is that the entire time you're focused only on the structure of the work, deferring the definition of the presentation until later. And this is an example of a natural human workflow that people engaged in long before they had computers, so it makes a nice metaphor for the computer-supported single-content paradigm. So the question becomes, can we design interfaces that build on this metaphor to encourage semantic definition of documents, rather than only providing a WYSIWYG style interface that encourages tying content to presentation? And can this metaphor be extended even to such vastly different presentation decisions like whether to publish a printable document, a web page, or a slideshow presentation?

When I brought this up to Micah, he mentioned that Dave Winer, the creator of Radio Userland, has been interested in outliners and the single-source paradigm for years. I'll have to set aside some time to read Scripting.com to see what his thoughts on the subject are.

This whole concept ties into another area I'm interested in; how do we as software designers develop interfaces that encourage our users to follow predefined best practices? Basically, the idea is that certain workflows can be inherently more productive than others, but users may be following a less-than-optimal work flow in their current practice. For example, most technical writers agree that creating outlines before working on a final document produces better-structured prose than just sitting down and writing (I even try to sketch out "mini-outlines" when I toss together these weblog entries). But many users of Word don't do this; they just sit down and start typing. How can we encourage people to follow the superior practice by making it easier than the alternative? In essence, how can we design "normative interfaces"?

Most HCI techniques help us get at the ways our users think and determine what would be intuitive for them, but don't help us design interfaces that help teach users a better way of working. Solving this problem may prove to be an important component of getting people to think structurally so the full benefits of the content-presentation separation paradigm can be realized.

[ Permanent Location ]

End-User Participation in Open Source Development

processes & methodologies, software development, usability

May 16, 2003, 03:57 PM

I talked to Jim Herbsleb today about my idea for doing an independent study on Open Source and Usability. He was interested in my ideas and provided some great feedback, so I think the project is a go.

I talked about the ideas I've already mentioned in my earlier post and added Andy's comments about usability logging and feedback. Jim mentioned a system he'd heard of at Bell Labs that was similar to the usability talkback idea Andy posted about awhile back. He thought this would fit well into an open source development process since it would involve the users more in the development of the application. He suggested I consider usability techniques that would take advantage of the unique qualities of open source development, such as open development processes and user participation. Specifically he wondered if there was a way to involve nontechnical users in the development of the software system so they could feel that they were involved with the project and contributing to its advancement.

I think this is a great idea and something I hadn't thought of, but it's also a difficult challenge. The main obstacle to getting real "end-user" participation in the open source development process is that developers tend to have all the power in an open source process. As Scott says, "He who writes the code gets the last word". Developers tend to fix their own problems, and since end users can't write code, their needs may wind up at a much lower priority. However, there may be something to learn from the participatory design tradition which has always made user involvement an integral part of the usability process. Maybe some of the incentives present there will also apply to open source.

These are just a few thoughts; I'm flying blind on this issue right now. Any comments or feedback that could help lead me in the right direction would be much appreciated.

[ Permanent Location ]

Drawing Work Over Time

processes & methodologies, teaching & learning, writing & communication

May 11, 2003, 09:02 AM

For our TCinC work, Matt and I are interviewing the students who just completed the course to get a picture of how they worked, what they learned, etc. We plan on performing this exercise again after Joe incorporates our redesigned lesson plans into his curriculum to hopefully show that our changes brought about real improvement.

I'm currently trying to develop a notation for mapping out what the students did through the course of the semester, what information or materials they used to do it, and what they learned at each phase. For our initial contextual inquiries, we built the "five faces of work" diagrams from Contextual Design. But for our problem, we found the diagrams contained a lot of repeated information and split important aspects of the students' experience across diagrams, for example, we would like the interaction of people and documents from the Work Flow diagram to be mixed with the sense of linear work-over-time from the Sequence diagram.

What we basically want to know is whether the concepts Joe teaches in class were used by the consultants at site, and how the overall process could be streamlined and improved both to facilitate the students learning and improve their work with their community partners. Thus, our diagramming notation needs to express the following concepts:

1. What students did at each stage of the process (class periods, meetings with their community partners, etc.).
2. What knowledge they internalized at each stage of the process and how this knowledge changed over time.
3. What artifacts they created and how these artifacts changed over time.
4. What opinions and thoughts they had at each stage and how these changed over time.

To try to address these issues, I've developed the following notation:

Since real user data is confidential, this is, of course, a mockup of a real diagram.

I'm hoping this diagram will address these issues at a glance. (1) should be addressed by the student activities which are mapped onto a timeline (you can even make the activities wider to indicate the relative lengths) which comes from the Sequence models. (2) and (4) should be addressed by the "thought bubbles" which come out of activities and can influence future activities, as well as morph into other thought bubbles as the student's understanding changes. (3) should be addressed by the document boxes that can be produced and used in activities. People and outcomes are important to include to see who was influencing the students and what they actually accomplished at each meeting.

I am concerned that cramming all this information onto a single diagram will make it appear cramped and difficult to use to get an overall picture as intended. But if I split some of this across multiple diagrams, I worry we'll have the same problem we did with the CI/CD models where a lack of a consistent process view made it hard to get a handle on the whole system being studied. I'm also concerned that comparing the "before and after" diagrams at the end of our studies will be less effective with something this complex; the improvements may not jump out at you as much as they would with a simpler notation.

[ Permanent Location ]

Bringing Usability to Open Source Software

processes & methodologies, software development, usability

April 23, 2003, 12:07 AM

I'm thinking of doing an independent study during the summer on the topic of "Improving the Usability of Open Source / Free Software". It is widely realized among open-source software users and the more level-headed developers that the usability of most open source software products is abysmal. Unfortunately, the response of many open-source advocates is to stick their heads in the ground, but even those who are seriously looking into the problem have yet to propose a sound human-centered open source development process that has a chance of actually getting used by J. Random Hacker (at least, no one has that I've heard of).

Basically, I'd like to write a "Usability HOWTO" that's based on some actual experience running a human-centered open source project. I have a few ideas to start with:

1. Developing and using scenarios and user personas as a way of encoding a shared perspective of the target user population among contributers to the project.
2. Using inspection methods like Heuristic Evaluations and Cognitive Walkthroughs periodically on the user interface and filing usability "bugs" in the project bug tracking tools.
3. PUI-style user testing where contributors go out into the world and test their interfaces on people who belong to the target user population of the project, then bring the results back to the other contributors to advise the interface design.

To flesh out and validate these ideas, I want to run a small open source project of my own. I was thinking of doing a server-side RSS News Aggregator since there don't seem to be many good ones in the open source world. But anything would do as long as usability is critical to the project.

I haven't decided for sure whether I'm going to do it or not; I'm not positive I have the right expertise. But I do think it's an important problem that needs to be solved, so I may start casting around for a faculty sponsor.

If anyone out there has any thoughts or suggestions, they'd be much appreciated as always.

[ Permanent Location ]

Stories as Pictures of the Users

processes & methodologies, usability

April 18, 2003, 01:45 PM

For my work on Usability and Software Architecture, Bonnie and Len want me to design and build a website to help disseminate our technique to professionals. In our meeting last Tuesday, Bonnie asked if I knew who the users are. I said, truthfully, that I didn't really. She replied, "Then you can't design a website."

She suggested I try writing up some personas instead of the more time intensive Contextual Inquiry process she'd taught us in Methods. So I asked Micah, who has read "The Inmates are Running the Asylum" to tell me a little bit about them. I decided Alan Cooper's full process was probably too heavyweight for this task, so I sat down to write up some portraits of our expected users based on my own understanding.

As I've thought about the high-level design of the site, I've expanded these profiles into slightly more extensive "stories of use" that describe the problems my stereotypical users will experience and what they're going to need to solve them. Of course, all I was really doing was recording and making explicit my own understanding of our users (although I've found this in and of itself is valuable); I had no evidence that my assumptions about their needs and problems were correct. So as a start, I've been asking a few of my colleagues who more or less fit the profiles to read over my scenarios and give me feedback on how realistic they seem based on their experiences. I've realized that I'm starting to follow my own Scenario-Based Design process, which makes me want to read Rosson & Carroll's "Usability Engineering" that's been sitting on my shelf since last September.

What's even more interesting about this exercise is that I find it's not just feeding into a website design, but it's giving me a better understanding of the needs of the people who will (hopefully) be using the U&SA technique. If I can sell this idea to Bonnie and Len, then I believe these profiles could be very helpful to us in evolving our understanding of our users and keeping us grounded in reality. If we want to make changes to our technique and one of us says "Wait a minute, Bob isn't going to like that change because..." then I think our discussions could become much more practical and user-focused.

I've really become sold on the power of stories as a means of driving design (and not just design of interfaces). When I first encountered scenario-based design, I'll admit, I thought it was a silly idea ("You expect people to take me seriously when I'm making up stories about people using our system??"). But since then I've seen the pragmatic value and unparalleled communicative power stories possess; people can share your vision of the system when you tell them a story in ways they can't when you just list functional requirements (or even show them screen mockups). Stories keep your thoughts concrete even as you envision that which does not yet exist. I'm beginning to believe that no design team can be effective without them.

[ Permanent Location ]

An Affinity for Thinking

personal, processes & methodologies

April 15, 2003, 10:22 PM

As part of my website redesign (which will be coming any day now! Really!) I'm attempting to come up with a good set of categories to help organize the content of this here weblog you're reading. I tried brainstorming a list of my interests and stuff I'm likely to want to post about, but the list quickly grew to unmanagable proportions. So I decided to try a novel exercise: I built an affinity diagram of my (intellectual) life.

For those of you who aren't familiar with the process, to build an affinity diagram you collect a large amount of unstructured information (could be brainstormed ideas, could be problems users are reporting with your product, in my case it was the list of my interests) write each piece down on a separate sticky note, then post the sticky notes to a whiteboard. When you put up a new sticky note, you place it near other sticky notes that it seems "related" to. When all the sticky notes are placed, you look at the mass o' notes and pull out the larger groups and categories that magically appear from your organization process. The magic comes from helping you to translate a complex cognitive task performed on lots of data (pulling together all that information in your head) into a spatial organization task, which we humans are much better at.

In my case, I brainstormed a list of my interests that I might want to post about. Then I augmented this list by scanning the titles of the books on my bookshelf, the lectures I listen to, the bookmarks in my browsers, etc. to pull out any interests I may have overlooked.

I then went to the whiteboard I have at work (had lots of stickies; needed big board) and started affinity diagramming. One discovery I made as I went along was that every time I'd built an affinity diagram in the past, it would inevitably break down into "bucket sorting" before all the stickies were up, meaning that we'd come up with the categories too soon and just start plopping stickies into the appropriate category instead of considering their affinities with other nearby stickies. This time I carefully avoided doing that, and I think I wound up with a more insightful affinity as a result.

Then I stepped back and started drawing categories. There was a lot of overlap between the groups of stickies, which is good; that means my interests are fairly interconnected. I was also encouraged to discover that technology-related interests, although significant, did not dominate the entire affinity. So it appears that I'm not becoming too narrow in my intellectual foci.

Somewhat less encouraging was how disconnected my "charity" interests were from the rest of the diagram. Also, some of my professed interests, such as playing music, martial arts, and eastern religions, didn't appear or only appeared in small, disconnected pockets. I certainly have more developing to do in some areas.

All in all, I'd say this was an interesting exercise; I gained a little bit of a deeper understanding about myself as a result of the process. I may repeat it again in a few years just to see how the landscape of my thoughts has changed.

I'd recommend giving it a shot if you have an hour or two to kill and a pack of spare stickies. Of course, whether it has actually solved my original problem of developing good weblog categories remains to be seen...

[ Permanent Location ]

CHI Report, Magic Numbers and Cranky Usability Consultants

processes & methodologies, usability

April 10, 2003, 11:43 PM

The past two days were packed, hence the reason this weblog entry is appearing a day late. Yesterday I got up early enough to make the morning session, so I went to a panel on "The Magic Number 5: Is It Enough for Web Testing?". The basic premise of the panel rested on Nielsen's Alert Box assertion a few years back that 5 users is enough to test a single interface; after testing five users you should iterate and test again or you are just wasting your time since you'll start finding mostly the same problems over and over again. Jared Spool, Gilbert Cockburn, and Rolf Molich (Nielsen was supposed to attend, but he was ill and did not come to CHI this year) disagreed with this claim on the grounds that the number of users you need to test to find a significant number of critical errors varies with the complexity of the interface and the goals of your study and is usually much higher than five. Carol Barnum agreed with the number if the discount usability engineering method is followed, and Dennis Wixon from Microsoft was the black sheep who argued that the entire preoccupation with finding "all the errors" was stupid and everyone should use the RITE method.

The debate was spirited and enjoyable to watch, but I got the feeling afterward that the panelists were all shouting loudly at each other in agreement. Everyone seemed to believe that you needed to do highly iterative development to ensure problems found actually got fixed and retested (although Rolf called for a more robust discipline that could get the interface close to right the first time so we wouldn't have to do much user testing; I think Jared's sarcastic comment that "Yes, we can solve this problem right now by always building usable interfaces!" summed up the problems with this approach fairly neatly). Everyone seemed to believe that ultimately you should run as many users as you can to ensure many iterations and a more usable product.

Ultimately, I came away from the talk believing all the more strongly in iterative development while still holding out hope that we can design interfaces to be as initially usable as possible (through CI/CD, etc). I'm also even more convinced of the potential for RITE.

[ Permanent Location ]

CHI Report, Day 1

processes & methodologies, software development, teaching & learning, usability

April 08, 2003, 09:58 AM

Yesterday was the first day of the Computer-Human Interaction conference at Fort Lauderdale, Florida, which is where I'm currently located. Not a whole lot of academics yesterday; after I got in I mostly hung out on the beach and in the pool with the gang. We all remarked that so far this was the best conference any of us had ever been to :).

In the evening hours I went to the U&SA tutorial that Bonnie and Len and I put together. It was interesting to see the reaction of professionals to our material; so far I had only seen the tutorial presented to students. All in all I think it went well, although Len was concerned that people didn't seem as excited about the material as they were last year. The tutorial attendee's comments also confirmed some thoughts I'd had on the major problems with the tutorial:

1. There was some confusion on what the distinction was between the scenarios and responsibilities (which are supposed to apply to all systems) and the sample architectural pattern (which may or may not apply to any given system).
2. Before Bonnie gave an in-depth explanation about our Benefits/Tactics matrix, some people didn't understand what process we were advocating for applying these scenarios (should every system implement every scenario, or do you need to decide which ones make sense for your system?)
3. One person raised an issue about reusable code support for the scenarios, which bears out my thought that people may like pointers to toolkits and frameworks that help implement the scenarios.
4. Someone mentioned that we should look at existing patterns in the Gang of Four's book, etc. to see if there are more general solutions to our patterns. I'm kind of kicking myself here, because I've actually already done some preliminary investigations in that area and neglected to mention it. I had a chance to look smart in front of people in my field and blew it. But it is good to have confirmation that this is a good path to go down, although I haven't found many patterns that directly apply to our scenarios and am beginning to think the right way to do this would be to investigate existing solutions in working architectures and generalize from them.
5. Someone was concerned with how we come up with the scenarios and how we distinguish between an "architecturally-sensitive usability scenario" and a "functional requirement". I have a simple criteria that we could use: has the scenario caused a "We can't change THAT!!!" situation?
6. Someone also raised the issue of how we could test to ensure these scenarios are implemented in an architecture. This I actually hadn't thought of, but certainly appears important.

In other news, I went to the networking event after the tutorial. Talked to several people, many of whom I already knew, but some new faces too. Then several of us went out to eat around midnight. I'm having a lot of fun; it's great to be here in Florida with beautiful weather at a top-notch conference in my field. And the best part is I'm here with some of the best people I've ever known. What could be finer?

[ Permanent Location ]

The Elegance of Simple Techniques

processes & methodologies

March 28, 2003, 11:21 AM

One of the approximately ten thousand things I am interested in is process or methodology design, that is, how do we come up with procedures and techniques that people can follow to help them do their jobs more effectively. I've been recently exposed to a couple of structured brainstorming techniques that are remarkable not only because of their effectiveness but also because they're easy to teach, understand, and follow. The first of these is affinity diagramming which we learned about last semester in Intro to HCI Methods as part of Contextual Design. The second is causal flow analysis, a technique for helping to brainstorm problems, opportunities, and project ideas based on a defined mission objective that I learned from Matt.

What's worth mentioning about both these techniques is that they're both powerful and extremely simple. I could teach both of them to you with 10 minutes of explanation and one practice session. Because of this they're highly accessible; many people from different backgrounds can come together and participate in brainstorming sessions using these techniques and contribute their expertise without having to learn a whole new way of working. What this also means, I believe, is that they may actually get used, unlike a lot of other methodologies I can think of (*cough* CMM! *cough* *cough*).

I suspect that there is a great deal of utility in keeping a process simple. Many people raise true if rather pedantic objections on how the process doesn't cover all the possible cases (academics are often guilty of this I've found). "But your programming methodology relies too much on the skill of your people! It's not repeatable!" "How do you know the work has been done properly?! Your process isn't sufficiently visible!" I think this is missing the point. There's a 90-10 rule at work here; I can feel it. A good process needs to find the simple solution that solves 90% of the problem, then rely on human skill for the other 10% if solving it directly complicates the whole procedure.

Now if only I had a simple process for defining how to create a simple process...

[ Permanent Location ]

© 2003-2010 Rob Adams ()

Last Modified: Wednesday September 17, 2008

Unless otherwise indicated, all original work appearing on this web page is licensed under the Attribution / No Derivitive Works Creative Commons license. Please before using any page content in a manner not covered by this license.