 |
« | Week of Nov 2 | < | Individual Entries | > | Week of Nov 16 | » |  |
 |
 |
Discussing Disconnects
design, processes & methodologies, software development, usability
November 15, 2003, 07:08 PM
At the SSS two days ago, I led a discussion on how we can bridge the gap between interaction designers, usability professionals, and software engineers working together on real-world development teams (as I promised I would). I started out with a quick presentation describing the personalities and goals of both "players" (designers and developers) through some rough personas, then opened the session up for discussion on what techniques might help in getting these two players to work together effectively.
Here's a list of the possibilities that were thrown out for consideration. If anyone out there in readerland was at the discussion and notices something I missed, please do add it in a comment.
- Emphasize more up-front design so design and engineering need not work in parallel. The idea is that this will reduce the amount that the two players need to communicate, thus reducing conflicts and making the process more efficient. It was also pointed out, however, that although spending time on up-front design is good, it doesn't solve all the problems. The final design, no matter how good, will still need to change quite a bit based on engineering's input and concerns, thus reintroducing the communication problem.
- Design must bring engineering onboard. Those who came from an engineering background emphasized the need for design to justify and "sell" their design solutions to the engineering team. The idea is that although engineering trusts design to be good at their job, they will be more effective participants in the overall development process if they understand what the important usability issues are and why the related design decisions were made.
- Design and development must "live" together. Basically, the two players should work closely throughout the entire process; there should be no "throw it over the wall" style information exchanges. I've suggested this before in the context of XP, going so far as to recommend that the two players work together in the same room.
- Designers should contribute to development work. Many designers have enough technical abilities to contribute to parts of the user interface development process. In addition to speeding things along, this gives engineering a better sense that the designers are true members of the team. Cooper and Reimann suggest that one individual cannot fill the role of a designer and developer, due to the fundamentally different mindsets, but everyone in the group thought it was possible to strike a happy medium. In order to facilitate this technique, the software architecture must separate the user interface modules from the rest of the application using a separation pattern such as Model-View-Controller (which still won't handle all concerns, of course).
- Designers should have engineers sit in on user tests. Watching users struggle with bad designs is often cited as a good way to motivate people who aren't willing to buy in to user-centered design. If this is the situation designers face, this is a technique they can try to bring engineers on board. Randy Pausch, ever famous for mistreating his engineering team, requires them to sit on their hands while watching people use their interface, and demands they keep silent unless they are willing to precede their comments with an apology for designing such a horrible interface. This might be a bit extremist for most industry teams to swallow, however...
- Both players need case studies on how to effectively work together and how not to ineffectively work together. Design and engineering (and management) can learn from examples. Even experienced industry veterans may not associate usability failures with market failures, since these two events are often separated by a long period of time, and humans are bad at associating effects that occur long after their causes. Of course, someone must put together a collection of these case studies before designers and engineers can learn from them.
Perhaps the most interesting part of the conversation, however, was the discussion of the benefits of "creative tension", or those times when design and engineering don't get along yet wind up with a solution that is better than what they would have come up with if they had. Haven brought up the case of the Harley-Davidson VROD, a case study where there were several examples of the designers putting their foots down and demanding that engineering figure out a way to implement their design, as well as examples of the engineers demanding that the designers come up with workable designs. In the end, this resulted in a better product than Harley-Davidson would have come up with had the two sides come up with a compromise solution.
This raises the question of whether it is necessarily every desirable for design and engineering to get along smoothly, or whether a certain amount of tension and controlled chaos is sometimes necessary to produce a quality solution.
Email Rob:
Radical Colocation is a Good Thing
processes & methodologies, software development
November 15, 2003, 03:32 PM
Joel has an article up on the subject of adopting methodologies for software development. He makes a lot of good points about the state of the practice, but when discussing the subject of whether software development should occur in private offices or public collaboration spaces, he makes the following claim:
But we don't have the data. We don't have any data. You can give us anecdotes left and right about how methodology X worked or didn't work, but you can't prove that when it worked it wasn't just because of one really, really good programmer on the team, and you can't prove that when it failed is wasn't just because the company was in the process of going bankrupt and everybody was too demoralized to do anything at all, Aeron chairs notwithstanding.
This isn't quite true. Earlier this year I came across an article that came from Judy Olson's research group on this subject called "How does radical collocation help a team succeed?". It's an amazing piece of empirical software engineering research that gives some hard data to back up the claim that putting everyone in the same room together increases productivity in software teams. Due to copyright concerns I can't repost it here, but you can find it in the ACM digital library if you're a member. Here's the abstract:
Companies are experimenting with putting teams into warrooms, hoping for some productivity enhancement. We conducted a field study of six such teams, tracking their activity, attitudes, use of technology and productivity. Teams in these warrooms showed a doubling of productivity. Why? Among other things, teams had easy access to each other for both coordination of their work and for learning, and the work artifacts they posted on the walls remained visible to all. These results imply that if we are to truly support remote teams, we should provide constant awareness and easy transitions in and out of spontaneous meetings.
In short, the article found that, in the situation under study (small teams, fixed time frame), radical collocation drastically increased the productivity of the software teams, as measured by a function point count that they compared to the count for the company's standard cubicle setting as well as to a whole industry baseline. They also found that although many of the participants were reluctant to work in the collocated environment at first, they came around by the end of the study. Not surprisingly, they did find that certain tasks with high cognitive demands were hard to perform in the collocated environment due to constant distractions, so they recommend including private "hoteling" areas for such tasks as well as for placing personal phone calls and the like.
The actual article is worth a read if you can get to it, especially since they conclude with recommendations for developing effective collocated and remote work environments based on their findings. And if you're a researcher, the study is a great model of the direction the field of empirical software engineering should go in if they wish to provide the kind of hard data that Joel correctly points out is sadly lacking in the current state of the science.
Posted by Rob on November 19, 2003 at 02:17 PM
A quick correction: the final version of these research was published in IEEE Transactions on Software Engineering, Volume 28, No. 7, July 2002, under the title "Rapid Software Development through Team Collocation". This version contains more detail on the findings as well as more discussion on the conclusions than the article I initially mentioned.
Email Rob:
Storytime and Pretty Pictures: Data That Informs Design
design, processes & methodologies
November 11, 2003, 11:47 PM
One of the things I'm interested in is studying how to guide design through empirical research, "science-guided design" you might call it. The problem with classical research methods is that although they are effective at uncovering truth, they are too general and reductivist to guide design by themselves. To fill this void, methods have arisen in both usability engineering and interaction design that are intended to help development teams distill fieldwork findings into forms that help designers creatively develop interfaces that satisfy user needs as well as test their interface assumptions against their understanding of the state of the world. Although there are many techniques to fill this role, all of them fall into two general forms. Very roughly, these two forms are "telling stories" and "drawing diagrams".
The "telling stories" approach involves distilling interview and observation data into descriptions of the target users and scenarios of them performing their tasks with the system. The most well-known technique that employs this approach is Goal-Directed Design, developed by Alan Cooper. Cooper's technique involves developing personas, or detailed descriptions of "prototypical" users that incorporate their personalities, goals, skills, etc. Cooper emphasizes that the persona descriptions must be realistic and believable; the team must envision these imaginary people as if they were real users of the system whose needs must be satisfied. Other techniques, such as Scenario-Based Design, take a similar approach but focus more on developing stories of users interacting with the system. But all techniques that follow this approach focus on developing natural language stories about the people who must use (and therefore are a part of) the system.
There are several advantages to this approach:
- Stories make it easier to understand the real human costs and benefits for a solution. Abstract numbers are important for understanding aggregate effects, but they are notorious for hiding away the true events that effect the lives of the people behind those numbers. The "telling stories" approach helps provide this level of detail. This is especially essential for designing desirable products, since desirability is very much a function of individual users' emotional responses to the product.
- Detailed descriptions of users brings focus to design. It helps you define clearly "Here are the people we are designing for. We must always check our designs to ensure that we're convinced they'll make these people happy and meet their goals." You can't do this with vague notions of what "the users" want, and its very hard to do this with purely numerical data (how can we know if we're meeting the goals of all these numbers?).
- Stories are easier to understand. Describing a scenario to people outside the design team (such as management or clients) is generally a much more effective and compelling way to present a problem or a design solution than specifications, diagrams, or screenshots. Explaining and selling your designs is very important, so representing your research data in this fashion helps leverage this fact.
- Stories are "fuzzy". This may or may not be an advantage depending on who you are dealing with. Designers, artists, and end-users may appreciate this quality of these techniques. Engineers, scientists, and (some) managers may not. Know your audience.
- Scenarios can be translated to system responsibilities. If you distill your stories down into specific scenarios, then these scenarios can be distilled, in collaboration with engineering, into system responsibilities that guide implementation efforts. For instance, in the popular UML notation, system requirements are represented as use cases, which are natural derivatives of scenarios of system use.
The "drawing diagrams" approach involves representing data collected during interviews and observations as diagrams and structured prose that describes pictorially the events, interactions, environments, etc. that you encountered. The method that defines this approach best is Contextual Design, which calls for distilling inquiry data into the "five faces of work" — five diagramming notations that describe the work flow, culture, physical layout, artifact usage, and sequence of tasks for every user observed. Beyer and Holtzblatt, the developers of Contextual Design, emphasize that these documents capture the relevant information about the world to guide design, and provide an entire methodology for transforming these diagrams into an actual interface design. Unlike Goal-Directed Design, this process guides you through each step of the interface development life cycle.
There are more techniques that employ diagramming than Contextual Design, however. Matt introduced me to causal flow analysis and systems thinking last spring, two techniques that employ diagramming notations to map out complex reasoning paths and identify patterns and breakdowns in organizational systems, respectively. Although their purposes vary, all these techniques employ diagrams for the same purpose — to focus attention on the aspects of the data that the methodology designers decided was important and needed to be explicitly represented.
There are a number of advantages to using diagrams:
- Diagrams help you understand the whole system. Pictures like this give a more holistic view of the state of the world than stories do, since stories tend to focus on individual people. Natural language often falls apart when you try to use it to describe abstract concepts like organizational systems or the structure of communication in a work environment. A good diagramming notation is much more effective for communicating this information at a glance.
- Diagrams get at data and structures that aren't obvious from descriptive prose. Many important concepts are hidden in plain language, if they appear at all. For example, oftentimes an argument may sound convincing when you read it in prose, but when you draw a causal flow analysis or apply another argument mapping technique, you quickly see the holes. Diagrams help for cases like these by highlighting the relevant parts of the prose and de-emphasizing others.
- Diagrams help you design how your changes will fit into the whole system. It's important to understand that any new system must fit into the larger systems that compose our reality. Diagrams make it easy to guess the results of changes in the systems they represent, since this generally only requires altering the diagram to include the new system.
- Diagrams are perceived as "rigorous". Again, whether this is desirable or not depends on who you are communicating with, but diagramming techniques are generally perceived as more rigorous than storytelling techniques. This tends to go over well with engineers and scientists, and possibly management.
I believe that both of these approaches are important for understanding different aspects of a design problem. Thus, when faced with a choice of which data analysis methods to use, the appropriate question should not be which is better in the abstract, but rather which is more tuned to solving the particular problem the team is facing (often, in fact, the correct answer may be to mix the two and use different techniques for different aspects of the problem).
Finally, I firmly believe that this work has a broader applicability than just interface design. Many, many fields could benefit from decision-making processes that are informed by empirical work. Why, for example, does Congress not charter fieldwork teams to investigate the needs of low-income households before altering welfare laws? Why do we not turn these design processes to systems built of humans, as well as systems built of software or raw materials? The need is there, and the techniques are emerging. I hope the right time for this movement is coming soon.
Email Rob:
SSL Certificates - Unusable and (Mostly) Useless
internet, usability
November 11, 2003, 02:40 PM
Matthew Thomas has an interesting rant about the poor usability of the SSL Security Certificate system, the mechanism pretty much all web browsers, email programs, and other internet clients use to establish secure connections to servers over the inherently insecure internet. He argues that the usability is so poor that the security mechanism is rendered useless in the vast majority of cases, and demonstrates this with a description of what really happens in an Alice-and-Bob scenario.
I tend to agree, and I can vouch firsthand for the absurd level of difficulty involved in procuring and setting up a certificate from a major certificate authority, since I had to perform this very task for my last job. It took me hours, and required a high level of technical sophistication. Had our security requirements not been so high, I would likely have given up and gone back to my programming tasks, which were already behind schedule.
SSL certificates are just one example of a system that has been studied and engineered to death, but has stopped just short of a usable, human-centered solution. By failing to take that step, we've effectively destroyed all the benefits the system was supposed to provide. I've pointed out that all software quality attributes, including security, are related to usability before. Although there's been some work on designing usable security systems, this area is still sadly lacking in answers. How many compromised systems, virus debacles, and internet-smashing worms do we need to suffer through before someone decides to do something about it?
Email Rob:
Usability and Creativity
aesthetics, software development, usability
November 10, 2003, 02:55 PM
At the last SSS, MHCI student Jesse Kriss gave a talk on digital music and live performance, focusing on the technologies available for changing the way musicians perform. The important feature of all these technologies, according to Jesse, is that they separate the instrument's input from its output. With a regular instrument, like a violin, the form of the instrument is more or less dictated by the sound you want it to make. You can't make a violin with a substantially different form that is played in a substantially different way without changing the sounds the instrument is capable of making. But with digital technology, the "input" (the way the instrument looks and is played) and the "output" (the sound the instrument produces) are entirely separate. This opens up whole new possibilities for instrument design.
While listening to Jesse's explanation, it struck me how similar these concepts are to software architecture patterns like Model-View-Controller, which emphasizes the separation of input, output, and processing responsibilities into three separate components. MVC aims for this separation for exactly this reason; many types of systems require frequent changes and recombination of the input, output, and processing components, and separating them into three distinct modules helps facilitate this change. Jesse also mentioned how important it was to be able to plug together many different devices to build the kinds of instruments he envisions. He prefers Macs because they make plugging things together easy by requiring a minimum of setup hassle. It's easy, of course, because the Mac device architecture supports device discovery and auto-setup very nicely; more evidence of the impact of architecture on usability.
But it's more than just plugging together hardware. Jesse's work also involves reconfiguring that hardware to map behaviors to sounds. The software he uses to do this allows him to build processing capabilities using audio filtering components and logic controls. Essentially, this is a domain-specific programming language that reminded me of LabView's G. The flexibility and complexity of digital audio technologies is astounding.
All this got me thinking about a question I've pondered before (and in the context of digital music, no less): how do we design interfaces to support creative tasks? As I discussed in the previous post, classic user-centered design techniques focus on designing for the user's goals by distilling these goals into tasks that the interface can support. But the very nature of creative endeavors ensures that the users' tasks are difficult to define; after all, creativity by definition involves doing things that have never been done before.
Bill Fulton, the head of Microsoft Games, gave a talk here at CMU last fall and was asked this very question. His answer was that he was no expert, but that he suspected the interface should just "get out of the way" and let the creator work as directly as possible with their medium. This is good advice as far as it goes, but I suspect it's more complicated than that. After all, many programs that support creativity have some of the most complex interfaces available on the market (think Photoshop, sound editing tools, and programming IDEs). Generally speaking, however, the user populations for these tools are willing to put up with this complexity (to an extent) for the power and flexibility that comes with it. Is there even a user-centered design problem here?
I hypothesize that there is, and that we can make these interfaces better by thinking about those aspects of an interface that are essential to creativity. In general, I suspect supporting creativity is related to supporting exploration and problem solving, two aspects that are relatively well understood in interface design. Here's a few rough guidelines:
- Make plugging things together easy, both in hardware and software. This involves building software abstractions that are robust enough to support current modules as well as future ones, and that can hide the technical complexity of configuring hardware and software interfaces to work with one another from nontechnical users. Plug-and-Play (PnP) technology has partially solved this problem already, and upcoming protocols like Jini, Rendezvous, and UPnP will hopefully take this even further. What we lack is usable equivalents for software components. Some component frameworks like J2EE try to tackle this problem, but we still have a long way to go.
- Make problems visible and obvious. When users get creative, they tend to make mistakes like configuring the system in ways it wasn't meant to be. The system needs to provide users with sufficient feedback for them to evaluate the problem and correct it. Even better, create interfaces that make the state of the system continuously visible so users are able to diagnose potential problems before they even occur.
- Make it easy to undo mistakes. Tog's guidelines on explorable interfaces, as well as Nielsen's heuristics and several other sources, emphasize the importance of creating an environment where users feel safe exploring by allowing them to back out of any operation they choose by accident. This is especially important in interfaces to support creativity, where almost every action the user takes is exploratory. In fact, many interfaces may benefit from a more sophisticated version of this feature than the ubiquitous "Undo" menu option. Programming IDEs have long offered integration with full-blown change management systems; perhaps its time for other applications to take a cue.
- Allow for data sharing across applications. I brought this up in my earlier rant. Creative users may need to step outside the bounds of your application for some of their tasks, no matter how well designed your system is. Support copy & paste and import / export functionality.
This is, of course, just a first attempt. More thinking and research needs to be done to generate a list that can even resemble completeness.
It's worth noting that all of these features may impact the software architecture design of the system. See our U&SA technique for further information, especially the scenarios on Maintaining Device Independence, Supporting Undo, and Reusing Information.
Posted by Viswanath Gondi on November 10, 2003 at 05:25 PM
You might find my article on "Making Rich Web Applications Usable" interesting. http://blogs.law.harvard.edu/vgondi/2003/09/28#a494
Posted by Viswanath Gondi on November 10, 2003 at 05:28 PM
You might find my article on "Making Rich Web Applications Usable" interesting. http://blogs.law.harvard.edu/vgondi/2003/09/28#a494. It is written on similar lines.
Email Rob:
Posted by rob shorrock on November 15, 2003 at 08:34 PM
Agree with your comments apart from the second to last paragraph. If "design and engineering don't get along" then you will almost certainly be looking at a dysfunctional situation. The instances where it works will be far fewer than those where it doesn't. Certainly, a little tension and controlled chaos is valid but my experience is that teams are MUCH more productive when they are working well together.