psychology

About

Syndication

Categories

Archives

Nielsen: Qualitative Is Better Than Quantitative

design, psychology

March 11, 2004, 03:44 PM

Jacob Nielsen has posted an alertbox column claiming that qualitative user studies are preferable to quantitative user studies in most circumstances. His reasoning is that quantitative studies, while useful for producing metrics that can guide decision making and resolve design arguments, are too difficult to conduct properly (thus leading to biased results) and tend to produce results that are so narrow as to be useless for guiding design.

My friend "Q" (she posts anonymously, which is why I'm not disclosing her identity) takes issue with Nielsen's proclamation. She tends to favor quantitative research, and points out that it's possible to to botch qualitative results as well, and that a good research program should include both quantitative findings and qualitative observations.

I agree that both kinds of studies need to be done carefully and by people with some training in the area. I also agree that it's good to be flexible and consider many types of studies depending on your research goals. However, I can see Nielsen's point; remember that he is not writing for psychologists or social science researchers, he is writing for practitioners in the design and usability fields. Most of these people do not have significant training in statistics, experiment design, or psychology theory and are unlikely to successfully design a valid quantitative experiment. Moreover, my experience with designing quantitative studies is in line with Nielsen's observations; it took me the better part of last semester to design a study intended to provide evidence that personas helped lend focus to remote design teams in open source software (note I said "provide evidence" not "prove"). And by the end, the study was so narrow in its focus that it wasn't at all clear that it could stand up as bulletproof evidence that remote design teams should always employ personas.

Of course, as Q points out, it's possible to get bad qualitative results as well. But I've found that its fairly easy for relatively untrained individuals to uncover useful design facts about their target user population merely by watching a few members of that population do their work, or by talking to them in an interview, or by giving them some tasks to perform with a prototype interface design. Of course, sometimes you do find users that exhibit outlier behavior, but often, as Nielsen says, it's easy enough to identify such behavior by forming hypotheses beforehand and reasoning about how the behavior you're seeing fits in with your understanding of the world.

Bottom line is, even in enlightened companies a user-centered designer is likely to have only a little time (a few weeks, maybe) to run user studies. To make the most of that time, it's usually best for this practitioner to focus on broad, qualitative, fuzzy data to gain a general picture of the problem, rather than focusing in and losing all that useful information that could be guiding design just to get a few numbers.

[ Permanent Location ]

The Quality of Consumption

psychology

February 28, 2004, 07:52 PM

I read an article found via Neema which references psychological studies that found that on average people appreciate experiential consumption more than material consumption. By experiential consumption, we mean those purchases that are made with the intent of acquiring some sort of life experience, such as traveling to a foreign country, going to a concert, or going out to a bar with friends and having good conversations. By material consumption, we mean those purchases that are made with the intent of acquiring some new artifact, like cars or cameras or clothes.

The article makes a few dubious claims in its analysis, but this fact alone is interesting; like Neema, I found it validated my own goals and values. I've sought to enhance my life's experiences over the amount of stuff I own; one of my goals for the future is to increase the quality, quantity, and diversity of these experiences.

It's interesting to reflect on the implications of this research for designers. If we hope to craft products that improve people's lives, are we kidding ourselves by believing that any new "thing" can really bring our customers any meaningful happiness? Food for thought, for sure.

It'd be great to see more research investigating what makes people happy. We might turn up other interesting surprises.

[ Permanent Location ]

Sleep Makes You Smarter

psychology

January 22, 2004, 02:03 PM

Mathilde found an article reporting on a study that found that adequate sleep makes you smarter and more creative (in addition to its productivity and health virtues). A quick selection:

Maquet and Ruby both say the study should be considered a warning to schools, employers and government agencies that sleep makes a huge difference in mental performance.

The results "give us good reason to fully respect our periods of sleep -- especially given the current trend to recklessly curtail them," they said.

The culture here at CMU could certainly stand to learn this lesson, as could the culture in many software companies. Frequent all-nighters just don't pay off, in the short term or the long term. Some management methodologies recognize this, like eXtreme Programming. But it's always nice to have scientific confirmation.

I'm thinking of printing out that article and posting it in the space of any project teams I'm on.

[ Permanent Location ]

A Splintered Psyche

personal, psychology

November 29, 2003, 02:57 PM

This entry is private. Forgot the password? Ask the Keymaster!

[ Permanent Location ]

Deos It Mttaer Waht Oredr the Ltteers in a Wrod Are?

language, psychology

October 13, 2003, 11:21 AM

Matt Davis, an actual researcher at Cambridge University, has an interesting page discussing the word scrambling and perception net-meme that was going around last month. The bottom line is that some of the claims in the email are at least partially true, but the central thesis (that we only read the first and last letters in a word) is false. Check out the page for the details, but note that it's designed very badly and you have to scroll past all the word-scrambling-in-multiple-languages stuff to get to the actual analysis.

Found via Andy.

[ Permanent Location ]

Surprise-Explain-Reward

psychology

September 17, 2003, 11:31 AM

Margaret brought up something else I found interesting in the talk I mentioned in my last post. One of her problems was getting buy-in from users that they even should test (remember, they are overconfident already). Engineers and designers often have an implicit theory that "if you build it, they will come" and are surprised (and annoyed) when their great system isn't immediately picked up by the populace. But from a user's perspective, using any new system is an investment that they may or may not choose to make. They must consider:

• The cost of learning the new system.
• The benefit of using the new system (which is often not made very clear).
• The risks of changing their current work practices (like spending lots of time re-entering your contacts into a PDA only to never use it).

To do this, she examined the psychology literature and learned about a theory called Surprise-Explain-Reward. This theory says that in order to sell a new work practice or technique to someone, you need to:

1. Surprise them with evidence that their current practice isn't working as well as they thought. You need to identify an "information gap" or a hole in their understanding that will make them curious. People will want to fill this hole if you can make them see it.
2. Explain to them the information that will fill this hole. Often the best way to do this isn't through direct lecturing, but through self-directed learning where you suggest the actions you want them to take then allow them to discover the solution "on their own".
3. Reward them with immediately perceivable benefits for using the technique. If all the benefits are long-term many users won't see the value in continuing to use the system. People are bad at connecting causes and effects that are separated by a large amount of time, so try to shrink this time down. If many of the benefits are inherently long-term try to provide smaller tidbits to encourage your users to keep going.

A neat idea, which is apparently backed up by empirical research. Something to think about for us inventor-types who are good at coming up with crazy ideas but aren't so good at selling them.

[ Permanent Location ]

Social Loafing, Free Riding, and Online Communities

psychology, society & sociology

September 10, 2003, 11:25 PM

In CSCW / Designing Online Communities last week, we read, processed, and discussed a huge volume of empirical studies literature on the problem of free riding and social loafing. These two concepts describe the same basic phenomenon except that the first is an economics term and the second is a social science term. Some may quibble with details, but essentially both refer to the observation that people tend to contribute less to a common effort when they are in groups than they do to their individual efforts. The presence of others, it appears, causes contribution and productivity to drop. My earlier post about empirical research on free riding examined one of these studies, now I'm going to try to distill something out of all of them.

Examining this issue is important for online communities, since all of them are, to some degree or another, a common, public good. All require some sort of member participation to stay alive; email lists and discussion groups require people to provide new posts, weblogs require the author to add new entries and possibly readers to add comments or send feedback, online games require a number of people willing to play seriously and fair. And an issue that may touch the hearts of many community administrators is that of donations; the question of how to get your thousands of readers to donate those two cents or whatever it takes to keep your community/webcomic/magazine/etc. alive has free riding / social loafing written all over it.

And so we start by asking: when do people loaf / free ride, and when do they not?

Social Science

To examine the social science perspective on the problem, we read a "Karau & Williams", a formal meta-analysis of several empirical studies on social loafing. For the masochistic, the full reference is: Karau, S. & Williams, K. (1993) Social loafing: A meta-analytic review and theoretical integration. Journal of Personality and Social Psychology, 65(4), 681-706. After slogging through the dry prose and statistics, I gleaned the following information from the article.

Social loafing is decreased by the following factors:

• Evaluation potential - if there is a chance that people will be evaluated on their individual contributions to the group, they are less likely to loaf.
• Task valence - if the task the participants are engaging in as a group is considered inherently interesting to them, they are less likely to loaf ("valence" is psychology lingo for an individual's feelings of attraction or aversion to a particular object or event).
• Group valence - if the participants feel a positive attraction or a sense of loyalty, commitment, or camaraderie towards the group, they are less likely to loaf.
• Opportunity for group evaluation - if the group's performance will be evaluated at a predetermined point, that group's participants are less likely to loaf.
• Individual is unique - if the individual feels they are the only one in the group that can provide a particular service or complete a particular task, then they are less likely to loaf.
• Female sex & Eastern culture - Easterners are less likely to loaf than westerners, and, for some reason, women are less likely to loaf than men.
• Task complexity - Participants are less likely to loaf if the task is quite complex than they are if the task is simple.

On the other side, social loafing is increased by the following factors:

• Greater number in group - the more people there are in a group, the more loafing will occur.
• Low expectations of coworkers - if participants thought their coworkers were incompetent, they were more likely to loaf. This contradicts the assumption that competent people will work harder to compensate for their less competent coworkers.
• Adult participants - for some reason, adults loaf more than children. Maybe loafing is a learned skill. Matt likes to talk about "skilled incompetence"; maybe this is related.

Economics

For the economic perspective, we read "Ledyard", which isn't a formal meta-analysis but is a smart guy who read a bunch of papers and tried to pull all the data together. Here's the reference: Ledyard, J. (1995). Public goods: A survey of experimental research. In J. H. Kagel & A. Roth (Eds.), The handbook of experimental economics (pp. 111-194.). Princeton, NJ: Princeton University Press. and here's the insights:

Free riding is decreased by the following factors:

• High marginal per-capita return (MPCR) - the MPCR is the amount of utility an individual expects to get back from contributing to a public good. Consider the game I discussed earlier; if the per-person return for investing in Group Exchange is 25 cents, people are less likely to contribute than if it is 75 cents.
• Common knowledge of who contributes - if everyone else can see how much you are contributing or withholding (and you can, in turn, see everyone else's contributions) everyone is more likely to contribute more to the public good.
• Everyone starts equal - if everyone starts out with an equal amount of money, people are less likely to free ride than they are if the initial distribution of funds is uneven.
• Donation threshold - if some form of threshold is in place (think of campaign contribution targets, for instance, or dollar-amount goals for cancer research fund-raisers) then this reduces free-riding. However, putting a threshold in place reduces the likelihood that the threshold will actually be reached. I don't entirely understand how this works.
• Friendship / group identification - Similar to the "group valence" finding in the social science article, if all the people in the group are friends or otherwise identify with the collective, then they are less likely to free ride.
• Communication allowed - if the members of the group are allowed to communicate with one another before playing the game, then they are less likely to free ride (even though theoretical economics like Prisoner's Dilemma strategies claim that communication is irrelevant).

Yep, you saw it coming. Free riding is increased by:

• Repeated games - if the game is played over and over again, people are more likely to free ride in later games than they are in earlier games.
• Economics training - people trained in economics are more likely to free ride than those who are not. Perhaps because they are familiar with the "optimal strategies" already.
• Experience playing the game - people who have participated in game theory economics experiments before are more likely to free ride than those who are not.
• Vote for contributions - this refers to an alteration in the game where, after the total amount is collected and the individual knows how much he stands to make or lose, everyone votes to determine if the contributions are applied to the public good or if the good is dissolved and the contributions are returned to their original owners. Although you'd think this would discourage free riding, apparently it only makes it worse.

Well, that was fun. Now the question becomes: How do we take all these facts and turn them into practical design recommendations for online community developers? Stay tuned, I hope...

[ Permanent Location ]

The Case Against Dualism

philosophy, psychology

August 30, 2003, 11:21 PM

I've decided that I've been posting about technology too much recently, so in this entry I'm going to talk about some good, old-fashioned philosophy of consciousness.

First, a few concepts. Philosophy of consciousness is that branch of the field that considers questions like "What does it mean to be self-aware?" and "What is 'the mind'?". The theory of substance dualism (also called "Cartesian dualism" since Rene Descartes was the main originator of the theory) claims that humans are divided into two distinct types of substances: "physical substances" which are directly observable and obey the well-known laws of physics, and "mental substances" that are not directly observable using any known measurement technique. Any theory that asserts the existence of a "soul" or a "spirit" that is separate from the body is probably a form of substance dualism.

Substance dualism is a very popular theory, which tends to irritate philosophers since its actually pretty easy to disprove. I believe that more people would doubt the theory if they were aware of the strength of the argument against it, so I'm going to take a shot at explaining it here. It's called "the argument from causal interaction".

This is a proof by contradiction, so assume for a moment that substance dualism is true. In essence, I am a "ghost in a machine": a noncorporeal spirit that controls a material body as my sole link to the physical world. It's important to note, then, that my body and my soul must engage in two-way causal interaction in order for this theory to make sense. In other words, my mind must be able to affect changes in my body (at a thought, I can move my arm up and down, for instance) and my body must be able to affect changes to my mind (when my finger is pricked with a needle, my mind experiences pain, for example). So far so good.

But in order to interact causally, these two entities have to obey certain physical laws. One of these laws is the First Law of Thermodynamics (Conservation): energy cannot be created or destroyed. Granted, the mind is a different type of substance (a mental substance, to be precise) and need not obey the laws of physics, but the body, at least, is a physical object and thus it is beholden to Conservation and everything else we know about the material world. So in order for the mind to causally interact with the body, it must expend energy to alter the body's state (probably in the region of the brain, since that's where all the neural impulses that control most of the body's processes originate from). Since we cannot observe the mind itself (it is a mental substance and indetectable by our instruments), this should create the illusion that energy is appearing and disappearing in our brains. Neural processes are being altered by an energy source that is invisible to us.

Problem is, no such phenomena has been observed. All the energy in our neural system is accounted for.

Thus we must conclude that either dualism is false, or physics is wrong. You can choose the latter if you'd like, but the consequences of that proposition's truth are far-reaching. Most of the consumer products we have in our homes shouldn't work if these fundamental laws of physics aren't universally true. The very way in which we interact with reality would become radically different if energy was not conserved. That's a lot to throw away for an unobservable ghost in a machine.

Here's the argument in brief:

1. If substance dualism is true, then the mind and the body must engage in two-way causal interaction.
2. But such an interaction violates the laws of physics, so if the mind and the body engage in two-way causal interaction, physics is wrong.
3. Physics is not wrong.
4. The mind and the body cannot engage in two-way causal interaction.
5. Therefore, substance dualism is false.

There. Now that we've got that out of the way, perhaps we can move on to some of the more interesting and less one-sided questions in philosophy of consciousness...

[ Permanent Location ]

Personality Typology

psychology

July 22, 2003, 09:27 PM

Micah and Dan have been talking about the Myers-Briggs (/Keirsey/Jung/etc.) personality test recently, so I thought now might be an appropriate time to weigh in on the subject.

I've read David Keirsey's Please Understand Me, which is an excellent book that I recommend highly. Keirsey builds on the basic personality type system by talking about personality temperments, or the four major groupings of the sixteen types. Here's how it breaks down.

To Keirsey, the most important type distinction is Intuition versus Sensing (N versus S). He argues that the visionary, head-in-the-clouds intuitives and the pragmatic, down-to-earth sensers have fundamentally different worldviews, and two people who differ along this type line will have to work hard to understand and accept one another. Thus this is the first major division among the types. The second division (to make four temperments) depends on the first; if you are a senser (S) type, the important division is Judging versus Perceiving (J versus P). If you are an intuitive (N) type, the important distinction is Thinking versus Feeling (T versus F). So we wind up with:

• the freedom-loving, egalitarian, thrill-seeking Dionysian or Artisan temperment (the SP)
• the duty-bound, paternal, tradition-oriented Epimethean or Guardian temperment (the SJ)
• the knowledge-based, power-seeking, skill-building Promethean or Rational temperment (the NT)
• and finally the self-actualizing, relationship-oriented, spiritual Apollonian or Idealist temperment (the NF).

To Keirsey, the Extraverted versus Introverted (E versus I) distinction is the least important and thus never factors in to temperment.

People with the same temperment have similar goals for and outlooks on life. They may find it easier to communicate with others of the same temperment. For example, as I've mentioned before, I test as an INTJ (although my Judging/Perceiving distinction is fairly even and I have a strong Feeling side to complement the Thinking). Note that I share the same temperment as Micah and Dan (the NT Rational), so we should find that our goals in life (building our skills, acquiring knowledge, reshaping the world, etc.) and our worldview (reality exists to be reworked, we will never live up to our own expectations, etc.) to be similar. This doesn't mean we'll agree on all the fundamental questions of life; we won't. But it does mean our personalities give us a similar understanding of the world that we won't share with our Artisan, Guardian, and Idealist friends.

So do I actually believe all this? Well, yes and no. No personality theories are currently considered scientifically valid by most academic psychologists. This is mostly because a fuzzy concept like "personality" isn't amenable to scientific studies; it varies too much to test in any controlled environment. Given this constraint, we need to ask whether its worth thinking about anyway. My answer to this question is always yes. So I'd argue Myers-Briggs is worth giving serious consideration.

That said, I'd also argue your consideration shouldn't be too serious. As Jung originally conceived of them, the personality types are archetypes, or idealized portraits of distilled personality traits. They do not describe real people. So if you take Keirsey's test and look over the type descriptions, you'll probably find your real personality overlaps a few of the types, as I alluded to by describing myself as "an INTJ with equal Perceiving and a strong Feeling side". So it's worth remembering that Myers-Briggs as a predictive model is a rough measure at best.

[ Permanent Location ]

The Ways We Read

internet, psychology, usability

June 25, 2003, 12:22 AM

Earlier this month, I ran an investigation of how people browse periodically updated websites (such as news sites, weblogs, or any page whose content changes and people keep coming back to view these changes). I ran these studies to inform the design of Newsable. They are directly intended to feed into the development of Newsable's user personas, but I promised several people I'd post the results when they became available, a promise I've been remiss in fulfilling thus far. So, at long last, here's the full writeup of my findings.

Experimental Method

To collect the data, I followed the basic Contextual Inquiry technique as described in Contextual Design. In essence, Contextual Inquiry is a quick-and-dirty form of ethnography. Like more rigorous ethnographic techniques, Contextual Inquiry relies on observing study participants in their "natural environment", i.e. while they are performing the tasks you are interested in as they would normally perform them. Unlike other techniques, Contextual Inquiry calls for lots of interaction between the experimenter and the participant; the experimenter constantly interrupts the participant's work to ask questions, clarify assumptions, explore areas of particular interest in greater depth, etc. A good contextual inquiry, then, keeps the participant "in context", or requires them to show the experimenter how they do their work rather than asking them to give vague summaries that won't contain many important details, while at the same time getting at the behavior the experimenter is interested in without having to spend months as a "fly on the wall" while dispassionately observing the participant.

For me, this translated into sitting down with the participants as they read their daily websites and asking them about what they were looking at, what they were thinking, why they chose to click this link and not that one, etc. In total, I observed eight users for about an hour to an hour and a half each. People who are trying to be rigorous about their contextual inquiries often videotape the entire session for later analysis. I just took notes.

Note that the nature of the data Contextual Inquiry is designed to collect is not statistically generalizable. Ethnography emphasizes deep investigation of the full range of rich behaviors for a small number of participants, not narrowly focusing on one tiny aspect of the behavior for a large number of participants that will yield a p of less than .05. This means that the findings I'm about to present are (hopefully) sufficient to guide the design of a news aggregator but not sufficient to make timeless generalizations about human behavior.

Results and Findings

Of the eight users I observed, two already used a news aggregator (NetNewsWire and Radio Userland). There was a clear correlation between the number of sites participants read and whether they used a news aggregator. The six who did not use an aggregator read around three or four sites every day. The two who did read upwards of fifty different sites on a daily basis. I think this speaks to an interesting quality of news aggregators. At first blush they don't appear to be very useful programs, after all, all they do is save you the minimal effort of clicking on a few entries in your bookmarks sidebar to hop from site to site. But by collecting the news from various sites into one easily-scannable location, news aggregators actually greatly increase the amount of content the reader is willing and capable of skimming regularly. This has the potential to expose people to a much wider variety of news sources and their corresponding perspectives.

All the participants I observed spent a lot of their time skimming content to see if it looked interesting to them. They all liked to visually scan for content that looked interesting; they didn't approach the news-reading task with many preconceived notions of what kind of content they were looking for, but instead glanced over what they found and made a judgement call on how interesting it was on a story-by-story basis. They generally preferred having both a title and a summary or short excerpt for each story rather than just the title; more information was more better (these observations explain why Google News decided to eschew Google's characteristic minimalist design in favor of a much denser information display).

For the most part, all participants appeared to prefer to see only the items that were "new to them", i.e. that had been added or modified since the last time they visited the site. Participants expressed a preference for sites that listed news stories in reverse-chronological order so they could read until they hit a story they recognized. A couple participants, however, actually preferred to read through the listing in chronological order and would skim down to a story they recognized then read backwards up to the top of the list. Finally, one participant remarked that she actually preferred to see the old stories as well as the new ones since the old stories helped her remember the context in which the new stories took place (if they were continuing coverage of some running event, for example).

Another common theme was the importance of the reputations of the authors of the stories and the publications they were writing from. As I've already mentioned, the participants depended heavily on their knowledge of the reputation of the authors of news commentary to determine whether they were worth reading or not. Participants would frequently read their favorite authors even if the topic of the essay didn't particularly interest them. Moreover, several participants had developed an understanding of the personality and interests of the people they read, and used this knowledge to help determine whether an article by that person was worth reading. Of all the criteria people used to determine what to read, the reputation of the author was consistently the most powerful.

Almost all of the participants would occasionally send stories they found interesting to friends via email or instant messanger. One participant remarked that she viewed reading the news as a very social activity and relied on her friends to keep her informed about sites she was interested in but not sufficiently to read on a daily basis. Another mentioned that he liked it when his friends read different websites than he did so they would have something to talk about later; he liked to be able to trade interesting news tidbits with his buddies.

After running two or three of these observations I become interested in how the participants reading habits changed over time. Although I wasn't able to observe this directly, participants generally described their reading habits as changing gradually, especially those without news aggregators who only read a few sites. Frequently, they would discover new sites by following links on their current news sites, and if the new site continued to come up with interesting content, they would sometimes add it to their daily "rotation list" of websites to check. Participants reading patterns also changed with their current interests; one participant was taking a photography course and thus was starting to read more weblogs on photography than she had before. Additionally, many would learn of new sites through friends who emailed them interesting stories, and sometimes these sites would be interesting enough to check daily. Some of the users without aggregators remarked that they knew of many sites that were interesting but not interesting enough to check every day, and thus they tended to check them very infrequently since doing so wasn't part of their daily habits. Participants with news aggregators tended not to have this problem.

Finally, one common theme was that participants tended not to remove sites from their daily rotation lists very frequently; one user (who did use a news aggregator) remarked that the cost of leaving a somewhat-uninteresting site in his list was so low that it wasn't worth the couple of clicks it would take to remove it, he only removed sites if they were both uninteresting and posted a lot of content that got in his way. Of course, this participant was also confronted by upwards of 300 stories on a daily basis...

Conclusions

The observations presented here have been distilled into the user personas of Newsable, the usable news aggregator (coming soon). If you're interested in this research, you may wish to read over the personas to get an idea of how all this comes together into a coherent picture of the user that website authors can design for. The personas are the "cannonical version" of this research and will be updated as new findings become available.

Thanks to everyone who put up with my obnoxious questions about their news reading habits to participate in my studies (without any compensation, no less). My understanding of what people need in a news aggregator has changed dramatically as a result of talking to you all.

If you have any questions or comments relating to these findings, please feel free to email me or leave them below. I plan to continue these studies as an "open source user research" project; more on that shortly.

[ Permanent Location ]

Whuffie and Rejection

psychology, society & sociology

May 24, 2003, 10:45 AM

A couple days back, Micah posted about whuffie, a "reputation currency" that appears to at least purport to be a metric for respect. This reminds me of a lecture I went to last fall given by the psychology department. The lecturer talked about the psychology of rejection. Among other things, he pointed out that to feel that someone has rejected you has nothing to do with how much they like you in some absolute sense. Rather, it depends on whether they like you in the way you want them to and to the degree you want them to. So if a girl I'm romantically interested in only considers me a very good friend, I'll feel rejected even though she likes me a lot in an absolute sense. And if a person I can't stand tells me he hates me, I don't feel rejected because I don't care if this person likes me at all.

So as we can see, the feelings of rejection are tied to how much you respect a person in the small and how much they respect you. The psychology lecturer, if I recall correctly, was trying to develop a scheme for measuring rejection which struck me as a particularly difficult thing to do. Likewise, I'm skeptical that this "whuffie" concept will be able to encompass enough of a complex emotion like respect to be useful. Some have complained that the Social Software people are reinventing intellectual wheels; they're recasting old ideas as new memes and confronting the problem of software in society without understanding the work of their predecessors. I'm not as vitriolic in my criticism, but I have started to get that feeling as well.

[ Permanent Location ]

© 2003-2008 Rob Adams ()

Last Modified: Friday January 25, 2008

Unless otherwise indicated, all original work appearing on this web page is licensed under the Attribution / No Derivitive Works Creative Commons license. Please before using any page content in a manner not covered by this license.