 |
« | Week of Sep 7 | < | Individual Entries | > | Week of Sep 21 | » |  |
 |
 |
The Role Of the University in Higher Education
society & sociology, teaching & learning
September 19, 2003, 06:05 PM
After attending the whole IT in the research university event as well as reflecting on my experiences in classes here at CMU and my illustrious schoolmates' comments and complaints on their own experiences, I've got to thinking about what exactly the role of a research university should be in the broader educational system. Although I'm no expert in education, I have been a student for many years now and have some experience with teaching, so I feel I have some authority to pronounce on this subject.
Here's my thoughts on what kind of learning would be appropriate at the various stages of education and who should be responsible for meeting these expectations:
- Undergraduate education should focus on giving students a broad base of knowledge to form the foundations of a productive intellectual life, as well as enough skills in a particular field to begin a career. Ideally students should all leave undergrad as T-shaped people. In my opinion, undergraduate degrees should not be conferred by research institutions; they should instead be granted by colleges that are specifically oriented towards teaching. Teaching well is a full-time job; you can't expect researchers to be able to just "fit it in" to their research schedules, especially when their incentive structure heavily favors deprioritizing it in favor of their research. These teaching colleges must also have the resources to offer courses tailored towards students with particular interests; it is not acceptable to expect a designer to learn everything they need to communicate with programmers by taking an introductory computer science course geared towards people who want to focus on programming, for instance.
- Professional graduate education, such as the programs I and most of my friends are in here at CMU, can be taught by research institutions, but must not be seen as "more undergraduate work". If you want to learn a new skill, you need to go back to a teaching college. Professional graduate education, on the other hand, should mostly involve courses run in a seminar style that focus on collaboration between the professors and students (most of whom are returning to school after spending some time practicing their trade) to distill the abstract research done by academics into a form that can be used by the industry practitioners. Essentially, professional graduate education should be viewed more as a form of technology transfer (numerous studies have found that transferring people is much more effective than merely transferring technology) and less as a form of instruction.
- Academic education, or PhD programs, should be run more or less as they are now. Essentially PhD programs are (or should be) a form of apprenticeship where the student learns by working with the master. Hopefully freeing researchers from some of their teaching burdens will give them more time to spend working with their PhD students, who are the type of people they are actually qualified to teach.
So that's my modest proposal.
Email Rob:
Libel Laws and Blogging
internet, politics
September 17, 2003, 12:54 PM
This is old news, but I think it's worth repeating (I'm just no good at being an information courier... :).
Last June, the Ninth Circuit Court of Appeals determined that webloggers can't be held responsible for libel for information they republish. Newspapers and other official media outlets can be sued if they republish false information that causes harm to someone (even if they themselves didn't make up the information). But the court decided that weblogs are more like informal discourse and less like carefully fact-checked news publications, and thus fell under the rules for speech rather than news.
I've ruminated about the issue of speech laws online before, as well as some other good news. Nice to see that the courts appear to be making sensible decisions about these sorts of things.
Email Rob:
Surprise-Explain-Reward
psychology
September 17, 2003, 11:31 AM
Margaret brought up something else I found interesting in the talk I mentioned in my last post. One of her problems was getting buy-in from users that they even should test (remember, they are overconfident already). Engineers and designers often have an implicit theory that "if you build it, they will come" and are surprised (and annoyed) when their great system isn't immediately picked up by the populace. But from a user's perspective, using any new system is an investment that they may or may not choose to make. They must consider:
- The cost of learning the new system.
- The benefit of using the new system (which is often not made very clear).
- The risks of changing their current work practices (like spending lots of time re-entering your contacts into a PDA only to never use it).
To do this, she examined the psychology literature and learned about a theory called Surprise-Explain-Reward. This theory says that in order to sell a new work practice or technique to someone, you need to:
- Surprise them with evidence that their current practice isn't working as well as they thought. You need to identify an "information gap" or a hole in their understanding that will make them curious. People will want to fill this hole if you can make them see it.
- Explain to them the information that will fill this hole. Often the best way to do this isn't through direct lecturing, but through self-directed learning where you suggest the actions you want them to take then allow them to discover the solution "on their own".
- Reward them with immediately perceivable benefits for using the technique. If all the benefits are long-term many users won't see the value in continuing to use the system. People are bad at connecting causes and effects that are separated by a large amount of time, so try to shrink this time down. If many of the benefits are inherently long-term try to provide smaller tidbits to encourage your users to keep going.
A neat idea, which is apparently backed up by empirical research. Something to think about for us inventor-types who are good at coming up with crazy ideas but aren't so good at selling them.
Email Rob:
End-User Software Engineering
software development, usability
September 17, 2003, 12:57 AM
Last Wednesday, a woman named Margaret Burnett from Oregon State University gave a talk at the HCII Seminar Series which I attended on the topic of "End-User Software Engineering". Here's the problem: many applications require end users (read: people who never program and don't want to) to write high-level, fourth generation programs to access certain advanced features of the software. Some examples include writing small Javascripts for your web page, setting up rules in your email client for automated organization of all the email you get, and (this was Margaret's "test" problem) creating formulas in spreadsheets to perform complex calculations. Margaret wanted to help these types of users produce better programs by making some software engineering techniques more accessible.
Margaret unveiled the rather disturbing statistic that, apparently, around 90% of production spreadsheets (that is, spreadsheets that are used for real work) contain errors in their formulas (and thus may be producing incorrect calculations). Moreover, these users are overconfident in their results; they believe their formulas are correct when they are not (this phenomenon is common among professional programmers, as well...). Her work attempted to attack both these problems by helping users write better code while also giving them the appropriate level of confidence in the result (since no code is ever perfect).
For her purposes, Margaret defined a "test" as "a decision if some output is right for a given input", so her interface had to help users make this decision, i.e. to execute tests. There's also the problem of test adequacy, that is, how do you know when you've done enough testing? (No amount of testing can guarantee correctness, it can only increase your confidence.) She then discussed the "WYSIWYT" or "What You See Is What You Test" paradigm, an attempt to make the fairly invisible process of testing more visible to the user. Her hypothesis was that making testing more visible and providing better feedback would help tackle both the execution and adequacy problems.
Since she was working with spreadsheets, Margaret's research group created an interface that allowed users to provide sample input values and automatically determined how fully the provided input had tested the formulas (this is basically white-box testing), then provided feedback on whether the formula was "not tested", "partially tested", or "fully tested" by changing the color of the cell. They also implemented a "Help-Me-Test" feature that provided suggested test values (to varying degrees of accuracy). Through empirical studies, they learned that this WYSIWYT paradigm:
- Resulted in higher test coverage for user's programs.
- Helped user's gain a functional understanding of their programs.
- Gave users a more appropriate level of confidence in their programs.
- Made users faster at debugging.
- Reduced (but did not eliminate) overconfidence.
To help further improve the overconfidence problem, Margaret's group also implemented an assertion mechanism into their spreadsheet testbed, which allowed users to define appropriate bounds for cell values. Assertion failures caused the values to get a big red circle around them. Empirical user tests revealed that users seemed to interpret these appropriately; Margaret quoted one user as remarking "Oh, there must be something wrong with the formula!".
This seems like some interesting and important research, and I'd love to see how it might apply beyond spreadsheets. Andy remarked that Margaret was collaborating with Microsoft to potentially get some of this stuff implemented. Which would bring us one step closer to bringing the power of programming (and thus the "true" power of the computer) to ordinary, everyday people.
Email Rob:
Posted by Dan on September 20, 2003 at 09:41 AM
Spot on. Having gone to reseach schools for both undergrad and grad, your observations are corect. However, I bet you might feel differently about PhD programs after you've gone through one....
Posted by Rob on September 20, 2003 at 10:12 AM
Yeah, I'll bet the reality of PhD-level education isn't as good as the theory. Not having ever been in a PhD program, I didn't have much to say about that.
Theoretically, however, I'm guessing an apprenticeship model is the best way to go for teaching advanced researchers. In fact it works pretty well in a lot of trades. I don't know why our society doesn't use it much anymore.
Posted by Dan on September 20, 2003 at 02:18 PM
Because it isn't profitable. Why have one apprentice that you have to pay, when, at a university as a professor, you can have five, paying $28k a year?
Posted by Rob on September 20, 2003 at 06:36 PM
In Ye Olde Days, apprentices worked for low pay in exchange for instruction, so the benefit to the master was cheap (albeit less skilled) labor (and the practice persists today in some fields; my dad, who is a woodworker, took on an apprentice for awhile about a year ago). This is essentially what the HCII here at CMU does; PhD students are fully funded by the department so they don't have to pay tuition and receive a modest stipend in exchange for their work. So it does cost profs money to support PhD students, but in exchange they get expanded capacity to research and publish papers.