From the current CAUT bulletin, “The end of student questionnaires?” Research is driving legal and institutional decisions about the appropriate use of student evaluations of teaching (SETs) — or if they are to be used at all.
Researchers Kristina Mitchel and Jonathan Martin discovered that students not only evaluate their professors differently based on whether they are women or men, but also the male instructor received statistically significantly higher evaluations than the female instructor. The researchers also found the language students used in official open-ended course evaluations and in online anonymous commentary suggests gender bias, with women more likely to be viewed as not having as much experience and education or as being less accomplished than men
. Ananalysis published last year by Bob Uttl (Mount Royal University), Carmela A. White (UBC-Okanagan) and Daniela Wong Gonzalez (University of Windsor) in Studies in Educational Evaluation found that students do not learn more from professors with higher student evaluation scores. In other words, student opinion ratings and student learning are unrelated.
Two reports commissioned by the Ontario Confederation of University Faculty Associations also concluded that student questionnaires were unreliable and were submitted as evidence by the Ryerson Faculty Association during arbitration. The former director of the Center for Teaching and Learning at the University of California (Berkeley), Richard Freishart, and statistician Philip B. Stark (Berkeley) highlighted major flaws in the methodology of student surveys on top of ethical concerns around confidentiality and informed consent that raised human right issues such as bias about race, gender, ethnicity, accent, age and “attractiveness.”“The end of student questionnaires?” (CAUT)