Generalizations 13# Vezzaro, L. (2018)

 At the current stage this project is about finding statistical generalizations in qualitative, danish educational research. That is to check my assumption that there is a tendency to make statistical generalizations on the basis of observations which cannot support such generalizations.

I will be posting excerpts from all those studies which to me seem problematic. This post is part of that and this is a link to the first post in the line.

Vezzaro, L. (2018)

Reference:

Vezzaro, L. (2018). Introducing complexity and uncertainty of environmental models in the education of future engineers. Dansk Universitetspædagogisk Tidsskrift, 13(25), 194–210. https://doi.org/10.7146/dut.v13i25.104487 

 

Abstract:

Environmental models are affected by significant sources of uncertainties com-pared to other engineering fields. However, traditional courses tend to provide a deterministic perspective, where the various sources of uncertainty (e.g. model structure, input data, implementation) are often neglected. This issue was high-lighted during a university teachers’ training programme, where the trainee proposed a solution that aimed to improve student understanding of uncer-tainty sources. The proposed solution was implemented in an existing MSc course (with 90 students). The course, originally based on problem-solving group work, was revised by introducing an assignment inspired by Problem-Based Learning, which can be used to introduce engineering students to com-plex issues. The new assignment introduced the students to new uncertainty sources (model structural and technical uncertainty) that are essential in the development and application of environmental models. The effects of the new approach on the students’ learning were monitored by using course evaluation questionnaires and written feedback from the students. The open-ended as-signment challenged the common habits of the students, highlighting the sub-jectivity in model applications and result interpretation. The students’ response was mixed, with major concerns linked to the high workload, which limited the time for deep reflection. Nevertheless, the learning objectives were successfully achieved, providing future environmental engineers with better understanding of the complexity of environmental modelling.

 

Testable hypothesis?:

No: "This  article  aims  at  showing  how  understanding  of  different  sources  of  uncertainty  can be introduced within the structure of existing environmental modelling courses. " s195

 

Method/materials:

"The author thus proposed a modification of an  existing  MSc  introductory  course  on  environmental  modelling,  based  on  the  fol-lowing concepts:•There  are  several  modelling  approaches  that  can  be  used  to  solve  environ-mental problems (model structure uncertainty)•Modelling  results  are  affected  by  several  sources  of  uncertainty,  including  subjective choices made by the modeller him/herself (model technical uncer-tainty)." s195

"Several  methods  were  used  to  monitor  the  implementation  of  the  new  module  structure:•A pre-testing test, carried out in the days before the start of Module 5,  the re-sults of  which  were  actively  discussed  in  the  first  class.  The  pre-test  focuseson  the  preconceptions  that  the  students  might  have  on  the  applicability  ofenvironmental models, and it is therefore designed to challenge the studentsand their ‘engineering common sense’, i.e. mental structures which have beenbuilt  throughout  their  engineering  education  but  which  might  represent  anobstacle  to  a  successful  use  of  the  core  elements  of  the  course.  The  quiz  isused  to  introduce  and  explain  the  structure  of  the  assignment  (which  differsfrom the previous four).•The standard course evaluation questionnaire, which is provided to studentsat  the  end  of  all  the  university  courses  and  which  enables  comparison  withprevious  academic  years.  The  questionnaire  also  includes  a  free  commentfield,  which  usually  provides    the  most  important  insight  into  the  students’perception  of  the  course  (i.e.  students  can  provide  more  comprehensivefeedback to teachers).•A supplementary Course Experience Questionnaire (CEQ) based on Ramsden(1991) and with specific questions developed in collaboration with the univer-sity’s teaching counsellors.•Oral feedback and discussions between teachers and students during the ac-tivities in the computer room. This oral feedback was subsequently discussedamong teachers and teaching assistants." s201

"A total of 90 students followed the course, with 75% answering the overall final eval-uation of the course and about 69% answering the CEQ" s202

 

Statistical generalizations:

1) "The  critical  thinking  of  students  was  improved,  as  well  as  their  understanding  of  the  concepts  involved  in  uncertainty  analysis.  " s208

 

Comments:

I have flagged this study for generalizations of type 4 (see typology).

Aside from the fact that "critical thinking" is not a validated construct, this study could not have proven that improvement in said construct occurred. This is a classic version of one of the problems that i would like to rid the world of. The quotation that i pulled from this study (shown above) could easily become the primary takeaway quoted by others. The degree to which it lends itself to misuse, as part of other researchers arguments, is too great.