Generalizations #49 Callesen, J. J., Hjernø, K., O’Neill, L., Carstensen, S. S., & Bloksgaard, M. (2021)

At the current stage this project is about finding statistical generalizations in qualitative, danish educational research. That is to check my assumption that there is a tendency to make statistical generalizations on the basis of observations which cannot support such generalizations.

I will be posting excerpts from all those studies which to me seem problematic. This post is part of that and this is a link to the first post in the line.

Callesen, J. J., Hjernø, K., O’Neill, L., Carstensen, S. S., & Bloksgaard, M. (2021)

Reference:

Callesen, J. J., Hjernø, K., O’Neill, L., Carstensen, S. S., & Bloksgaard, M. (2021). Scaffolding students’ preparation for a pharmacology practical improves their self-efficacy and learning. Dansk Universitetspædagogisk Tidsskrift, 16(30). https://doi.org/10.7146/dut.v16i30.122602

Abstract:

Laboratory practicals are important learning elements in science teaching. We used principles of active learning to develop interactive online teaching materials to scaffold students’ preparation for a simulated pharmacology practical. We aimed at increasing students’ self-efficacy and evaluated the effect of the intervention qualitatively and quantitatively. The online material contained elements of formative assessment for the students to follow their learning progress. Students’ readiness for the practical was assessed through a mandatory multiple-choice test. Students (73%) agree or strongly agree that the course of the practical increased their competences in pharmacology. We infer from the evaluations that the interactive material increased students’ self-efficacy, informed refinement of the conceptual understanding of pharmacology and increased their ability to apply content knowledge to solve the inquiries for the practical. The pedagogical principles, e-learning tools and learning elements demonstrated in this development project can be used in many other educational contexts and disciplines.

Testable hypothesis?:

Nej: Kan ikke finde

Method/materials:

"The computer simulation exercise is conducted using Virtual Physiology’s SimVessel© software module (Philipps University of Marburg, Engineering Office of Biomedicine and Technology, Marburg, Germany) running on the students’ individual computers via an institution license."

"53 students participated in the course evaluation; " s4

"Our study is limited by our study group being rather small (41 students only). " s17

Statistical generalizations:

1) "Although not evaluated through objective measures of attainment, we infer from these findings that scaffolding students’  preparation  for  the  pharmacology  practical  increased  their  scientific  content  self-efficacy  and  experienced learning outcome." s17

2) "Our findings support the use of educational IT-supported learning. The pedagogical principles, e-learning tools and  learning  elements  can  be  applied  in  many  other  scientific  areas  and  at  many  other  learning  institutions,  from primary school to higher education" s18

Comments:

I have flagged this study for generalizations of type 4 (see typology).

As is said in the first quote, there were no "...objective measures of attainment...", that is to say, as is clear from the article, there were no validated measures used. Somewhat carefully the researchers go on to announce that they merely infer the effect on this specific type of scaffolding on scientific content, self-efficacy and experienced learning outcomes. The implicit and generally reasonable expectation is that scaffolding works, and will work in the future, which is a statistically general expectation (type 4). While generally reasonable this study can't support it.

The second quote further piles on the meaning of the first, extending the claim to "educational IT-supported learning", as well as other fields (type 4).