Generalizations 1# Langhorn, R. et al. (2018)

At the current stage this project is about finding statistical generalizations in qualitative, danish educational research. That is to check my assumption that there is a tendency to make statistical generalizations on the basis of observations which cannot support such generalizations.

I will be posting excerpts from all those studies which to me seem problematic. I will do this for two reasons: Firstly, there is no consistent method of identifying unwarranted, statistical generalizations, which is not ultimately dependent on my judgement. For this reason it should be possible to review my judgement. Secondly, many of the generalizations that i have identified, do not feel problematic to read for the simple reason that many researchers do use careful language, which is a good thing. It is important to be able to recognize that while a generalization may be technically problematic, it can be delivered in such a way, that most people would not find it problematic. Therefore it must also be possible to discuss whether such cases are or are not problematic. Does proper delivery excuse the problem? There are certainly many cases of research coming under criticism, not because it is bad research, but because it was sold (in media) as being more than it was. 

Anyway this is the first of the studies that i claim to be problematic:

The study

Reference:

Langhorn, R., Bjørnvad, C. R., Fog-Larsen, A. M., Willesen, J. L., May, M., & Langebæk, R. (2018). A Virtual Veterinary Emergency Clinic – investigation of students’ perceptions and self-efficacy beliefs. Dansk Universitetspædagogisk Tidsskrift, 13(25), 120–133. 

https://doi.org/10.7146/dut.v13i25.103936  

Abstract:

"At the University of Copenhagen, companion animal emergency medicine is taught in a clinical environment after students’ completion of basic theoretical and clinical courses. Students are often anxious about emergency shift partici-pation and the prospect of being the responsible veterinarian in emergency sit-uations. This study aimed to investigate whether inclusion of virtual patients in addition to real-life patients would increase students’ perceived self-efficacy in emergency medicine. Sixty-seven students were divided into two groups, one of which participated in regular emergency rotations, while the other also learned in a Virtual Emergency Clinic (VEC). Participating students were given a ques-tionnaire regarding course experience and self-efficacy, with responses on a 10-point Likert scale. The VEC group expressed a higher level of knowledge and sig-nificantly higher level of exposure to and ability to handle emergency patients. In addition, virtual problem-based learning appeared to increase veterinary students’ self-efficacy with regard to managing emergency patients in their fu-ture careers." 

Testable hypothesis?: 

Yes: "The aim of this study was to investigate whether inclusion of virtual patients in addi-tion  to  the  real-life  emergency  patients  at  the  UHCA  would  lead  to  an  increase  in  students’ perceived self-efficacy in dealing with emergency patients. "

Method/materials:

"Twenty-nine students participated in the first EOCA course in the spring of 2015 and were designated to the 'No VEC' group. This group completed the course in the tradi-tional manner based on patient intake in the hospital and with no access to the VEC. Thirty-eight  students  participated  in  the  second  EOCA  course  and  were  designated  to the 'VEC group'. During the introduction week, a thorough demonstration of the e-learning  program  was  given,  and  written  instructions  were  supplied.  Each  student  was to access the VEC on-line and complete four virtual patient cases, write a medi-cal  record  for  each  case  and  send  these  to  the  course  instructor  before  the  end  of  the  rotation.  Prior  to  the  EOCA  course  both  groups  had  participated  in  a  general  practice  course  at  the  same  hospital,  but  had  not  yet  encountered  critically  ill  or  emergency  cases.  All  students  volunteered  to  participate  in  the  study,  which  was  approved by the Veterinary Study Board at the University of Copenhagen. Three stu-dents who participated in the course prior to initiation of the study had volunteered to  be  interviewed  before  and  after  their  course  in  order  to  clarify  students'  percep-tions  of  their  self-efficacy  in  relation  to  emergency  medicine  and  to  obtain  infor-mation that could identify key subjects to be included in a questionnaire." s123

Statistical generalizations:

1) "The  results  of  this  study  demonstrate  that  including  problem-based  learning  in  the  form of a virtual clinic in the EOCA course increases the self-efficacy of Danish veter-inary  students  for  managing  emergency  patients  under  supervision  during  their  training and indicate a similar effect with regards to patients in their future careers. We  believe  that  this  is  likely  to  be  true  for  veterinary  students  internationally.  " s130

2) "This encourages us to priori-tize this educational tool, add more cases and further improve the VEC so that, in the future, it  will  hopefully  simulate  the  emergency  room  so  closely  that  only  the  live  animal is lacking." s131

Comments:

I have flagged this study for generalizations of type 4 and 5 (see typology).

The first example is a direct generalization, mostly due to the last sentence. The language is a little careful, since the researchers describe the generalization as a matter of their belief ("We believe..."). The study does not however hold enough observations for them to be generalized beyond the borders of Denmark. Perhaps they cannot even be generalized beyond Copenhagen. The first sentence in the first example can also be seen as problematic, mostly due to the fact that the study is self-report. For this reason it can only indicate a change in opinion of self-efficacy. One might argue that self-efficacy is a validated construct, but this would only apply to my critcism had the authors of the present study utilized one of the validated inventories. The questionaire of this study however comprises four items, only two of which are actually sentences that could be used to indicate self-efficacy. As such the sentence: "The results of this study demonstrate" is simply too strong and borders on misinformation.

The second example is less problematic. I note it only to indicate another shade of the same problem. Whether the studies i have read for this projects, include type 4 generalizations or not, the researchers nonetheless often make comments which are not in themselves statistical generalizations, but that imply a belief that the results of the study are in fact statistically generalizable. The second example, as noted here, is not a very grim case of this situation. Yet it implies that the results of the study are strong enough, that they warrant further development of the educational tool that was tested. It only makes sense to make such a statement, if it is implied that the tool in question will have similar effects on similar populations in the future. In other words the statement rests on a statistical generalization.

First overview of the dataset

 I have finished my first read-through of the 124 articles from the last four years of the journals mentioned here. Here is a screenshot of the overview of the dataset so far (it's in danish):

(sry about the quality)

Unødvendig akademisering

Det akademiske sprog

This text is a side note on a discussion i noticed in one of the articles i am reading from one of the journals implicated in my project. The discussion is interesting because it is about "academisation" or whatever one might call what is essentially the unnecessary use of extraordinary academic vocabulary. The journal in which the discussion takes place is a great example of a journal where researchers often indulge in the aforementioned type of criminality (rest in danish, since it is from my notes - write me if you are interested in translation).

Den ene af de to journaler jeg undersøger som en del af mit projekt er SLP (Studier I læreruddannelse Og -Profession). I en af artiklerne (citeret herunder) kan man finde en diskussion af "akademisering":

“I pressemeddelelsen “Læreruddannelsen skal tætt ere på praksis” lancerede uddannelses-  og  forskningsminister  Ane  Halsboe-Jørgensen  i  august  2020  regeringens  målsætning  om  at  sikre  et  højt  ambitionsniveau  for  lærerud-dannelsen  gennem  “mere  praksis,  ikke  [...]  mere  akademisering”  (UFM,  Læreruddannelsen skal tætt ere på praksis, 2020). Med formuleringen stiller Ane  Halsboe-Jørgensen  en  modsætning  op  mellem  praksis  og  akademise-ring. Modsætningen, der bliver opstillet, er problematisk, og argumentet, der ligger  bag,  har  stråmandskarakter.  Ordet  akademisering  bærer  en  indbyg-get  negativitet  i  sig. Akademisering betyder  ifølge  Den  Danske  Ordbog“d e t  at  gøre  (unødig)  akademisk;  det  at  give  et  (for)  teoretisk  og  formelt  præg”.  Ingen kan ønske at gøre noget – heller ikke læreruddannelsen – unødig aka-demisk  eller  give  noget  et  for  teoretisk  præg.  Ordet  henter  uden  nærmere  bestemmelse  betydning  i  ønsket  om  mere  praksis,  og  det  implicerer  en  afstandtagen til viden, der er abstrakt og uden forbindelse til skolens under-visning. En anden pressemeddelelse, “Minister sætt  er  gang  i  arbejdet  med  at  udvikle  læreruddannelsen”,  bestyrker  denne  antagelse.  Her  hedder  det  nemlig, at “Læreruddannelsen skal endnu tætt  ere på virkeligheden i klasse-lokalerne”, og regeringen “ønsker derfor [...] at der kommer mere praksis ind i læreruddannelsen” (UFM, Minister sætt er gang i arbejdet med at udvikle læreruddannelsen, 2020). Overvejelserne munder ud i formuleringen af tre overordnede  temaer.  Det  første  tema  er  “Styrket  praktik  og  praksissamar-bejde”.  I  det  underliggende  notat,  “Nytænkning  af  læreruddannelsen  –  et  ambitiøst  udviklingsarbejde”,  uddybes  temaet.  Praktik  og  praksissamar-bejde skal styrkes, og det skal blandt andet ske ved, at de “lærerstuderende skal  klædes  bedre  på  til  at  træde  ud  i  virkeligheden  i  folkeskolen”,  fx  ved  at de møder “mere virkelighedsnære praktikforløb” (UFM, Nytænkning af læreruddannelsen, 2020). “ M. Andersen, K., Aaskov Iversen, R., Eskildsen Jepsen, R., & Ipsen, M. (2021:153)

Jeg skal ikke kunne påstå at jeg ved meget om hvad lærerstuderende bliver udsat for, men jeg vil dog fremhæve et par eksempler på unødvendig akademisering fra journalen hvor den citerede artikel findes:

Citater fra artikler i SLP:

"Undersøgelserne  er  blevet  forenet  under  hermeneutisk  ontologi  og  epistemologi, ved at vi har betragtet mentorsamtalerne som fænomener, der kun er tilgængelige for os i lyset af den horisont og forforståelse, vi er bærere af." - Lunde Frederiksen, L., & Halse, E. (2021:139) 

Sætningen er intetsigende i sin fuldstændige almengyldighed. Jeg betragter det som en novice illusionist der gerne vil være ligeså god som Judith Butler, men stadigt ikke helt magter vokabularet. Se min kritik af Judith Butler's "Excitable Speech" her

"Metodisk  er  deltagerbaseret  aktionsforskning  et  forskningsparadigme,  der  integrerer teori og aktion med det formål ”at adressere organisatoriske, samfundsmæssige og sociale problemstillinger sammen med dem, der oplever disse” (Cunningham, 2014, s. 3). I KDA kombineres aktionsforskningsparadigmet med kritisk teori og anfægter således de traditionelle antagelser om magtforholdet imellem grupper i samfundet, her magtforholdet imellem..." - Jensen, B. F. (2020:122) 

Så det er en metodologi som bare handler om hele samfundet...

"Artiklens sigte er dermed udvikling af en andenordensdidaktik, som indebærer et dobbelt blik på læreruddannelse og skole. Dette blik fordrer refleksivitet i forholdet mellem didaktik og undervisning i de to arenaer, og som en støtte hertil indlejres den almen didaktiske model for skolen i en tilsvarende læreruddannelsesdidaktisk model. Denne formidles som en planlægnings- og analysemodel, der kan bidrage til at reflektere andenordensperspektivet." - Iskov, T. (2020:94)

Læs denne anmeldelse for nogle generelle udsagn om sociologisk systemteori og ideer om ordener

"Den socio-teknologiske fantasi og formåen har et videre dannelses-perspektiv, da sigtet er at bidrage til, at nye generationer af lærere og elever kan forholde sig kritisk-konstruktivt og skabende til forholdet menneske-tek-nologi-samfund." - Hansen, T. I., Thruelsen, D. K., & Skinnebach, L. H. (2019:12) 

"Som  det  er  fremgået  af  analysen,  identifi  ceres  indledningsvist  to  nodal-punkter,  hvorigennem  der  produceres  og  reproduceres  betydninger  om  stressproblematikken:  historicitet  og  kultur.  Ud  fra  disse  to  nodalpunkter  udkrystalliseres særlige betydninger om stressproblematikken set i lyset af fagets  og  personernes  ’historical  bodies’,  ’interaction  orders’  og  discourses  in place’. " - Kappelgaard, L. H. (2019:245)

Der er mange flere eksempler i SLP, men det var nogle eksempler som faldt i øjnene.

Litteraturliste

Hansen, T. I., Thruelsen, D. K., & Skinnebach, L. H. (2019). Socio-teknologisk fantasi og formåen: – et dobbelt didaktisk perspektiv på teknologiforståelse i læreruddannelsen. Studier I læreruddannelse Og -Profession, 4(1), 10–32. Hentet fra https://tidsskrift.dk/SLP/article/view/117976

Iskov, T. (2020). Læreruddannelsens andenordensdidaktik. Studier I læreruddannelse Og -Profession, 5(1), 92–114. https://doi.org/10.7146/lup.v5i1.116353 

Jensen, B. F. (2020). Kvalitet i engelskundervisningen på mellemtrinet: et kritisk blik på indhold og metoder. Studier I læreruddannelse Og -Profession, 5(2), 116–137. https://doi.org/10.7146/lup.v5i2.123522 

Lunde Frederiksen, L., & Halse, E. (2021). Mentorsamtaler og professionel udvikling for nye lærere: Array. Studier I læreruddannelse Og -Profession, 6(1), 135–151. https://doi.org/10.7146/lup.v6i1.127797

M. Andersen, K., Aaskov Iversen, R., Eskildsen Jepsen, R., & Ipsen, M. (2021). Med Pinocchio i skole: - om udvikling af den professionelle dømmekraft i undersøgelsesfællesskaber. Studier I læreruddannelse Og -Profession, 6(1), 152–170. https://doi.org/10.7146/lup.v6i1.127798

Kappelgaard, L. H. (2019). Stressdiskurser i det senmoderne lærerliv. Studier I læreruddannelse Og -Profession, 4(1), 228–248. Hentet fra https://tidsskrift.dk/SLP/article/view/117989 

The problem of the belief in induction in qualitative danish educational research

There are multiple reasons why it can be problematic that qualitative, educational researchers do not work with explicit hypotheses. The most obvious problem is the lack of ability to falsify statements about the world, which is an important feature of science. A less obvious but more important problem is that the lack of hypotheses also means a lack of ability to accumulate knowledge. No hypotheses means that any work of standardization and comparison is very limited and very time consuming. A field which does not work with hypotheses will be unable to claim that it has produced advances in our knowledge, because it will be unable to properly collect and systematize the knowledge it has produced. 

There is also a more subtle problem at play. Instead of using hypotheses, qualitative researchers tend to use research questions of the form “why”, “what, “how”. Not in the sense of delimiting a subject and formulating a theory with testable hypotheses, but in the sense of using such questions directly in conducting research (doing interviews, making observations etc.). This would not be a problematic thing to do for a journalist. It is a problematic thing to do for a scientist because it limits the potential for falsifying the various, tentative explanations that may seem useful for explaining whatever is observed. Also it reflects a belief in induction, or else a lack of belief that it is worth communicating ones own assumptions and theories. To ask “what” something is or “how” something was as a directly applied research question, means that one expects the world to answer. With no explicit hypotheses, the researcher will be observing his own assumptions fed back to him in the misperceived form of objective observations and analysis. Normally the researcher reveals his preferred deductions by stating one or more hypotheses. When researchers do not do this they effectively withhold information. They distance themselves from the fact that they are part of the research that they conduct. They uphold the positivist notion of an objective reality simply presenting itself as knowledge for their consumption. Which is ironic because qualitative researchers often emphasize the fact that knowledge is grounded, socially constructed etc. 

Working with generalizations

 The following is an excerpt from an article i am working on, as such it is written in english. One of the hypotheses that i am interested in concerns the use, in qualitative danish educational research, of unwarrented generalizations. That is generalizations made as part of the conclusion of a research article, which are not supported by the knowledge produced in the article. The point is not to say that this is a problem in itself. That is to say, it sounds bad, but i have learned, through reading many examples, that wording does mean a lot. 

Anyway i have developed an adhoc typology of generalizations which will be relevant to reading many of the things i post here. It is messy because i actively use it when registering in my dataset, and for this reason any changes are very time-consuming.

The text:

Types of statistical generalizations in danish qualitative, educational research articles, presented in two journals (DUT and SLP):

Type 1: General statements made on the basis of the results of the study, that are not unwarranted.

Type 2: Unwarranted general statements not made on the basis of the results of the study, but with reference to one or more other studies.

Type 3: Unwarranted general statements made on the basis of the results of the study, but interwoven with a claim about one or more other studies that seem to support the same findings.

Type 4: Unwarranted general statements made on the basis of the results of the study.

Type 5: Unwarranted general statements made on the basis of the results of the study, presented in the guise of ethical guidelines or suggestions for practice.

Type 1 is, as stated in the definition, an anti-type, used in those situations where no unwarranted general statement was made on the back of the results of the study. Suffice it to say, this type was not observed.

Type 2

Type 2 generalizations while probably the most observed type, was not used often. Checking every general statement made with reference to other studies, was not central to satisfying the hypotheses here dealt with. (EXAMPLE - Literature study)

Type 3

Type 3 can also appear as an example of type 4, that is “a direct generalization”. What determines the difference is whether the generalizing statement refers to the results of other research, and not just the research of the article itself. One might argue that this must certainly be unproblematic, and that on the contrary, it must be the duty of the researcher to point out similarities between their own research and other research. And indeed, this is why we have literature reviews. It is always relevant for researchers and readers to know what other research exists on the subject of an article they are writing or reading. However, the results of a study do not become more generalizable, just because a similar study exists. The plausibility of such a statement and the generalizability of the study in question, can only be claimed with a proper description of the similarities of the studies involved. Especially when we are talking about qualitative research. We cannot simultaneously hold that people answering a questionnaire, to determine their personality type, may mean completely different things with the same answer, while also maintaining that case-studies have similar results, if only the results look alike. Two case-studies made about the same subject, may not be comparable at all. In coding they may vary greatly simply because the studies were made in different countries. To make a warranted type 3 generalization in a study that is not generalizable in itself, at least two criteria must be met:

• It must be clear precisely why and how the other studies involved are comparable.
• The other studies involved must contain at least one study large enough to be generalizable, or the aggregate of the studies must be generalizable. 

Note that a study need not be giant if the population that any generalization is concerned with, is not giant. If there are only a hundred researchers of education in Denmark, we should not need to interview all of them in order to make generalizations about the rest. However, such generalizations would still depend on the types of information we are trying to collect. 

Example of a type 3 generalization:

Oksbjerg, M. (2022) poses the following research question:

“Which conceptions do teachers of the subject danish, hold regarding the formation [(from danish: “dannelse”, also often expressed with the german word: “bildung”)] of students when it comes to teaching about literature, and how do the teachers see these conceptions supported in didactical learning materials, that they choose to teach with.” -Oksbjerg, M. (2022:77)

The article is based on four interviews with teachers of the primary school subject danish. They all used different parts of the same digital, didactical learning material, called Fandango (digital platform) for teaching (Oksbjerg, M. (2022:83)). 

Among other things Oksbjerg concludes:

“Since recent research by Bremholm et al. (2017) shows that learning materials [(digital)] play a central role in teaching the subject danish in primary school, the results of this article can point to a need for supplementary education of teachers in this field.” -Oksbjerg, M. (2022:94)

Through discussion, danish, qualitative researchers might reach the conclusion that a sentence, such as the one quoted, is quite unproblematic. Nonetheless such a discussion must be had, because the sentence in question is an example of an unwarranted generalization. Oksbjerg assumes that the potential, educational needs of four teachers of the subject danish, indicates a need among “... teachers in this field.” as such. This point is made to be supported with a reference to other research (Bremholm), which does not support it. The book by Bremholm et al. is about learning materials used by teachers of danish, but not about their educational needs. Oksbjerg correctly does not cite it as such either, but remarks merely that learning materials “...play a central role…”, which Bremholm et al. does show. Nonetheless the wording of the sentence is: “Since recent research…”, “... the results of this article can…”. In other words the sentence suggests an argumentative interaction between the research referred to and the results of the study in question. Oksbjerg argues further for this interaction on the same page:

“This investigation, as noted, is based on a qualitative case-study, where the mentioned teacher-interviews are included. Within the frame of the conditions of the chosen research design I reason that the results can function as grounds for discussion and further development of both the learning materials of the subject of danish and educational practices. I base this on the information oriented selection of teachers, where differences with regards to educational experience and professional preconditions have been criteria. Moreover the great amount of use, that the specific learning material [(Fandango)] has in danish schools teaching of literature, also adds legitimacy to the results.” -Oksbjerg, M. (2022:94)

The last sentence more overtly suggests the interaction in question. Bremholm et al., as noted before, proves “...the great amount of use…”. Oksbjerg directly concludes that this must add “...legitimacy to the results.”. Even though those results are made to be about, as noted, supplementary education for teachers. Something not covered by Bremholm et al.

Type 4

A direct generalization, in this case, is simply a sentence that either explicitly or implicitly suggests that the results of the study in question also hold for populations beyond the one in the study itself. A sentence that assumes or requires the generalizability of the results of the research in question. As such it is the same as a type 3 generalization, with the exception that type 4 does not include reference to other work in order to sustain the idea of generalizability.

An example of a type 4 generalization:

Drejer, S. N., & Bang-Larsen, A. (2021) looks into what experiences from educational guidance counseling young people bring with them when beginning a bachelor’s degree and what guidance-needs can be identified during the initial year of university (Drejer, S. N., & Bang-Larsen, A. (2021:36)). The article is based on a reanalysis of two case-studies with one student in each, Luna and Veronika (Drejer, S. N., & Bang-Larsen, A. (2021:40)). 

The researchers conclude among other things:

“If Luna and Veronika experience cultural introduction [(from: “indkulturering”)] as a challenging non-linear process, then other students, who to a lesser extent have acquired the content of the subject, will probably also experience cultural introduction as a challenge. Thus we argue that the findings from the analyses of Luna and Veronika are relevant in broader contexts [my translation].”

In this example the authors argue that the analyses of two cases, with two persons, can be statistically generalized. Again, we find two problems beyond the generalization itself. Firstly, the fact that a statement is generally acceptable, does not mean that research supporting that statement, becomes more statistically feasible. That is, unless the research in question is comparable to generalizable research supporting the same statement. Secondly, as with the type 3 example, the problem becomes one of vagueness. Generally, how many people would argue that becoming a part of a new culture, either educational, or work related, or at a societal level, is not challenging? The researchers, whether knowingly or not, make their research seem more broadly applicable, by using it to draw conclusions that most people would accept anyway.

Type 5

Unwarranted general statements made on the basis of the results of the study, presented in the guise of ethical guidelines or suggestions for practice.

To draw conclusions about educational or pedagogical practice, on the basis of the results of a study, implies that those results are expected to reappear in future situations. Therefore such conclusions are implicit examples of statistical generalizations. 

Example of a type 5 generalization:

Beierholm, M. H., Hjorth, M., & Basballe, D. A. (2022) pose the following research question, regarding a new subject being introduced into primary school, and primary school teacher-education, called “technology understanding” (my translation):

“How are the areas of competence of the subject Technology Understanding to be integrated into classical teacher professionalism, so that Technology Understanding becomes partly a fundamental part of teacher professionalism and partly a contributor to the “formation” [(“dannelse” or “bildung”)] of those who study to become teachers. -(Beierholm, M. H., Hjorth, M., & Basballe, D. A. (2022:13))

The study is based on a course run with a class of teacher-students. The materials include observational notes and video. The results however are primarily derived from a report written by the students as part of the course. The researchers received 22 reports from the students, which is the best indication in the article of the actual number of students. From those 22 reports only 8 were used as part of answering the research question of the study, since those 8 included problems related to Technology Understanding-professionalism (Beierholm, M. H., Hjorth, M., & Basballe, D. A. (2022:21)). 

The researchers make the following type 5 generalizations:

“Finally [conclusion, not an exclamation] it looks as if it could be potentially fruitful with teaching as an interaction between use/experience, which leads the students astray, and production/experience, which leads the students behind the technology, as a way toward digital empowerment and formation” -(Beierholm, M. H., Hjorth, M., & Basballe, D. A. (2022:27)). 

The researchers make a very careful generalization, by suggesting a certain way of teaching when it comes to digital empowerment. It becomes a statistical statement because of the suggestion that, due to the results of the study, it is probable that students who, in the future, experience the interaction described, will become more digitally empowered. 

This is, as noted, an example of a very careful generalization, and it is of course appropriate to discuss and question whether it is unproblematic to make unwarranted generalizations, if only the language used is sufficiently careful? Perhaps the argument that researchers should communicate their work very carefully can, if enforced too rigorously, become an obstruction to language as such.

Eksempel på ikke underbyggede udsagn

The following is an example relevant to the hypothesis that danish educational and pedagogical research has a non-zero tendency to make claims with reference to research which cannot support those claims. It is from my reading of danish educational research articles from two journals, in search of unwarrented generalizations. The study mentioned here will not be included in that project, since it is theoretical in nature. Also the following is in danish, except there are alot of excerpts from articles written in english.

Frederiksen, L. L., & Halse, E. (2022). Uddannelse til kompetente mentorer for nyuddannede lærere: Array. Studier I læreruddannelse Og -Profession, 7(2), 24. https://doi.org/10.7146/lup.v7i2.132894

Studiets formål:

"Hvilken  viden  og  hvilke  færdigheder  sætt  es  der  fokus  på  i  nyere  studier  vedrø-rende mentoruddannelse for nyuddannede lærere, hvis uddannelsen skal give de stu-derende kompetencer i at understøtt e den nye lærers professionelle udvikling?" s55

Studiets metode:

"Inspireret  af  Stenalt  et  al.  (2019)  har  litt  eraturstudiets  metode  fokuseret  på  at  udvikle  en  søgestrategi.  Dett  e  har  indebåret  udarbejdelse  af  inklusions-  og eksklusionskriterier, litt eratursøgning og en screeningsproces. I analyse-strategien er vi inspireret af Braun & Clarkes forståelse af tematisk analyse (2006; 2013)." s55

"Søgestrategien  endte  med  ni  studier,  der  var  relevante  i  henhold  til  vores  inklusions- og eksklusionskriterier og samtidig blev vurderet til at have god kvalitet." s59

Generaliseringer:

Citat 1: "Bemærkelsesværdigt  er  det,  at  ingen  af  studierne  primært  eller  i  særlig  høj grad har fokus på skolekulturens betydning for mentoring og den nye lærers  professionelle  udvikling,  trivsel  og  læring,  hverken  når  det  gælder  indhold  i  mentoruddannelsen  eller  udførelse  af  mentoring.  Studierne  har  primært fokus på individniveau, både hvad angår nye lærere og mentorer." s59

Citat 2: "Det står klart, at de forskellige temaer i dett e studie hænger sammen og er afh   ængige  af  hinanden.  Udvikling  af  gode,  tillidsfulde  relationer  er  vigtig  for at kunne åbne sig og refl ektere sammen med andre." s72

Citat 3: "I en mentoruddannelse må der i særdeleshed være fokus på refl eksion i tre perspektiver:  a)  teori,  b)  at  lære  nye  lærere  at  refl  ektere  og  have  en  kritisk  tilgang til refl eksionen og c) metarefl eksion over egen mentoring.De studerende kan også med fordel opøves i en samarbejdende, under-søgende  tilgang  med  fokus  på  educative  mentoring  med  et  trifokalt  blik.  Derudover er viden og færdigheder om relationer vigtigt, både for at kunne give feedback på undervisning om det relationelle og for at kunne etablere gode mentorsamtaler, hvor der åbnes op for kritisk refl eksion. Det er vigtigt at have fokus på det relationelle de studerende imellem i uddannelsen, da det i høj grad bidrager til udvikling af mentoridentitet.  Det er vigtigt at være opmærksom på teoretisk viden og færdigheder om forskellige  mentoringtilgange,  kommunikation,  observation,  teori-praksi-skobling og voksenlæring.Mentorkompetencer erhverves bedre, hvis uddannelsen strækker sig over tid, så man kan kombinere nyerhvervet viden og færdigheder med konkret eksperimenteren  i  praksis,  der  kan  refl  ekteres  over,  f.eks.  de  studerende  imellem eller med erfarne mentorer. I en uddannelse, der strækker sig over tid, kan man med fordel inddrage aktionslæringsprocesser." s71

Vurdering:

Generalisering type: 4 og 5: Første citat er kun med som et eksempel på at forskerne godt ved, at de ni studier de har fundet til deres litteraturreview, i det hele taget ikke giver den store vidensmæssige slagskraft og i den sammenhæng heller ikke fordrer generalisering. Andet citat er et eksempel på en direkte generalisering hvori forskerne fastslår at “...gode, tillidsfulde relationer…” er vigtige for at “... kunne åbne sig og reflektere sammen med andre”. Det antages således at situationer hvori man selv og andre skal reflektere, vil være mere “effektive” hvis man har sådanne relationer. Citatet forekommer i den sammenhæng også almengyldigt. Det betyder dog ikke at studiet kan underbygge påstanden. 

Med hensyn til tredje citat er der i de to studier (af de i alt ni som forskerne finder) som muligvis kan generaliseres, ingen underbygning af de ideer om refleksion som præsenteres. De fokuspunkter/adfærdsregler som gives i tredje citat er således eksempler på en implicit, uberettiget antagelse om at litteraturstudiet har den observationsmæssige styrke til at støtte sådanne statistiske generaliseringer. Statistiske i den forstand at man antager at den foreslåede adfærd, hvis den tages i betragtning, vil have indflydelse i fremtidige situationer i mentoruddannelser.

Gennemgang af de to studier som kunne generaliseres, med henblik på eventuelt viden de måtte producere vedrørende de citerede udsagn om “refleksion”:

I “Schatz  -Oppenheimer, O. (2017). Being a Mentor: Novice Teachers’ Mentors’ Conceptions of Mentoring Prior to Training. Professional Development in Education, 43(2), 274–292. htt  ps://doi.org/10.1080/19415257.2016.1152591”-studiet der præsenterer viden fra 170 undervisere der er startede på mentor-kursus, konkluderes det at deltagerne ikke, i deres refleksioner, tager højde for refleksion som et vigtigt redskab i det at være mentor. For at underbygge hvorfor refleksion er et vigtigt redskab refererer forskerne til en række andre studier, men finder intet bevis i deres eget. Spørgsmålet bliver derfor om disse andre studier kan bakke op om de hypoteser/påstande som forskerne fremhæver omkring refleksion:

“The findings show that although reflection is a key mentoring tool, it is hardly mentioned in participants’ statements. The professional literature indicates that reflective abilities are developed before, during and after teaching (Schön 1983, 1988). At each stage, reflection consists of inner contemplation of the educational act by the person performing it. It is a tool, a means of learning about overt and covert processes (Harrison et al. 2005) that is particularly important to mentors because it enriches the cognitive, emotional and interpersonal aspects of mentoring (Crasborn et al. 2010). Mentors who develop self-reflective ability regarding their performance, who can identify their own weak and strong points, will increase their self-confidence and also encourage novice teachers to trust in themselves and their own qualities, to uncover and examine their working methods and the quality of their performance (Rowley 2006, Cook 2009). Transmission of lesson content based primarily on personal example (role modeling) and not on development of reflective thinking is insufficient and may limit the novice teacher’s success (Jones and Straker 2006). Reflection as a learning act engenders critical thinking that is neither spontaneous nor intuitive, but rather constitutes an integral component of one’s performance, a major objective that calls for intensification of insights concerning the connection between practices and their theoretical foundations, interpersonal communication skills and awareness of the emotional processes that motivate interpersonal ties. All such elements are substantive” s288

“and even critical in MNT work, and should be an essential part of mentor training courses (Molner Kelley 2004).” s289

Citatet kan deles op i seks hypoteser. Hypotserne er som følge:

1: “The professional literature indicates that reflective abilities are developed before, during and after teaching (Schön 1983, 1988)”.

2: “At each stage, reflection consists of inner contemplation of the educational act by the person performing it. It is a tool, a means of learning about overt and covert processes (Harrison et al. 2005).”

3: [hypotese 2] “is particularly important to mentors because it enriches the cognitive, emotional and interpersonal aspects of mentoring (Crasborn et al. 2010)”

4: “Mentors who develop self-reflective ability regarding their performance, who can identify their own weak and strong points, will increase their self-confidence and also encourage novice teachers to trust in themselves and their own qualities, to uncover and examine their working methods and the quality of their performance (Rowley 2006, Cook 2009).”

5: “Transmission of lesson content based primarily on personal example (role modeling) and not on development of reflective thinking is insufficient and may limit the novice teacher’s success (Jones and Straker 2006).”

6: “Reflection as a learning act engenders critical thinking that is neither spontaneous nor intuitive, but rather constitutes an integral component of one’s performance, a major objective that calls for intensification of insights concerning the connection between practices and their theoretical foundations, interpersonal communication skills and awareness of the emotional processes that motivate interpersonal ties. All such elements are substantive and even critical in MNT work, and should be an essential part of mentor training courses (Molner Kelley 2004).”

Det giver ikke mening at undersøge første hypotese, fordi udsagnet er almengyldigt. Selvfølgelig udvikles “refleksive evner” før, under og efter undervisning. Hertil udvikles de konstant igennem livet og derfor også før, under og efter en hvilken som helst af de situationer som livet består af.  Sådanne evner kan selvfølgelig ikke undersøges i nogen nærgående forstand som kobler reelle biologiske fænomener til psykologiske tilstande og adfærdsmæssige situationer. Vi ved at de er der fordi de fleste børn bliver mere eftertænksomme når de bliver ældre.

Anden hypotese præsenteres ligeledes som om den er specifik til undervisning, men de opstillede termer, at det handler om indre overvejelser og læring er også generelle karakteristika ved refleksion i det hele taget. Det er således ikke et informativt udsagn, og det at referere til en kilde for at underbygge udsagnet er unødvendigt. Hypotesen forlænges dog i tredje hypotese som påstår at refleksion, som beskrevet er særligt vigtigt for mentorer, hvilket er en påstand som kræver bevis, fordi refleksion, som sagt, præsenteres i så generelle termer at man skulle antage det lige vigtigt for alle. 

Crasborn, F. et al. -studiet, som i denne sammenhæng skal underbygge påstanden at: “[reflection is] important to mentors because it enriches the cognitive, emotional and interpersonal aspects of mentoring”, fungerer ikke til formålet. Crasborn observerer to mentor-dialoger for hver af tredive deltagere. Én før og én efter at de blev trænet i “supervisory skills”. Studiet forsøger at måle forskellige niveauer af “bevidsthed” (intern definition) i mentor-situationen, samt at kvantificere refleksion. Målet med studiet er således ikke omtalte udsagn, og havde det alligevel fundet noget lignende, kunne resultaterne ikke generaliseres til alle mentorer i Holland, hvor studiet er lavet, grundet mængden af deltagere, og heller ikke til alle mentorer i Danmark. 

Fjerde hypotese er i virkeligheden flere hypoteser, hvorfor der refereres til flere studier. Kan studiernes påstande generaliseres? 

ROWLEY er en træningsbog - nogle studier?

Cook undersøger 10 første-års engelsk-undervisere:

“The purpose of this study is to explore the experiences of fi rst-year English teachers through the lens of phenomenology in order to better understand the nature, or the essence, of this phenomenon—the first year of teaching English. I have investigated what it is about the fi rst year of teaching that makes it such an infamously trying time. In addition to my central research question here, “What is it like to be a fi rst-year English teacher?” I explored several other offshoots of inquiry in this study, such as (a) how novices come to see themselves as teachers, and (b) how new teachers experience their own learning and development “on the job.”” s275

Kunne studiet have vist nogle af elementer i påstanden, kunne resultaterne alligevel ikke generaliseres. 

Femte hypotese underbygges med reference til Jones and Straker 2006, som laver et litteraturreview for at undersøge mentorers rolle i professionel vidensudvikling i fire forskellige discipliner: sygepleje, paramedicin, socialt arbejde og undervisning. 

Studiets formål har således ikke umiddelbart noget med påstanden at gøre, omend der måske kan findes en sætning omkring fænomenet. Måske en grund til at det beskrives så vagt. Det forekommer almengyldigt, at hvis man skal undervise en ny underviser i noget undervisnings-materiale, og de ikke reflekterer over undervisningen, at det ikke er en ønskelig situation. 

Sjette hypotese refererer til Molner Kelley, L., 2004.

Studiet beskæftiger sig med programmer der skal gøre det muligt for nyere undervisere i USA at blive indlemmet i arbejdet på amerikanske skoler. Omkring 146 undervisere inddrages i det hele over en længere periode, og primært med det fokus at se om de forblev på de institutioner hvor de blevet optaget, og hvad der gjorde forskellen. Der er således ingen konsistent ide om “refleksion” i studiet. Det er blot noget man kan få mere eller mindre tid til alt efter hvilken type af program man indgår i som ny underviser. F.eks.:

“Because principals familiar with the PIE Program consistently report high levels of satisfaction with the growth and reflectivity these novice teachers demonstrate, they actively seek new PIE candidates from the applicant pools generated during each hiring cycle. The hope is that developing dispositions toward inquiry and reflection on practice early in a teacher’s career will have a lasting effect in later years”. s446

Studiet omhandler som så heller ikke mentorer, men undervisere. Det kan i øvrigt ikke nødvendigvis generaliseres udenfor USA. 

Referencerne:

Schön, D., 1983. Educating the reflective practitioner. San Francisco: Jossey-Bass.

Schön, D., 1988. Coaching reflective teaching. In: P. Grimmett and G. Erickson, eds. Reflection in teacher education. New York, NY: Teachers College Press, 19–29.

Harrison, J., Lawson, T. and Wortley, A., 2005. Facilitating the professional learning of new teachers through critical reflection on practice during mentoring meetings. European journal of teacher education, 28, 267–292.

Crasborn, F., et al., 2010. Capturing mentor teachers’ reflective moments during mentoring dialogues. Teachers and teaching: theory and practice, 16 (1), 7–29.

Rowley, B.J., 2006. Becoming a high performance mentor: a guide to reflection and action. Thousand Oaks, CA: Corwin.

Cook, J.S., 2009. Coming into my own as a teacher: Identity, disequilibrium, and the first year of teaching. The new educator, 5 (4), 274–292

Jones, M. and Straker, K., 2006. What informs mentors’ practice when working with trainees and newly qualified teachers? an investigation into mentors’ professional knowledge base. Journal of education for teaching, 32 (2), 165–184

Molner Kelley, L., 2004. Why induction matters. Journal of teacher education, 55 (5), 438–448.

Konklusion på Schatz  -Oppenheimer, O. (2017).

Som beskrevet underbygges Schatz - Oppenheimers påstande omkring refleksion kun i meget begrænset omfang. Således kan Frederiksen, L. L., & Halse, E. (2022), ovennævnte litteraturstudie, ikke umiddelbart drage sine konklusioner fra nævnte studie. 

Det eneste andet studie som kunne generaliseres var Lejonberg, E., Elstad, E., & Christophersen, K.-A. (2015) (Lejonberg, E., Elstad, E., & Christophersen, K.-A. (2015). Mentor education: challenging mentors’ beliefs about mentoring. International Journal of Mentoring and Coaching in Education, 4(2), 142–158. htt ps://doi.org/10.1108/IJMCE-10-2014-0034). Studiet baserer sig på spørgeskemabesvarelser fra 146 mentorer, som deltog i mentor-undervisnings-programmer i Norge. Studiet testede tre hypoteser og et udforskende spørgsmål:

H1: “Completion of mentor education is related to lower levels of beliefs consistent with judgementoring.” s146

H2: “Higher levels of reported role clarity are related to lower levels of beliefs consistent with judgementoring.” s147

H3: “Lengthy experience as a mentor is related to lower levels of beliefs consistent with judgementoring.” s147

Udforskende spørgsmål: “Does the reported level of self-efficacy related to the mentor role predict the level of beliefs consistent with judgementoring?” s146

“Judgementoring” som er et kernebegreb i undersøgelsen defineres:

“[…] a one to one relationship between a relatively inexperienced teacher (the mentee) and a relatively experienced one (the mentor) in which the latter, in revealing too readily and/or too often her/his own judgements on or evaluations of the mentee’s planning and teaching (e.g. through “comments”, “feedback”, advice, praise or criticism), compromises the mentoring relationship and its potential benefits (Hobson and Malderez, 2013, p. 90).” s143

Det handler altså om hvorvidt forholdet mellem en underviser og en mentor med mere erfaring end underviseren, kompromiteres af en fra mentorens side, underudviklet teknik. 

Givet hypoteserne og studiets vinkel er det ikke underligt at der i studiet ikke rigtig konkluderes noget, som kan underbygge det tredje citat (fra studiet som det her handler om: Frederiksen, L. L., & Halse, E. (2022)).