T1 this week is a research summary of a great paper I read on the impact of devices in the classroom. Since reading it I have changed my approach slightly. I hope you find the research summary helpful.

T2 was interesting too, at the end of it I include some critical questions that I'll be asking myself in future when I read an RCT.

T3 I'm particularly excited about and I'll be trying out these Teach Like a Champion resources in the near future.

I was also surprised by the quality of the STEM Project resources linked to in T4. There are a heap of projects from simply fun projects like balloon cars and spaghetti bridges, to more ‘global citizen' type projects that require students to design floating gardens to help people in areas that are flooded due to climate change. Well worth a look!

T5 a great intro to retrieval practice.

T6 and T7 are two for the literature teachers. However, T6 is pretty applicable to all teachers with its in-depth treatment of the structure of knowledge!

T8, The education practice guides are a great resource that are well worth storing away for the next time you want to check out the resource base for a multitude of topics.

And T9, 10, and 11 can speak for themselves…

Enjoy : )

(all past TOTs here), sign up to get these articles emailed to you each week here.

The impact of electronic devices in the classroom. This paper had a big impact on me (research paper summary)

Last week I read a paper that has had a big influence on my understanding of the impact of electronic devices in the classroom. The paper was by Arnold Glass and Mengxue Kang and, interestingly, considered the impact of electronic devices in both the short and long term. The first thing that struck me was the really creative study design. To minimise any findings attributable to differences between control and intervention groups, they simply made them the same group! They allowed the 118 college students to use devices ½ the time.

Impacts on short-term performance were measured by a questionnaire that was administered at the end of EVERY single class. Students answered MC questions to explore the effect (if any) of having the option of using devices in class. Here are the results:

Hmm, peculiar. In classes in which students were allowed to use their devices, the class average was actually higher than in the ‘Not Allow’ condition (difference was not significant). Glass and Kang speculate on the reasons for this:

Essentially, they argue that students are pretty good at working out when to listen, and when not to listen. This strategy seems to sufficiently facilitate performance in the short term, especially when the stakes are recall on a soon-to-be-asked MC question. To consider longer term performance, Glass and Kang looked at student performance in the unit exam and the final exam. They compared the ‘allowed’ and ‘non-allowed’ conditions by considering results for content taught in device allowed vs. non-allowed lessons. Results:

As can be clearly seen, the impact of allowing devices in the classroom was negative for both the unit exam, and the final exam, with the impact being more pronounced the longer the delay before testing (difference for Unit and Final were significant). Glass and Kang comment:

But just when I thought it was all over, the plot thickened! When students are allowed to use devices in class, clearly, not all students will do so. Thus, the authors next examined the diff in achievement between those who did and didn’t use devices when allowed. Results:

Now, to me this was the most surprising and depressing finding. Even if a student chooses to not use a device, simply being in a lesson in which device use is allowed will reduce their performance in the final exam. WHAT??? Glass and Kang’s analysis makes a lot of sense:

Essentially, if Stevie next to you is watching hilarious youtube clips, or playing a game, even the most studious student is likely to get involved in what happens when you put a propane tank in the microwave, or whether or not Stevie is going to make it to the next level.

Two key points to emphasise. The first is the impact of the lack of negative impact on performance in the short term, but large impacts on learning. This is dangerous because it means students have a distorted sense of the impact of devices on their learning:

The final point of concern regarding this paper is that the negative impact of devices in this context represents a lower bound. This is because all presented content was replicated in textbooks and slides, and the learning was bolstered by the retrieval practice.

The next day after reading this paper I went into class, told my students about it, and told them that, because of this study's findings, I was going to be more strict on the ‘no phones’ policy from now on. However, I’d be reluctant to generalise from this study to the use of… all devices by all students. The next thing that I’d love for researchers to look at is HOW students use their devices in class. I have an inkling that there’s a subset of sdts who have the self-discipline and know-how to use devices effectively for learning. I’d love to… see this subset considered in more detail to examine whether the impact of allowing devices on learning reverses (is positive) for such a subset. The Q after that would be, ‘Can we teach all students to use devices in a way that enhances rather than diminish their learning?’. I hope someone does this research soon. Alternatively, I'd be grateful for any pointers to any such research already out there. In the mean time, you can find the paper here: (http://psycnet.apa.org/record/2018-37560-001)

A systematic review of RCTs in education. Strengths, weaknesses, further directions? (research paper summary)

A systematic review of the use of randomised controlled trials in education has just been released. It aimed to answer four key criticisms of RCTs in education. Here are the four criticisms it addressed.

The authors, Paul Connolly, Ciara Keenan, and Karolina Urbanska, (Connolly is a big name in educational RCTs, authoring the book: Using Randomised Controlled Trials in Education) present the following responses in relation to the Qs. Criticism 1: You can’t do RCTs in education.

And it’s interesting to see how the prevalence of RCTs has increased over time.

Criticism 2, RCTs are blunt research designs that ignore context and experience. Response:

Criticism 3, RCTs tend to generate simplistic ‘universal laws’ of ‘cause and effect’. Response:

Criticism 4, RCTs contribute little to theory. Response:

In sum, the authors suggest that it isn’t surprising to see criticisms of RCTs, given that nearly two thirds them have not included a process evaluation component. Further, ½ have not looked beyond the overall effects to consider impacts on sub-groups. The authors also suggest that ‘it is difficult to challenge the view that RCTs promote a simplistic, atheoretical approach to educational research when nearly 40% of trials in this analysis have failed to reflect upon the implications of their findings for theory.’

However, let’s avoid baby + bath water fiascos.

For me, the takeaway from this paper is that when I read an RCT from now on I’ll be asking myself three main questions: 1. ‘Have the researchers evaluated their process? (think: fidelity measures, ATI factors, the ‘who’ of the intervention),

Have they reported on context, generalisability, and differential impacts on subgroups? 3. How have the researchers related their findings to theory? 

For more detail, you can find the paper (free) here: (tandfonline.com/doi/full/10.10…)

Teach Like a Champion online Training Modules, via @Doug_Lemov

If you're looking for STEM project resources, look no further! HT @hrogerson

Retrieval practice, a great summary page, via @vandycft

How can we build, structure, and link knowledge in English literature, via @tharby

How can literacy teachers effectively use multiple choice questions? via @A_R_zanetti

Education Research ‘Practice Guides', readable and well evidenced research, covering a whole host of topics

A good catch-all post regarding the recent phonics debate

Parents don't just shape their children, it goes the other way too

Good collection of studies on gender inequality, via @kenwongart

Related