Arnold Glass (AG): So I dreamed about how I would use all those technologies before they were invented. So as soon as they were invented, I was primed to go out and to start to use them to see in what ways I could now apply laboratory phenomenon and actually improve exam performance. So that was the environment in which first I had success and then as I mentioned, I had failure, which got me to look around and see the unintended social consequences of smartphones, which had led to unintended cognitive and classroom effects of smartphones.

Kaitlyn Regehr (KR): That was Arnold Glass, Professor of Cognitive Psychology at Rutgers University. Much of Arnold’s research has focused on the psychology and neuroscience of learning, and particularly on how teacher instructions affect learning and memory – and that’s what we’ll be discussing today.

[How Researchers Changed the World introductory music]

KR: Welcome to How Researchers Changed the World: a podcast series supported by Taylor & Francis which will demonstrate the real-world relevance, value and impact of academic research; and highlight the people and stories behind the research. My name is Dr. Kaitlyn Regehr. I’m an academic researcher; an author and a scholar of digital and modern culture and I’m interested in how new technologies can broaden the reach and real-world impact of academic research.

In today’s episode, we’re speaking with Arnold Glass and exploring his research into the use of technology in the classroom. Specifically, we’ll be unpacking his 2018 paper: ‘Dividing attention in the classroom reduces exam performance’. Arnold’s research has centered on the field of cognitive psychology. That is, the scientific study of human mental processes, such as attention, perception, memory, thinking, and creativity, and how these processes affect behavior. It’s a field you’re likely to have heard of by now, and tends to be a key focus of college and university courses in psychology but that wasn’t always the case. When Arnold first started exploring the world of psychological research, ‘cognitive psychology’ didn’t even exist.

AG: I came along just at the time when psychology was undergoing a revolution. So as late as 1967 if you went to study psychology in college, you studied either experimental psychology or social psychology or maybe clinical psychology. By ’67 a book had been written called cognitive psychology by a professor by the name of Ulric Neisser and that had sparked a revolution and over the next four years psychology departments were transformed, they all had cognitive psychology programs for the first time and cognitive psychology became a driving intellectual force over the next forty years.

KR: Ulric Neisser is widely referred to as the ‘father of cognitive psychology’, and he was the first person to hypothesize that a person’s mental processes could be measured, and therefore analyzed. Before Neisser, the dominant approach to understanding human behavior was ‘behaviorism’, which assumed that all behaviors are either a response to the environment around us, or a consequence of our history as a person. It’s still an approach that is used today, but it only gives a partial view of the reality, because it focuses solely on what can be observed and ignores other factors such as thoughts and emotions.

So, you can see why Neisser’s work was so revolutionary. In 1967 he released the book Cognitive Psychology. It challenged the assumptions of behaviorism and it presented his alternative. It was an instant success, and it transformed the way psychology was taught. And Arnold was right there during this transformation.

AG: So, coming in right at the beginning of this, I could see what everyone else could see, that there were great things to be done in studying human cognition. So I picked that up right from the day I was interested in psychology. I wasn’t just interested in psychology; I was interested in cognitive psychology and I’ve been carried along by that wave ever since.

KR: Within cognitive psychology, Arnold’s work has specifically focused on learning and memory. As a lecturer and teacher himself, his research has had personal relevance too, in terms of how he teaches his own classes.

AG: I always thought that if there’s anyone in academics who should be able to teach an effective college course, it should be someone who’s spending their life studying learning and memory.  I had a special obligation that if they should learn any courses, I should apply what I knew, and they should do well in my course.

KR: Interestingly, Arnold also views his own classroom as a sort of experimental laboratory. His aim is always that his students get the best grades they possibly can in their exams, and he likes to test different methods to ensure he’s giving them the best shot he can as their teacher.

AG: My entire classroom is this ongoing test bed in which I investigate new instructional methodologies. And for many years as I introduced new instructional methodologies, it was very gratifying to see students were doing better and better every single year. But about 10 years ago their performance began to plateau. They were no longer getting better. In fact, their performance in the, on exams began to decline and then decline significantly. So this was a concern to me.

KR: Naturally, Arnold wanted to understand why the exam performance of his students was starting to decline, and if he could rectify it. So, he started to investigate. One thing that quickly became apparent to him was that students were looking up at him less during lectures. It might sound insignificant, but for Arnold it was the start of a hypothesis for why students’ exam performance was suffering. 

AG: One obvious reason why performance had started to decline was up until about 10 years ago when I would prepare my lectures and demonstrations for the classroom that were all carefully constructed so that they would learn the material and do well on the exam. Students would look up at me when I was lecturing, and they would put all their effort into the demonstrations. But what had changed now is that students in the classroom were only occasionally looking up at me. They were spending their primary time paying attention to their cell phone or their tablet or their laptop, no matter what was going on in the classroom. So, they had turned the classroom experience from a single-tasked experience into a divided attention task.

KR: For Arnold this discovery was particularly frustrating, because he had had such high hopes previously for how technological advances could enhance learning for his students.

AG: All through the last century, I really dreamed about the kind of technology that has become available in this century. In particular, personal response systems, clickers so you can continually interact with the classroom and continually get responses, so you knew what they learned at every single moment and course platforms so you would know how much time they were studying, what they were studying and what they knew then because you could ask questions online which they would answer. You would have everything timestamped. So I dreamed about how I would use all those technologies before they were invented. So as soon as they were invented, I was primed to go out and to start to use them to see in what ways I could now apply laboratory phenomenon and actually improve exam performance. So that was the environment in which first I had success and then as I mentioned, I had failure, which got me to look around and see the unintended social consequences of smartphones, which had led to unintended cognitive and classroom effects of smartphones.

KR: In reality, it seemed like technological advancements had made teaching more difficult by making divided attention in the classroom the norm. And of course, this isn’t just in the classroom. Today we’re all used to meeting a friend for a coffee and having to compete with their phone for attention!

AG:  When I was in school it was clearly understood by human beings in general that when you talk to someone, you look them in the eye and when you, when someone was talking to you, you looked them in the eye. And that was an absolute social rule, and to look away when someone was talking to you was a social transgression. This was so well established that if you were in a classroom or a lecture hall and someone, the teacher was speaking at the front of the room, they’d just assume that all eyes would be upon them. And all the students would assume that they would be looking at the instructor and in fact if at some point someone actually turned to the person next to them to make a comment and they were noticed by the instructor or someone else having done that, they could well be called out. Someone could say “well what are you doing? Why don’t you share it with all of us or even if since you’re not interested, please leave the room.”

So that was a very well understood social convention. And I myself got called out for it. I mean I wasn’t perfect. There were times when the teacher would be saying something, and I would actually have a book in my lap and I was trying to divide attention and the instructor saw me. And they would walk up to me and they’d take the book, close it up and carry it away because I wasn’t supposed to be doing that and I knew I wasn’t supposed to be doing that. So that was a very strict social convention. And then cell phones were invented, and then we came to have more and more interesting things on them. And people developed very strong habits for constantly checking them. And the, since then, the entire social convention of giving undivided attention to someone who is talking to you has simply, completely eroded away. And it’s obviously not part of society today at all.

Look, I mean you have such extreme cases as Benedict Cumberbatch stopping a performance to tell the audience that it was extremely distressing to him to try and give a performance while people in the front row were checking their cell phones during his performance. So, you can see the extent to which it is eroded, and people paid a lot of money for those tickets but are still so habitually forced to continually check their cell phones that they’re dividing in a situation like that. So, so yeah that whole convention has completely disappeared.

KR:  And if theatre-goers can’t concentrate on the performance of Benedict Cumberbatch for an hour without checking their phone, then it goes without saying that students were struggling to make it through Arnold’s lecture without checking theirs too!

Arnold knew from looking at previous research that by dividing your attention between more than one task, or ‘multi-tasking’ as it’s more commonly known, you learn less from all of those tasks. But this had never been explored specifically within a classroom setting. So, was it as Arnold suspected, that students were limiting their learning by using their phones, tablets, and laptops during lectures? He had to find out – and what better way to do so than to use that ‘ongoing test bed’ of his own classroom? We’ll delve further into exactly how Arnold did this, after this short break.

[Break intro music]

In this podcast, we explore the multitude of ways that research impacts the world, from the influence on political discussions to how it can revolutionize practices within a field, to how it can shape public opinion and worldviews. We believe that every researcher has the power to change the world in some way and we want to help make that happen for you. That’s why we’ve worked with the team at leading publisher, Taylor & Francis Group, to develop two free 12-week learning programs for researchers.

If you’re an early career researcher, our learning program delivers the complete guide to getting your research published and establishing your research profile. For mid-career researchers, our second learning program is a go-to guide for managing mid-career challenges, boosting the impact of your published work, and enhancing your research profile. 

Our learning programs are designed to fit your workload, so they’re delivered to you online with one email each week for 12 weeks. That means you can access each chapter whenever it’s most convenient for you. Plus, you can save emails and come back to them throughout your research career. 

Want to know more? Head to howresearchers.com/learning-programs. 

[Break outro music]

Before the break Arnold had his suspicions that his students’ use of phones, tablets, and laptops during class was having a negative impact on their final exam results. He tested this theory with a classroom experiment.

AG:  What I did is I, what I always do when I go and I want to test a factor that may affect learning and memory. I teach the same class back to back Tuesdays and Thursdays.  So on Tuesdays in one class students were not allowed to use any electronic device, they could only pay attention to me. And on Thursday for the other class, students were not allowed to use any electronic device for an extraneous purpose, they could only pay attention to me. And then I looked at the performance for both classes on the exams. And it was not a shock to discover that the material that was presented on the days that students were allowed to ignore me, they did much more poorly than the exams that covered material on the days that they were forced to pay attention.

So that’s the result that I wrote up that has appeared now in the report: ‘The effect of divided attention in the classroom.’ The effect is that if you divide attention in a classroom you do more poorly on your unit exams, and you do even more poorly on your final exams. By the final exam the difference was 87% correct for material where students had to pay attention and only 80% correct when they were allowed to divide attention.

KR: That’s a huge difference! 7% could easily mean the difference between one grade and the next. It’s easy to see why the exam results of Arnold’s students were decreasing, but you wouldn’t think that something so small as checking your phone during class could have such a large impact on your final results. So why is this?

AG: That’s because the main, one of the effects of divided attention is specifically to affect long-term retention as opposed to immediate retention. So the further out in time you measure retention of what they learned, the greater the difference you get between people who were paying attention to one thing and people who were dividing attention to more than one thing.

Unfortunately, because this affect is long-term, people are unaware of it in the moment. So, in the classroom, even in the divided attention classroom, if I immediately asked a question on what I had just presented, even students who were engaging in divided attention were still able to answer that question correctly. So naturally they assume that therefore they had learned it and remembered it just as well and would remember it just as well because you don’t have any intuition if you know something in the moment, whether you’re gonna still remember it a week or two weeks later. But that insidious affect is nevertheless there. So even though they thought they would remember it one or two weeks later because in the moment they seem to be aware of everything that was going on, in fact two weeks later, performance is much poorer for situations in which you divided attention.

You essentially remember you were in that situation and you were dividing attention between the two tasks, but you remember very little of each task, and that’s what lead to the decline in exam performance for the students in the class.

KR: From his classroom experiment, Arnold had the first data to suggest that his hypothesis was correct: that using phones, tablets, or laptops during classes was negatively impacting students’ exam results because their attention was divided between multiple tasks. But what next? Could Arnold stop students using technology in classrooms?

AG: I had very good evidence that if you, if a student divided attention in class, they were going to do less well on their exams. And I also had pretty good evidence that the students themselves had no awareness of this effect and therefore would doubt that it was true. And therefore I felt that as a citizen, as someone who was doing all this as a teacher to help the students, that therefore there was a special obligation on me, since I was the one who first got this finding, to do anything I could to bring this to the public’s attention. So at least people would be aware of this as a likely consequence of their actions, because I couldn’t expect students to change their behavior if they had no awareness of the finding in the first place. So I felt that it was up to me to make as many students as possible aware of this so at least they appreciated that there was a genuine consequence to dividing attention in class and then they could decide for themselves at least, even if I wasn’t going to get institutional rules in place, of whether or not they wanted to engage in the behavior.

KR: It’s fair to say that Arnold achieved this. He published his results in 2017, with a paper entitled ‘Dividing attention in the classroom reduces exam performance’. Since publication, the paper has been read over 26,000 times. It’s also made a huge splash in the media, with 45 news articles written about the paper, meaning Arnold’s research was able to reach much further than his own academic field. Of course, this doesn’t mean that there was a mass ban on mobile phones in classrooms…

AG: We have a long way to go before people realize that this is, doesn’t do anyone any good in the classroom and it should not be permitted. And there will be push back. I heard a talk, a school district made students when they came to school, they had these pockets where you can buy them from various vendors where they would put the cell phones in the pocket and they would seal the pocket up and no one else could take the cell phone out because it had a lock on it. At the end of the classroom day, they could go and they could open the pocket and take their cell phone and go about their business. And what the very savvy teachers and principle of the school discovered, was that the students were bringing burner phones to class and putting them in the pockets and keeping their real phones to continue to use them.

KR: But it’s a great starting point for increasing our awareness and understanding about this important topic and Arnold is very clear that more work needs to be done to further his research.

AG: I think a lot more research needs to be done because considering how important this is, it’s amazing how little research has been done on the effects of new technologies.

It’s, we have to distinguish between two kinds of research, one kind of research… well even three kinds. One kind of research is survey research where you just, where you find out how much people are doing different kinds of things. The next kind of research is correlational research. You collect information about different changes in society and then you see what’s correlated with what. That’s very useful to do but correlational research does not demonstrate cause and effect, and you can’t really infer it from correlations. So then we come to the kinds of experiments you need to do to understand what’s actually happening in the world. Experimental studies where you manipulate a variable and therefore at the end you can determine cause and effect, like I did in my experimental study. If you look at the number of experimental studies that have been done, right, to demonstrate that divided attention in the classroom impairs exam performance, the answer is two. Me and one other person, where they had a lot of control, a group of people at West Point Military Academy so they had good control. Other than that, there are no experimental studies. And that’s terrible considering how important this is in the world.

But there are no experimental studies because there is no national science agency that will fund experimental studies of this topic. There are no private philanthropies that will fund experimental studies in this topic. It is difficult to get studies of this topic published in prominent journals. This is just something that is not a priority for society and the world, and so it’s just happening without any awareness or any intervention. And that’s a bad thing because the effects are much larger than anyone realizes and they’re not going to realize it because they never study it.

KR: Even though he acknowledges that further research needs to be done to persuade more and more people of the importance of this topic, Arnold remains positive about the power that research has to ultimately change the status quo.

AG: It’s hard to think of profound changes in history which don’t go back to fundamental scientific discoveries. From the renaissance forward, that’s what’s been driving progress. It’s been scientific discovery all around the world and it’s not, you know, slowing up.

KR: This is one area where we really need that scientific discovery and progress. To find out more about this podcast and today’s topic, visit howresearchers.com/dividedattention. In the next episode of How Researchers we’ll be speaking with Siobhan Brooks on her paper ‘Black on Black love: Black lesbian and bisexual women, marriage and symbolic meaning.’

We’d love to hear your feedback so please follow us on Twitter, Facebook or LinkedIn @howresearchers. This podcast was written and produced by Monchü and recorded at Under the Apple Tree Studios. Our producers were Ryan Howe and Tabitha Whiting with editing, mixing and mastering by Miles Myerscough-Harris at WBBC. We would like to acknowledge the incredible support of Taylor & Francis Group with a special thank you to Elaine Devine and Claire Dodd.

I’m Dr Kaitlyn Regehr. Join us next time for How Researchers Change the World. Thanks for listening.

TW:  Hello listeners, this is Tabitha Whiting from the How Researchers Changed the World team. We wanted to let you know that we’re now going to be taking a short mid-season break, and our next episode, with Siobhan Brooks, will be out in 4 weeks time. Make sure you subscribe to the How Researchers Changed the World podcast on your podcast provider to be notified when the new episode is available in August.

In the meantime, this is the perfect chance to go back and binge listen to any episodes you’ve missed! So far we’ve covered research on music therapy and dementia, the p-values controversy, conservation and tree planting, the ethics of AI, art therapy, and technology in the classroom. Whilst you’re at it, we’d love to hear your feedback so that we can keep improving this podcast – so make sure you leave us a review or rating on your podcast provider. And if you’re a budding researcher yourself, then don’t forget to check out our 12-week learning programs at howresearchers.com. See you in August!