Numerous past studies have found that individuals have a limited ability to perform multiple tasks simultaneously. Moreover, a negative impact on memory has been found when subjects engaged in divided attention tasks at time of encoding. This study examined the effect of device-delivered notifications on the accuracy of memory through the use of a word recognition paradigm. Two groups of students were presented with the same word list and following this were tested with a two-alternative forced-choice recognition test consisting of previously presented and novel words. One group of students received numeric notifications during presentation which they were instructed to note down while the other group received no notifications. The results showed that false alarm rates were significantly greater and hit rates were significantly lower for the group which received notifications, demonstrating a mirror effect, and showing that notification-derived divided attention had a negative impact on memory.
Keywords: divided attention, two-alternative forced-choice recognition, multitasking, notifications
Technology has become incredibly pervasive in modern society, for a multitude of reasons, not least of which has been the increasing portability of electronic devices. For instance, it was reported by Statistics Canada (2017) that approximately 76% of Canadians owned a smartphone. This rapid spread of technology has led to an increased sense of connectivity between individuals, and indubitably has generated a slew of various benefits. However, the rapid spread of portable technology was also found to have significant downsides with a significant amount of research focused on the potential dangers and downsides of using such technology in various contexts. Perhaps one of the most widely reported risks of using portable technology was the increased number of distracted driving accidents caused by drivers using cellular devices while driving their automobile. Specifically, according to the National Safety Council (2013) as many as 27% of all automobile accidents that occurred in the United States in that year were caused by drivers who were distracted by their cellular devices. Moreover, other studies have found that operating a cellular phone while driving was as dangerous as driving drunk due to the distraction that the use of a cellular phone caused the driver (Strayer, Drews, & Crouch, 2006).
Clearly, despite the numerous demonstrated benefits of mobile technology, researchers found significant downsides such as the powerful ability of this technology to distract its users. However, the popularity of mobile technology and its wide set of applications and capabilities has led to its acceptance in a variety of contexts and environments, including most workplaces and places of education ranging from elementary schools through to universities. Therefore, the majority of the student and working population alike has become likely to have some form of technological device in their possession at all times and moreover, such devices are being actively utilized in most classroom and workplace settings. This effect had been examined in the educational context by a variety of researchers who found that when students utilized devices in the classroom setting, they effectively divided their attention between their device and the lesson being taught (Glass & Kang, 2018). Moreover, even if such devices are not being actively used, many applications for such devices create audible and/or tactile notifications that are designed to capture the attention of their users. Such notifications themselves have been shown to potentially lead to attention-related errors even if they were not actively attended to by the subjects (Marty-Dugas, Ralph, Moakman, & Smilek, 2018).Therefore, the use of portable devices in their various shapes and forms had been found to lead to a divided attention phenomenon. Importantly, a large body of research had been conducted on this phenomenon both prior and following the advent of mobile technology which concluded that human ability to split attention between multiple tasks was severely limited (Broadbent, 1958). On this basis, researchers examined the effects of multitasking involving technology in the educational setting. Specifically, the study conducted by Hembrooke and Gay (2003) was conducted to determine whether students who used technology during lecture demonstrated significantly decremented memory for the content of the lecture as opposed to students who were not allowed to use such devices. In other words, if the results of the experiment demonstrated that the students who used technology at the time of encoding demonstrated worse memory performance than those who did not, this would indicate that technology had an adverse effect on memory.
In the experiment conducted in the Hembrooke and Gay (2003) study, the researchers assigned students to two conditions wherein one half of the subjects were instructed to use their devices during the lecture and the other half of the subjects were instructed to not use them. Following the lecture, both groups were tested on their memory for the content of the lecture. This process was replicated with the two groups of subjects switching roles wherein the students which were allowed to use devices in the first part of the study were prohibited from using them in the second part, and vice versa. The results of the study showed that the students that were allowed to use technology during the lecture performed significantly worse on the recognition test in both the original and the replicated portion of the study. The researchers concluded that the students that were allowed to use devices during lecture effectively performed multiple tasks simultaneously during the lecture, and it was this that led to the significantly lower scores that that group demonstrated at the time of testing.
A separate study was conducted by Glass and Kang (2018) in order to further investigate the role of technology in the classroom and its role in creating a divided attention effect. In this study, subjects were also separated into two groups wherein the first group was allowed to use devices during lectures held on Tuesday and not during lectures held on Thursday. The second group was allowed to use devices only during lectures held on Thursday and not during lectures held on Tuesday. The memory of the students was tested during each lecture, and also with final and unit exams. Therefore, this study extended the study conducted by Hembrooke and Gay (2003) in that the subjects were tested over 23 lectures rather than just one.
The results of the study demonstrated that while the performance of students on the daily classroom quizzes was not affected by whether students were allowed to use their devices, the exam performance of the students was worse for the material that was taught in the classes when devices were permitted. The researchers therefore concluded that while short-term memory and comprehension was unaffected by technology, long-term memory was affected, indicating that the effect of divided attention caused by technology affected long-term retention of material.
Therefore, these studies demonstrated that the usage of technology caused worse memory performance, due to the fact that the subjects split their attention between the target content and using their device. The study conducted by Stothart, Mitchum, and Yehnert (2015) investigated whether the auditory and/or tactile notifications generated by such devices could themselves be distracting to the subjects. In other words, the researchers sought to examine the situation in which subjects did not actively utilize their devices throughout the study period but rather merely received notifications and the researchers wished to determine whether such a condition would cause a similar distractive effect. The subjects were split into three groups where the first group received calls during the experiment, the second received text messages, and the third received no notifications from the experimenters. The subjects were presented with numbers that were presented for 1 second with participants instructed to press a key each time a number was presented, unless it was a certain nontarget lure. The results were analyzed with two measures of attention performance which had been previously found to be associated with task-unrelated thoughts (Cheyne, Solman, Carriere, & Smilek, 2009). Interestingly, the results showed no significant difference between the two kinds of distractors used in the experiment. In other words, participants showed a similar decrease in memory performance both when their devices were ringing and when they merely received notifications. Therefore, the researchers concluded that even unattended device notifications still significantly impaired the performance of subjects on attention-demanding tasks.
Given the results of these studies that indicated that the use of technological devices during study had an effect on the memory performance of subjects, and particularly that even unattended notifications had a significant negative effect on attentional tasks, the current study was designed to extend prior research on the distracting role of technology and its negative impact on learning and memory tasks. Specifically, as the use of devices in general and notifications in particular had been found to cause divided attention in subjects, this experiment was designed to examine the impact that notifications had on memory. The participants of this study were divided into two groups wherein the experimental group received regular notifications during the study period while the control group received none. Both groups were separately presented with the same 40 unrelated words and both groups of subjects were instructed to attempt to memorize the words. The experimental group received 3-digit number notifications throughout the presentation of the words and were instructed to note them down. Both groups of participants were then presented with identical word recognition tests that consisted of previously presented and novel words, with participants asked to identify the old words.
On the basis of the previously described research, it was predicted that the memory performance of the subjects who were given the divided attention task would be worse, with those subjects demonstrating greater false alarm rates and lower hit rates on the test than the subjects in the control conditions. This would be in accordance with the findings made by Stothart et al. (2015).
Participants
Participants were 20 undergraduate university students enrolled in a senior laboratory course in cognitive psychology at Wilfrid Laurier University. Participants were randomly assigned to one of two groups: the group receiving notifications during study or to the control group which did not receive notifications during study. Subjects were compensated for their participation through participation marks that were awarded towards their course grades for taking part in the study.
Apparatus and Materials
The subjects were presented with a study list that consisted of 40 unrelated words. The words were selected from the MRC Psycholinguistic Database and were unrelated common nouns, 5-8 letters in length, and that were rated from 300-500 on the concreteness and frequency scales that vary from 100-700. The total number of words in the word pool was 60. All stimuli were presented on a screen located at the front of the classroom through the use of the Google Docs web-based service and an overhead projector. The notifications were delivered to participants through the use of the web-based Slack client, with each participant signed into a separate account, with 12 Slack accounts being created for this purpose. The notifications were delivered through the classroom desktop computers, with one computer assigned to each participant in the notification condition. The numbers that were used for the notifications were three-digit numbers that were randomly generated using the Random.org website.
Procedure
The experiment was conducted in a computer laboratory classroom. Subjects were randomly assigned to either the notification or control condition; the students assigned to the second condition were instructed to leave the classroom and wait in the hallway. The remaining students were situated at seats distributed throughout the room, in front of computers that each had a browser open to a logged-in Slack session with the volume set to a level of 60%. Each of these participants also had a physical response sheet for recording the numerical notifications delivered via Slack. The study list was projected onto a screen positioned at the front of the classroom.
The students were then presented with the study list that consisted of a total of 40 words that were presented at a rate of 2 seconds per item. Each study word was presented in uppercase in a black Calibri font of size 60 centered on a plain white background. The subjects were instructed to try to remember each word that was presented to them and to also write down the 3-digit numbers that were delivered to them via Slack notifications. During the presentation, participants received notifications every 10 seconds, for a total of 8 times throughout the presentation period. Following the presentation, there was a 1.5-minute retention interval between the study period and the test which was occupied with distributing the physical response sheets for the test and communicating instructions to participants. After this, the subjects were presented with a 2-alternative forced-choice recognition test that consisted of 20 words that were previously presented and 20 new words, for a total of 40 words. The test words were shown at a presentation rate of 4 seconds per item. Each test word was also presented in uppercase in a black Calibri font of size 60 centered on a plain white background. The participants were asked to indicate whether they believed that a given word had been shown previously in the study list by circling ‘Yes or ‘No’ on their test response sheets at the time of each presentation. The results were recorded by the participant physically and processed electronically by the researchers. These results indicated the individual accurate recall rate for the old words and the false alarm rate for the new words.
Following this, the response sheets were collected by the researchers and the first group of students was sent out into the hallway and the second group was invited into the classroom. This group was presented with the same study list shown at an identical rate of presentation but did not receive Slack notifications during the presentation. Following the presentation, an identical retention period was used, and participants were tested using the same recognition test as the first group, using an identical procedure.
The main effect of recognition type was reliable, F(1,17) = 35.5, MSe = .035, p < .001. The hit rate was greater in the control condition (.68) than in the notification condition (.59), indicating that word recognition was significantly higher for the words that were presented in the study list rather than for the novel words in the test list. At the same time, the false alarm rate was found to be greater in the notification condition (.37) than in the control condition (.14). The main effect of the group type was not found to be reliable, F(1,17) = 2.641, p = .123. This suggests that neither group answered with a ‘Yes’ response significantly more than the other. Finally, a significant interaction effect was found between recognition type and the group type on word recognition, F(1,17) = 6.06, MSe = .035, p =.025. This indicated that there was a significant interaction between recognition type and group type on word recognition, such that correct word recognition was greater for the control group than the notification group. Moreover, the false alarm rate was found to be higher for the experimental group than for the control group, therefore demonstrating a mirror effect. Put another way, the group that was distracted by notifications during presentation made significantly more false alarms and made significantly fewer hits than the participants in the control condition.
In order to further interpret the interaction, Corrected Recognition Scores were calculated which were equivalent to the hit rate less the false alarm rate. The mean and standard deviation proportions of the Corrected Recognition Scores for each respective group are shown in Table 2. An independent t-test was performed on the Corrected Recognition Scores and it was found that the participants significantly differed in their recognition performance, t(17) = -2.462, p = .025. Specifically, participants in the notification condition demonstrated a significantly worse performance at recognizing previously studied items, thus demonstrating that the split attention effect that was caused by the notifications in the learning context resulted in a diminished memory for the learned content.
The results of the experiment indicated that the main effect of recognition type was reliable, with a higher hit rate in the control condition than in the notification condition and with a higher false alarm rate in the notification condition than in the control condition. There was also no reliable main effect of group type, suggesting that neither group claimed to recognize old words significantly more than the other. It was also found that there was a significant interaction effect between recognition type and the group type. This implied that the correct word recognition rate (hit rate) was higher in the control condition than in the notification condition, and that also the false alarm rate was greater in the notification condition than in the control condition, thusly demonstrating a mirror effect. Furthermore, an analysis of Corrected Recognition Scores showed that subjects differed significantly in their recognition performance, with participants in the notification condition demonstrating a significantly worse performance at identifying which words were previously presented. This therefore confirmed the hypothesis that split attention caused by notifications during the encoding of information would lead to reduced memory performance for that information. This result was in line with the result found in the Glass and Kang (2018) study that indicated that testing performance was worse for the students who were allowed to use electronic devices during study. Similarly, the Hembrooke and Gay (2003) study found that the usage of electronics during lectures significantly reduced the memory of students for those lectures. Clearly, split attention at time of encoding can be caused by the use of electronic devices in general and is particularly likely to be caused by electronic notifications, and this has a negative effect on the retention of the material to be encoded.
The present study was split into two groups – one that received notifications that they were instructed to pay attention to and record, and the other that received no notifications whatsoever. Future research could further explore the distracting role of electronic notifications through splitting the experimental group into further sub-groups: one that is required to note the content of the notifications, one that is merely instructed to view the notification, and one that merely receives an audio or tactile signal without actively interacting with the device. The current body of evidence suggests that the first experimental sub-group would be the one that would experience the most significant reduction in memory performance, but it remains to be determined as to whether the other sub-groups would also display a significant reduction in memory retention.
Broadbent’s theory of selective attention proposes that information passes through a limited processing channel after the sensory processing stage but prior to being encoded in the short-term memory store (Broadbent, 1958). The theory claims that when this channel is overloaded such as in the case when subjects receive notifications while they are also attempting to attend to a different cognitive task, some information is filtered out by this channel and therefore remains unprocessed, while other information is retained and is therefore analyzed further by higher-level brain processes. The theory is generally practiced in a particular paradigm wherein subjects perform some primary task while they are instructed to also monitor a secondary task for information, a paradigm that is reflected in this study. As has been found in numerous prior studies, the fact that the two tasks are being performed simultaneously leads to a reduction in performance in the primary task. Therefore, electronic devices and particularly the notifications that they tend to frequently display can be seen as effectively replicating this paradigm – making the primary tasks that the users of such devices perform into split-attention tasks. The devices and the notifications that the devices display therefore become a secondary task that these individuals attend to while they work on the primary task that they are performing. It is therefore a rather sobering, if not a particularly surprising conclusion that devices and the notifications that they deliver seem to facilitate a significant detrimental effect on the primary task and memory performance of their users.
REFERENCES
i Broadbent, D. (1958). The Selective Nature of Learning. Perception and Communication, 244-267. doi:10.1016/b978-1-4832-0079-8.50012-8
ii Cheyne, J. A., Solman, G. J., Carriere, J. S., & Smilek, D. (2009). Anatomy of an error: A bidirectional state model of task engagement/disengagement and attention-related errors. Cognition, 111(1), 98-113. doi:10.1016/j.cognition.2008.12.009
iii Glass, A. L., & Kang, M. (2018). Dividing attention in the classroom reduces exam performance. Educational Psychology, 1-14. doi:10.1080/01443410.2018.1489046
ivHembrooke, H., & Gay, G. (2003). The laptop and the lecture: The effects of multitasking in learning environments. Journal of Computing in Higher Education, 15(1), 46-64. doi:10.1007/bf02940852
vMarty-Dugas, J., Ralph, B. C., Oakman, J. M., & Smilek, D. (2018). The relation between smartphone use and everyday inattention. Psychology of Consciousness: Theory, Research, and Practice, 5(1), 46-62. doi:10.1037/cns0000131
viNational Safety Council. (2015, May 18). Cell phones are involved in an estimated 27 percent of all car crashes, says National Safety Council. PR Newswire. Retrieved from https://www.prnewswire.com/news-releases/cell-phones-are-involved-in-an-estimated-27-percent-of-all-car-crashes-says-national-safety-council-300084920.html
viiStatistics Canada. (2017). Life in the fast lane: How are Canadians managing?, 2016. Ottawa, ON: Statistics Canada.
viiStothart, C., Mitchum, A., & Yehnert, C. (2015). The attentional cost of receiving a cell phone notification. Journal of Experimental Psychology: Human Perception and Performance, 41(4), 893-897. doi:10.1037/xhp0000100
ixStrayer, D. L., Drews, F. A., & Crouch, D. J. (2006). A Comparison of the Cell Phone Driver and the Drunk Driver. Human Factors: The Journal of the Human Factors and Ergonomics Society, 48(2), 381-391. doi:10.1518/001872006777724471