Popularization of efficient science-based study techniques through actions in schools , media and internet

Research shows that students and teachers are often unaware of the study techniques that best promote long-lasting learning. Here we describe the popularization of two learning techniques: retrieval practice and distributed practice, which were disseminated through lectures, articles in the media, social networks, and conferences. We also showed that these techniques could be used in everyday life through cell phones (mobile-learning) with a free app. In 2015-2016, institutions of 6 cities in Brazil were visited by the researchers and news of the project was released on social networks. More than 1300 students attended the lectures in person. We tracked 30 releases in print journals and websites news that published articles about this project. We conclude that study techniques can be disseminated to students and teachers on a large scale by the scientific community with the help from news media. This is especially relevant in developing countries where educational practices are still underfunded.


Introduction
Misinformation about how people learn (e.g., neuromyths, or myths about the brain: PASQUINELLI, 2012) is rife and in dire need of correction (EKUNI; POMPEIA, 2016).With this in mind, scientists should increase awareness about scientifically based facts that can actually help people to learn.One point that deserves attention is that students often choose inefficient ways of studying, such as highlighting and/or rereading textbooks and notes (BJORK; DUNLOSKY; KORNELL, 2013;KARPICKE;BUTLER;ROEDIGER, 2009).Evidencebased learning points to the utility of two learning techniques that are seldom used by teachers and students and which are easy to implement and efficient in producing long-lasting learning: retrieval practice and spaced or distributed practice (DUNLOSKY et al., 2013;ROEDIGER, 2013).
Retrieval practice involves trying to recall information that was previously encountered (ROEDIGER; KARPICKE, 2006).This can be achieved by doing tests, exercises, thinking about real-life examples of the learned content or associating it with other known facts (PUTNAM; SUNGKHASETTEE; ROEDIGER, 2016).
These activities boost learning considerably more than simply rereading information (ROEDIGER; KARPICKE, 2006;ROEDIGER;PUTNAM;SMITH, 2011;ROWLAND, 2014), the preferred study method by most students (KARPICKE; BUTLER; ROEDIGER, 2009).However, in general, students are encouraged to recall information only in exams, which aim to assess how much students have learned rather than being a means of promoting lasting learning (ROEDIGER; NESTOJKO, 2015).Understandably, because students associate testing with being evaluated, they dislike test taking (ROEDIGER; KARPICKE, 2006).This is regrettable, as retrieval practice is probably one of the two best technique to "make information stick" (DUNLOSKY et al., 2013;ROEDIGER, 2013).
Another efficient learning technique is known as distributing practice or the spacing effect, that is, distributing study through time, or spacing (CEPEDA et al., 2008).Even when the total study time on different occasions is equivalent to studying all the content at once (massing), spacing leads to better learning (SON, 2004).Despite this evidence, students seem to prefer massing to study in a distributed fashion (SON, 2004).Any form of spacing, such as at fixed (e.g., reviewing content every three days), or with expanded intervals (e.g., reviewing content after one day, again after five days, and again after nine days) has been shown to be advantageous for long-term retention (KARPICKE; ROEDIGER, 2010).
Despite the clear advantages of retrieval practice and spacing in improving learning, these ways of studying are unknown by the majority of the population.If we are to improve educational practices, both students and teachers should be made aware of these techniques, especially because they are easy to implement, need not involve extra study or teaching time, nor financial costs (ROEDIGER; PYC, 2012), just a slight change in the way that teaching and/or studying takes place.This awareness is especially true in developing countries, given the political issues of the lack of adequate investment in education.Brazil is one example as it does not reach the goals for quality education (PISA, 2015).
Thus, the present extension project aimed to divulge to students and teachers the findings of learning sciences regarding effective study techniques, as these are important to popularize scientific findings for the lay public ( EKUNI et al., 2014;SUGIMOTO et al., 2013).

Method
In order to disseminate the scientific evidence about the learning techniques explained in the Introduction section, several actions were proposed:

a) Lectures and Conferences
A 30-minute lecture was offered to schools containing the following topics: i) a discussion on how the students study and what techniques they use; ii) the study techniques reviewed by Dunlosky et al. (2013), which detail which ones have low, moderate and high utility, and explanations regarding those which have higher utility (retrieval practice and distribute practice); iii) explanations on how to use the "pomodoro technique", useful for avoiding inertia, procrastination and difficulties in concentration (based on the course -Learning how to learn 1 ); iv) presentation of a free application software that involved both techniques, detailed below; v) answering students' questions.
The researches were also invited to give lectures at: i) scientific events (Academic Weeks for undergraduate students; poster presentation in a Conference for professionals working with adolescents); and ii) educational events (Education Meeting for High School students, Opening lectures for pre-college preparation courses).

b) Posts and videos on social networks
The researchers prepared posts for dissemination of the learning techniques on Facebook 2 , YouTube 3 , a website 4 , translated Wikipedia entries (see HARSHAW; MARCINOWSKI; CAMPBELL, 2014) on the topic 5 and adapted a Retrieval Practice Guide into Portuguese to allow this information to reach people who do not read English.The Guide, "How to use retrieval practice to improve learning" (AGARWAL et al., 2013) explains what retrieval practice is, how teachers can implement this technique in the classroom and provides tips and strategies for those who want to implement it.To ensure that the Guide was adequate for use in Brazil: i) the text was first translated into Portuguese with the permission of the authors and necessary adjustments were made considering the local culture; ii) the translation/adaptation was analyzed by a group of 6 Brazilian specialists in cognitive sciences, who suggested modifications which were all implemented; iii) the translation was back-translated into English by a native speaker in both languages; iv) the back-translation was submitted to the North American first author of the Guide for her approval, followed by minor adjustments.
For Facebook postings the researchers: i) wrote posts based on the literature, with quotations from scientific sources, all of which were reviewed by specialists in each field; ii) prepared an image to attract the attention of the public (the images were made in PowerPoint, Photoshop or using the free website Canva 6 .YouTube videos were created followed these steps: i) creation of scripts in accessible language, all of which were reviewed by at least two independent specialists; ii) recording of the narration (if applicable); iii) video recording or video editing via the Powtoon software (www.powtoon.com);iv) appreciation of videos by third parties to check if the language was accessible, the order of contents was adequate, among other aspects; and v) publication of material on a YouTube channel 7 .

c) Dissemination of the m-learning APProvado application in various media
We developed a free pilot cellphone application software (APProvado) based on mobile-learning [m-learning: learn anytime, anywhere (HASHEMI et al., 2011)] that involves retrieval practice and distribute practice, aimed at pre-college students.Such procedure was done because we believed that providing a free application would serve to attract pre-college students, motivated to learn of effective learning techniques to prepare for University entrance examinations.Releases about the APProvado application and the learning techniques on which it was based were prepared and divulged by Press Offices of the executing Universities (Universidade Estadual do Norte do Paraná and Universidade Federal de São Paulo), as well as by a third party hired by the researchers.Additionally, the researchers contacted the press and websites communicating the importance of the project and suggesting the inclusion of the topics on their agenda.

d) Development of a structured platform based on mlearning and concepts of retrieval practice and distributed practice
The developed web platform was named APProvado (www.approvado.comtrademark registered at INPI -National Institute of Intellectual Property 8 ).The title makes reference to "app" for an Android mobile application as a support for the platform 9 and "aprovado", which means "approved" in Portuguese; this name was chosen because our target students were those preparing for University entrance examinations.
The structure of the APProvado system involved the website platform and the mobile application.The platform was developed using HTML5, PHP programming language and MySQL as a database.To host the system and database we contracted a private cloud server from the company umbler.comfrom $20/monthly.The mobile application was developed based on Android smartphones.From this application, users could access the daily questions from their smartphones and set an alarm to remind them to answer the questions.
Users who did not have a smartphone, or owned phones with different operating systems [e.g., IOS (Apple); Windows Phone (Microsoft)], could access and use the platform through the web address using the browser on their computers/smartphones.However, only the Android version of the APProvado has a system to remind participants to answer the daily tests (users of other operating systems received the reminder via e-mail).
The APProvado system has functionalities that allow the promotion of learning using m-learning, retrieval practice and distributing practice.In this pilot version of the application, the material to be learned covered 100 different contents in biology.Biology was selected because the wording of the questions was short and allowed short answers, easily read and typed into smartphones, respectively.Shortly, users were sent questions drawn from prior University entrance examinations from one of the most prestigious University in Brazil (www.fuvest.br).Five questions per day during weekdays were used over a 12 weeks period (distributed practice).Questions were presented once, twice, three and four times at different time intervals (one, three and seven days) (effectiveness of the testing schedule will be published elsewhere).After reading each question, users had to type in a short answer (retrieval practice), which is a format that leads to better retention than the multiplechoice format (KANG; MCDERMOTT; ROEDIGER, 2007).Because feedback improves retrieval practice effects (BUTLER; ROEDIGER, 2008) and it would not be feasible for each short-answer question to be checked for each user throughout the 12 week experiment, as soon as users typed in their answers, five alternative answers were shown (multiple-choice answer format).Participants then chose one of the alternatives and received immediate feedback.Hence, participants had two consecutive opportunities to answer tests: first trying to retrieve answers from memory with no cues and subsequently through a five multiple-choice test.In this platform, links to our Facebook fan page and YouTube channel we made available, as well as tutorial videos showing how the mobile application worked and what scientific concepts the application was based on.

a) Lectures and conferences
The qualitative analyses of the results of the lectures are based on the speakers' reports.Of the thirteen schools contacted by the researchers, eight accepted to receive the lecture (5 were public/state school and 3 were private institutions).Furthermore, the researchers were invited to talk about this topic at two preparatory courses for University entrance examinations, one lecture in a scientific event (Nursing Week) and one in an Education Meeting focused on high school students, totaling 7 different cities in the South and Southeast of Brazil.More than 1,300 students (number reported by the coordinators of the schools or events) were reached through this action.
Descriptively speaking, some of the most frequently asked questions were: "Can I study using retrieval practice alone with the tests available in the book or should I elaborate my own questions?"; "Why do I function better by studying more at night than by day?"; " I study highlighting text and notes; should I stop?".The answers given by the researches are mentioned in the Discussion session.

b) Posts and videos on social networks about retrieval practice and distributed practice
We only made one paid posting (U$ 7.00 to advertise the APProvado mobile application on Facebook).The others were organic publications, which did not involve payment to boost visualizations.In relation to the time of day with higher accesses, according to the weekly records, the peak times were at noon and 10 P.M., and were only slightly lower between these peaks (Figure 1).On the YouTube channel, the 5 videos that disseminated learning techniques had between 33 and 530 views in the 5 months duration of divulgation of the project.This number does not consider views via Facebook, which can only be counted if the video is uploaded onto that network, which we were unaware of when the project began.
For a higher dissemination of the targeted learning techniques for the public that only reads Portuguese, the Wikipedia texts in this language were made available 4 Since their creation (up to June, 2017) they had 520 and 216 page views, respectively.
Regarding the translation of the Retrieval Practice Guide, the Portuguese version was posted online and is available for free 10 .This process resulted in an international collaboration to disseminate scientific findings in this area to the community, making both English and Portuguese videos 11.In the period of 5 months after the launch of the APProvado, we had 1682 downloads of the application.Among these, 1237 students answered an initial demographic questionnaire that enquired about study habits (results will be published elsewhere).

Discussion
Dissemination of scientific findings to the general population is important to popularize science (see SUGIMOTO et al., 2013).In the case of this project, the goal was to divulge efficient learning techniques (DUNLOSKY et al., 2013).Most contacted teaching institutions and students showed interest in housing and attending lectures on the subject, respectively.At the end of the lectures, questions raised by the students were answered based on the scientific literature.The most common questions mirrored facts discussed by Bjork et al. (2013) based on their experience with students in other countries.For instance, many attendees asked if the origin of tests or exercises to be used in retrieval practices mattered.It was emphasized that this made no difference to improve learning (WEINSTEIN; MCDERMOTT; ROEDIGER, 2010) and that the critical thing to do is to try to retrieve information from memory in variable ways with the minimal number of clues as possible.
Another frequently asked question was about the best time of the day to study.It was explained that people vary regarding the time of optimum cognitive performance and that each one should study when they feel more alert (LEVANDOVSKI; SASSO; HIDALGO, 2013).The researchers then indicated one of the videos on our YouTube channel 14 with explanations on chronotype.Another common question referred to the use of low-utility techniques, such as highlighting texts or re-reading notes.It was explained that these methods could be effective for experienced students who know what material is relevant (DUNLOSKY et al., 2013) but that, crucially, our point was that all students must strive to insert retrieval practice into their study schedules, preferably spacing out the study of each contents.
The lectures were focused on teaching students to study using retrieval practices, which are avoided by students because they require more effort than merely rereading content (BJORK; BJORK, 2011).Likewise, distributed and spaced study is avoided because it involves organizing study times, which is harder than leaving all studying for the last minute, just before exams.It is of note that cramming before exams lead to the same performance as distributed practice, so students fail to see its benefit, which is better retention in the long term, as occurs with retrieval practice (CEPEDA et al., 2008).Therefore, we believe that disseminating the advantages of these techniques to students can serve as an incentive for a change in their study habits.Also, teachers must be targeted (WEINSTEIN et al., 2010) so that they can pass on this knowledge to their students and aid them in providing tests and questions so that they can practice retrieval.To this end, we translated the retrieval practice Guide (AGARWAL et al., 2013) which is now freely available online.
Regarding making tools available to use the techniques of interest (in our case, the APProvado application) we found that it is crucial to make the existence of such instruments known using the media, as it has a broader reach and can be used to disseminate science ( EKUNI; POMPEIA, 2016;HIJMANS;PLEIJTER;WESTER, 2003).As we showed in Figure 2, when we were able to post news about the application on well-accessed media, the number of downloads increased considerably.
Previous researches have shown that Facebook can be used as a tool to support classroom teaching, mainly because most of the students access this social media daily (VARGAS et al., 2014).Based on how Facebook posts operate, if the people who follow a Facebook page are also following many others pages, their Facebook timeline becomes full.Hence, if the posts are more frequent, they will increase the chances to be visualized.Our data showed that the public targeted in the present work accesses their Facebook more between noon and 10 PM, which is essential in order to schedule publication times since in the Facebook social media new information is ordered according to how recently it was posted.Consequently, the chance of postings for this public being viewed is higher if postings occur soon before noon or during the afternoon.Furthermore, researchers who aim to reach the public with videos should post them on YouTube, which allows the quantification of accesses.Such videos should also be posted on Facebook (not the Youtube link).Otherwise, it is not possible to quantify video accesses in this social network.
The number of people who were interested in the application, indicated by its download rate (n=1,682), was far superior to the low number of views of our Facebook page and YouTube channel, even though the dissemination of all these means of learning to study more efficiently were made in the vast majority of the media publications on the project.This shows that it is easier to get students' attention about how to study more efficiently if some product or software is made available to them for free, even when students are going through a phase in which to study effectively is crucial for their admittance to tertiary education.Providing information about how to study was much less appealing to the targeted public, even by making it clear that advice comes from experts, as shown by the low number of views and subscribers on YouTube and Facebook.This is not surprising considering that international channels such as Lasting Learning 15 , which involves the most renowned scientists in the field and has a beautiful presentation and impeccable content 16 , had less than 400 subscribers after being online for more than two years 17 .
In contrast, YouTube posts made by students showing how very popular they study are.For example, at the time of writing this manuscript, by searching YouTube with the phrase "how to study for exams" ("Como estudar para provas" in Portuguese), the three most viewed videos had more than 1 million views each.The most viewed one was posted by a student 18 who describes her own experience: it has over 1.8 million views 19 .The second one, posted by a Biology Teacher 20 , has over 1.3 million views and describes how he believes that the students can become better learners.The third video describes how to solve multiple-choice questions 21  and has over 1.3 million views.None of these videos made references to scientific evidence on how best to study.They included tips based only on the protagonists' own experience, among which, admittedly, there were some good tips such as studying a little every day.
Contrary to the public of TED talks, which seeks scientific information and prefer to watch videos that are presented by experts (SUGIMOTO et al., 2013), our result shows that, when it comes to information on how to study, the adolescent lay public prefers to be informed by peers and is less willing to access what scientists have to say about how to study.People usually believe others like them are more trustworthy (FISKE; DUPREE, 2014).Also, adolescents tend to process the facial expression of peers better than those of caregivers, unlike younger children (PICCI; SCHERF, 2016).Furthermore, peer context influences students' educational outcomes (CARBONARO; WORKMAN, 2016).Indeed, identification with specific speakers can exert more influence and even shape what people recall (COMAN; HIRST, 2015).Therefore, the scientist must seek other means to reach their target audience.It is possible that having students present this type of information, explained to them by the scientists, may be a good means to do so.In this scenario, scientists must rethink how to disseminate scientific findings regarding efficient ways of studying in order to effectively popularize scientific findings that may help people boost their memory more effectively.Additionally, work must be done in developing teachers' and students' scientific literacy (BRITT; RICHTER; ROUET, 2014) and their ability to judge the credibility of the source of knowledge, especially on web-based information (BRATEN;BRAASCH, 2017).This will allow them to make educated choices about how to improve their study habits.
We conclude that the dissemination of scientific findings about effective learning methods can be carried out in various ways.Lectures in schools were well accepted.Most contacted establishments were open to receiving researchers to talk to students, and many youngsters attended the talks.However, this only reaches a small number of individuals and involves many hours of researchers.Just posting this type of information online, even when using animated videos, had almost no impact, even when highly popular news media covered the existence of the sites and links that disseminated our targeted information.Differently, posts in highly visited sites about the availability of a free application to improve learning attracted much more interest.This may be a way to get students to learn more about how best to study.Notably, traffic on the site about the free application occurred more often at specific times (between noon and 10 P.M.).To increase visibility, this period of the day is, therefore, the best to post science-based information regarding learning.In this way, this type of information can better compete with the great amount of other content youngsters have to access every day.We hope our results will incentivize more scientists to disseminate information about effective learning strategies to laypeople so as to help them better cope with academic and work-related issues that involve learning in everyday life.

Figure 1 .
Figure 1.Example of a Print screen of Facebooks analyses with the time of higher access of the Facebook's page on the week after the initiation of the posts.As the page is in Portuguese, the analyses from Facebook is also in Portuguese.

Figure 2
Figure2shows the number of downloads of the APProvado application and the weeks in which there were important publication on at least one media news.In the others weeks, information on the app was disseminated only by the researchers [publication on their pages (Facebook, YouTube and website) and lectures].

Figure 2 .
Figure 2. Number of downloads during 5 months after the launch of the APProvado application 11 .Arrows indicate weeks in which a higher number of downloads were observed after publications in at least one popular media: Week 4 -Jornal Folha de Londrina (newspaper), Website of Universidade Estadual do Norte do Paraná and Universidade Federal de São Paulo; Week 9 -Post on Ministério da Educação (MEC)'s Facebook; Week 10 -First post on Guia do Estudante (Abril Press); Week 11 -Second post on Guia do Estudante (Abril Press) and Television Program on TV Cultura; Week 12 -Third post on Guia do Estudante (Abril Press); Week 13 -Fourth post on Guia do Estudante (Abril Press); Week 16post on Revista Galileu website and Facebook; Week 18 -Post on Universidade Estadual do Norte do Paraná Facebook page.Week 1 June 20 th , 2016; Week 25 -December 19 th , 2016.