This website uses cookies and similar technologies to understand visitors' experiences. By continuing to use this website, you accept our use of cookies and similar technologies,Terms of Use, and Privacy Policy.

Oct 01 2013 - 08:00 PM
Evaluating the Effectiveness of Digital Learning Tools

Title: Use of Web 2.0 technologies in K-12 and higher education: The search for evidence-based practice (2012)

Authors: Khe Foon Hew & Wing Sum Cheung

Source: Educational Research Review

Research Question: Is there enough evidence to prove that the use of Web 2.0 technologies effectively enhances student learning?

Study Design: As campuses and classrooms are flooded with tablets and new devices, teachers and parents are pausing to question whether there is enough evidence to support the wide-scale investment in new technology. A paper by the National Institute of Education in Singapore assesses the current state of research on the effectiveness of digital technologies, primarily the use of podcasts, blogs, wikis, Twitter, and 3D immersive virtual worlds in the classroom.

The researchers analyzed the 27 most relevant empirical studies and evaluated them according to discipline, learning goals, cognitive processes, pedagogical approaches, and specific learning activities. Most of the studies examined technology tools across different disciplines and usually at the post-secondary level. The authors qualify that this was not an exhaustive list of studies, most studies were limited to the span of one semester or less, and studies did not report effect sizes, so the results were interpreted cautiously.

Findings: Unfortunately for educators and administrators who want to validate tech integration with supportive statistics, actual research-based evidence of the effectiveness of digital technologies is weak. Although there is not much data to support the contention that digital tools are always better, they generally had a positive impact on learning, and never had a negative impact. Among other tools discussed, the paper reviewed the effectiveness of podcasts and blogs.

Of the nine studies that examined podcasts, seven reported increases in student performance. The key to successful incorporation of podcasts was not the audio itself, but how teachers incorporated them into lessons. Podcasts that provided additional information and just-in-time facts (i.e., a narrative garden tour for landscape architecture students) led to higher student grades. A standout success story was a psychology class at American University where supplemental podcasts enabled students to score significantly higher on exams than students who only had access to traditional lectures. Cases where podcasts did not result in higher scores were attributed to small, invalid sample sizes and technical difficulty accessing the podcasts.

Six studies evaluated blogs in classes ranging from English language learning to science to physical education. In a Teaching English as a Foreign Language class in Turkey, students were divided into two groups taught by the same instructor: one that practiced process writing after reviewing model paragraphs and constructive feedback and one that followed a similar path but read tutor blogs that provided more model paragraphs and links to teaching websites. The instructor found that students in the latter group improved their writing compared to the group that did not have blog access. The majority of other studies also showed improvements in student writing, although the results were not statistically significant.

Moving Forward: Small but insignificant positive gains were repeatedly found for studies on wikis, Twitter, and 3D Virtual worlds. Unfortunately some studies were underdeveloped and had confounding variables, weak experimental designs, and other limitations that made it difficult to generalize broadly about digital learning tools. This highlights the need for institutions to design studies that deliberately discern where, when, and how these tools are effective.

In any case, improvements in student learning shouldn’t be viewed as a direct result of merely having new technology, but as a result of how teachers integrate it in the classroom. Successful models incorporated questioning, peer reviews, and self-reflection. Some technologies pair neatly with different perspectives on learning. For example, podcasts may facilitate activities based on memorization and retention, whereas wikis emphasize learning that requires social negotiation. More carefully designed studies over longer periods of time and those that consider the convergence of digital and mobile tools are needed before researchers can definitively say how technology influences learning.

Image: Johan Larsson via Flicker