Insights from efficacy studies
Independent Evaluation: Stanford Research Institute
See the final report, published by SRI Education, as part of the Next-Generation Courseware Challenge, funded by the Gates Foundation. Other useful research about the program can be found at www.coursewarechallenge.org, including a case study on the CriticalChem offering.
A number of the findings from the report are summarized below. In addition, several universities and colleges using courseware offered by Inspark Science Network have conducted efficacy studies and shared those results with our team.
Institutions using the offerings BioBeyond and HabWorlds won 2 out of the 10 OLC Digital Learning Innovation Awards in the Faculty-Lead Team category in 2017, and then again in 2018. BioBeyond won the 2017 Reimagine Education E-Learning Award.
BioBeyond has proven that it helps students better understand introductory biology, persist through their course, and improve final grades.
1. Higher end-of-course grades
SRI Education conducted an independent evaluation at four higher education institutions: Arizona State University, Miami Dade College, Mohave Community College, National University. Researchers measured the impact of incorporating BioBeyond science courseware (620 students) compared to a traditional course (2,900 students) for in Fall 2016. The study found that use of BioBeyond improved end-of-course grades by up to half a letter grade.
The statistically significant improvements were roughly half a letter grade for the Arizona State University fully-online course (+0.42) and the Miami Dade College blended/hybrid course (+0.46), and nearly a third of a letter grade for the Mohave Community College blended/hybrid course (+0.26). Results at National University weren’t statistically significant, but still showed positive impact on student learning (+0.22).
- Controlled for potential confounding variables, including incoming GPA, demographics, and other administrative variables
- n = 620 BioBeyond students, 2,900 traditional students
- Instruction: Fully-online and Blended/Hybrid
- When: Fall 2016
- 0 to 4 grade scale, with 4.0 = A
- Statistically significant positive impact on grades for 3 of the 4 institutions (* = p < 0.05, *** = p < 0.001)
2. Improved success across student demographics
The SRI study also controlled for student demographics. In some very exciting results, they found that gender and underrepresented status had no effect on the ability to predict student performance. One of the goals in creating this course was to enable access to high-quality learning experiences for students in disadvantaged learning groups, so both the Inspark Science Network and Smart Sparrow teams were doubly thrilled to see these results.
3. Same instructor, same exam, better results
Another study, not conducted by SRI Education, compared instruction using BioBeyond to traditional methods across 7 semesters of introductory biology at Miami Dade College. The course was taught by the same instructor, culminating in the same exam — and showed better student results when using BioBeyond. The study suggested that use of the BioBeyond course led to more As and Bs compared to the traditional Biology 100 course — 33% vs. 19% of the final grades, respectively (p < .001).
- Common exam and instructor
- n = 239
- Sections: 2 BioBeyond, 5 traditional
- X2(1) = 20.994
- p < .001
4. Further reading about BioBeyond efficacy
- How BioBeyond engages students with interactive “video game-like” learning experiences (Case study)
- How BioBeyond evolved a professor’s role from lecturer to educator (Case study)
- New Research Finds Digital Courseware Increases Student Success in Critical Introductory Science College Courses (Article)
- Adaptive Bio Course for Non-Majors Finds Success with Simulation, Virtual Field Trips (Article)
- How Exploratory and Adaptive Learning is Changing Biology Classes (Article)
1. Increased grades & decreased withdraw rates
In a controlled study at Glendale Community College, study and analysis of the use of HabWorlds in an introductory astronomy course showed an upward shift in student grades across different section. Students earned significantly more As and Bs, fewer Ds and Fs; there were also fewer course withdrawals.
- Common exam and instructor across all; exam = 60% of course grade
- n = 306
- Sections: 5 HabWorlds, 3 traditional
- X2(1) = 6.751
- p = .009
2. Further reading about HabWorlds efficacy
- Habitable Worlds: Delivering on the Promises of Online Education (Journal publication)
- How one instructor uses HabWorlds simulations to enable active learning in a blended class (Case study)
- How HabWorlds leads students to deeper understanding of science (Case study)
- Digital Learning’s Pioneers Are Cautiously Optimistic (Article)