A well-educated citizenry is an economic and social necessity. But there is little consensus about what it takes to deliver a quality education. Our latest research offers global findings as well as deep regional analysis—the focus of this article is Asia.
In two previous reports, one on the world’s best-performing school systems (2007) and the other on the most improved ones (2010), we examined what great school systems look like and how they can sustain significant improvements from any starting point. In this report, we switch our focus from systems to student-level performance, by applying advanced analytics and machine learning to the results of the Organisation for Economic Co-operation and Development’s (OECD) Program for International Student Assessment (PISA). Beginning in 2000, and every three years since, the OECD has tested 15-year-olds around the world on math, reading, and science; it also surveys students, principals, teachers, and parents on their social, economic, and attitudinal attributes (Exhibit 1).
Using this rich data set, we have created five regional reports—covering Asia, Europe, Latin America, the Middle East and North Africa, and North America—that consider the drivers of student performance. In Asia, 13 countries and autonomous territories participated in the 2015 PISA. For our analysis, we divided these into three categories based on performance. High-performing Asia is composed of China (specifically the cities of Beijing, Guangdong, Jiangsu, and Shanghai), Hong Kong, Japan, Macao, Singapore, South Korea, Taiwan, and Vietnam. Oceania refers to Australia and New Zealand. Developing Asia is composed of Indonesia, Malaysia, and Thailand. High-performing Asia has high yet flat achievement; Oceania performs generally well, but scores appear to be declining; and Developing Asia is improving, but slowly and from a low base.
Our research is not intended as a road map to system improvement; that was the theme of our 2010 report, which set out the interventions school systems need to undertake to move from poor to fair to good to great to excellent performance. Instead, this report examines four specific factors that we found to be particularly important to student outcomes: mindsets, teaching practices, information technology, and early childhood education.
The report’s findings include the following four highlights.
Student mindsets have double the effect of socioeconomic background on outcomes
It is hardly news that students’ attitudes and beliefs influence their academic performance. The magnitude of this effect, and which mindsets matter most, is still under debate; we focused our research on these areas. While there is likely overlap between socioeconomics and student mindsets, we measured the effect of mindsets that is not explained by socioeconomics alone. By analyzing the PISA data, we found that mindset factors have double the predictive power (31 percent) compared to home environment and demographics (15 percent) on student PISA scores in Asia (Exhibit 2). This relationship also holds true in all other regions, reinforcing the importance of this finding.
Some mindsets are more important than others. For example, we compared motivation calibration (being able to identify what motivation looks like in day-to-day life, including “working on tasks until everything is perfect” and “doing more than expected”) to self-identified motivation (“wanting to be the best” or “wanting to get top grades”). In the 2015 PISA assessment, motivation calibration has four times the impact of self-identified motivation. Across Asia, students who have good motivation calibration score 8 to 14 percent higher on the science test than poorly calibrated ones. By contrast, students with high self-identified motivation score only 6 to 8 percent higher (Exhibit 3).
The impact of motivation calibration varies by geography and type of student. In Developing Asia, the relationship is particularly strong for the three-quarters of students in poorly performing schools, where having a well-calibrated motivation mindset is equivalent to vaulting into a higher socioeconomic status. In these schools, students in the lowest socioeconomic quartile who are well calibrated perform better than those in the highest socioeconomic quartile who are poorly calibrated. In high-performing Asia and Oceania, the relationship is particularly strong for lower socioeconomic status students—double the effect as for students from wealthier households.
Would you like to learn more about our Social Sector Practice?
Having a growth mindset is also strongly linked to student outcomes. Across Asia, students with a strong growth mindset—those who believe they can succeed if they work hard—outperform students with a fixed mindset—those who believe that their capabilities are static. The difference in score between having a growth versus fixed mindset is 12 percent in Developing Asia, 14 percent in high-performing Asia, and 20 percent in Oceania.
To be clear, mindsets alone cannot overcome economic and social barriers, and researchers still debate the extent to which school-system-level interventions can shift student mindsets. Our research does, however, suggest that they matter—a lot, and particularly for those living in the most challenging circumstances. The research on this subject is both nascent and predominantly US-based. Considering its importance, local experimentation should be a priority.
Students who receive a blend of inquiry-based and teacher-directed instruction have the best outcomes
High-performing and fast-improving school systems require high-quality instruction. It’s that simple—and that difficult. We evaluated two types of science instruction to understand how different teaching styles affect student outcomes. The first type is “teacher-directed instruction,” where the teacher explains and demonstrates scientific ideas, discusses questions, and leads classroom discussions. The second is “inquiry-based learning,” where students play a more active role, creating their own questions and engaging in experiments.
Our research found that student outcomes are highest with a combination of teacher-directed instruction in most or almost all classes, with inquiry-based learning in some (Exhibit 4). If all students experienced this blend, average PISA scores would rise by 3.8 percent in high-performing Asia, 3.4 percent in Oceania, and 1.2 percent in Developing Asia. For high-performing Asia and Oceania, this is equivalent to approximately half a year of schooling.
Given the strong support for inquiry-based pedagogy, these results may seem counterintuitive. We offer two hypotheses. First, students cannot progress to inquiry-based methods without a strong foundation of knowledge, gained through teacher-directed instruction. Second, inquiry-based teaching is inherently more challenging to deliver, and teachers who attempt it without sufficient training and support tend to struggle. Better teacher training, high-quality lesson plans, and school-based support can help. It’s also important to note that some kinds of inquiry-based teaching are better than others. For example, explaining how a science concept can be applied to a real-world situation appears to boost outcomes; having students design their own experiments seems to do the opposite.
School-based technology yields the best results when placed in the hands of teachers
Screens are not the problem when it comes to student outcomes—but neither are they the answer. Our research examined the impact of first exposure to information and communications technologies (ICT), the impact of ICT use at home, and also the impact during school. Across Asia, students with their first digital exposure before the age of six score 12 percent higher than those exposed at age 13 or later (controlling for socioeconomic status, school type, and location). Students of higher socioeconomic status are more likely to start using devices at an early age, which has worrying implications for the equity gap.
When 15-year-olds were asked how much time they spend using the Internet at home, interesting differences emerged. In Developing Asia and Oceania, moderate use of the Internet (defined as two to four hours per day) correlated with higher PISA scores. Beyond four hours, the positive effects tended to decline, with negative implications when students spend more than six hours per day on screens outside of school. However, in most high-performing Asian countries, students who spend more than about an hour per day saw declining benefits. Across High-performing Asia, only 65 percent of students report spending more than 30 minutes online a day, versus 95 percent of students in Oceania, suggesting very different cultural norms.
How to improve student educational outcomes: New insights from data analytics
The impact of ICT use during the school day is much more mixed: from −17 to +8 percent, depending on the type of hardware. Most important, we found that deploying ICT to teachers works best. For example, students who report use of data projectors in their classrooms score 8 percent higher than those who do not. The lift for desktop or Internet-connected student computers, by contrast, is only 3 and 5 percent, respectively. These results all hold controlling for socioeconomic status, school type, and location. Some student-based classroom technologies, such as laptops in high-performing Asia, and tablets and e-book readers across all of Asia, actually appear to hurt performance (Exhibit 5).
It is important to note that these results describe the impact of education technology now, not its eventual potential. Nor do these findings consider software or how teachers are using the technology in the classroom. Even so, Asian leaders should not assume the impact of ICT will always be positive or even neutral. Systems should ensure that ICT programs are fully integrated with the curriculum and instruction and are supported by teacher professional development and coaching.
Early childhood education has a positive impact on student scores, but the quality and type of care is important
Many studies have shown that quality early childhood education (ECE) improves social and academic outcomes, although there are some concerns about fade-out in later years. Our findings, like other research, validate the overall positive impact of ECE. Across Asia, students who report some ECE perform 21 PISA points (over half a school year) better on the PISA science test a decade later than students who attended no ECE, controlling for socioeconomic status, school type, and location.
There are meaningful differences across the categories. First, the lift for high-performing Asia is six PISA points, versus 11 for Oceania (there was not enough data for Developing Asia to draw statistically significant conclusions). Second, students in high-performing Asia and Oceania do best at age 15 when they start ECE at age three; for Developing Asia, the best performers start at age four. This may reflect variation in the quality of care available for younger children.
Type of care matters also. The parent data from three high-performing school systems (Hong Kong, Macao, and South Korea) indicates that children who went to more formal, structured pre-primary programs significantly outscored those who went to less formal ones or had no ECE at all.
The data suggests that Asian governments should continue to prioritize providing early childhood education and should carefully monitor the quality of provision.
We are mindful of the limits of these findings. One cannot hope to gain definitive answers from a single source, no matter how broad or well designed. The direction of causality, sample sizes, missing variables, and nonlinear relationships are important issues. There are still many questions that need to be resolved through a thoughtful research agenda and longitudinal experimentation. That said, we believe that these findings provide important insights into how students succeed. Asian educators should incorporate them into their school-improvement programs to deliver the progress that their students deserve.
Download the full report on which this article is based, Drivers of student performance: Asia Insights (PDF–6 MB) and the executive summaries in Chinese (PDF–9 MB) and Japanese (PDF–9 MB).
