dissertation on blended learning

CURRICULUM, INSTRUCTION, AND PEDAGOGY article

Students' perceptions of a blended learning environment to promote critical thinking.

• School of Foreign Languages, Northeast Normal University, Changchun, China

Critical thinking is considered as one of the indispensable skills that must be possessed by the citizens of modern society, and its cultivation with blended learning has drawn much attention from researchers and practitioners. This study proposed the construction of a blended learning environment, where the pedagogical, social, and technical design was directed to fostering critical thinking. The purpose of the study was to find out students' perceptions of the learning environment concerning its design and its influence on their critical thinking. Adopting the mixed method, the study used questionnaire and interview as the instruments for data collection. The analysis of the data revealed that the students generally held positive perceptions of the environment, and they believed that the blended learning environment could help promote their critical thinking in different aspects.

Introduction

The development of critical thinking has drawn attention of the education ministries and institutions of different levels in countries all over the world. In the last two decades, researchers and practitioners have been exploring the ways to integrate critical thinking cultivation into the instruction of different disciplines, proposing strategies and interventions to promote critical thinking, among which blended learning has been widely recognized (e.g., Van Gelder and Bulker, 2000 ; Gilbert and Dabbagh, 2005 ; Yukawa, 2006 ). Blended learning is proposed as focusing on optimizing achievement of learning objectives by applying the “right” personal learning technologies to the “right” person at the “right” time and “right” place ( Singh, 2003 ). A blended learning environment, integrating the advantages of the e-learning method and traditional method, is believed to be more effective than a face-to-face or online learning environment alone ( Kim and Bonk, 2006 ; Watson, 2008 ; Yen and Lee, 2011 ). Studies have been conducted to construct blended learning environments to improve students' critical thinking. Most of them, however, adopted standardized tests or coding schemes to examine the effectiveness of the learning environments on students' critical thinking ( Chou et al., 2018 ), paying less attention to students' perceptions and attitudes. Therefore, the purpose of the current study is to address this gap.

Critical Thinking

There are a vast number of definitions of critical thinking in the literature (e.g., Paul, 1992 ; Ennis, 1996 ; Fisher and Scriven, 1997 ). Despite the emphasis on different aspects, the core of critical thinking entails taking charge of one's thinking to improve it. Paul and Elder's definition and model of critical thinking were adopted in the study. According to Elder and Paul (1994) , critical thinking refers to “the ability of individuals to take charge of their own thinking and develop appropriate criteria and standards for analyzing their own thinking” (p. 34). They proposed that critical thinking is composed of three dimensions: elements of thinking, intellectual standards, and intellectual traits. People demonstrate critical thinking when they use intellectual standards (clarity, precision, accuracy, importance, relevance, sufficiency, logic, fairness, breadth, depth) to measure elements of thinking (purposes, assumptions, questions, points of view, information, implications, concepts, inferences) ( Paul and Elder, 1999 ).

Critical Thinking Cultivation With Information Communication Technology Tools

Studies applying ICT tools to cultivate critical thinking have been increasingly emerging in the literature. The systematic review conducted by Chou et al. (2018) analyzed and reported the trends and features of critical thinking studies with ICT tools. According to the findings of the review, the most often used tools include online discussion (e.g., Cheong and Cheung, 2008 ), coding or game design or Wikibooks creation (e.g., Yang and Chang, 2013 ), and concept or argument maps (e.g., Rosen and Tager, 2014 ). As for the method involved, the studies adopted both quantitative and qualitative research methods (e.g., Shamir et al., 2008 ; Yang, 2008 ; Yang and Chou, 2008 ; Butchart et al., 2009 ; de Leng et al., 2009 ; Yeh, 2009 ). Data from various measurements revealed overall positive results of using ICT tools in critical thinking cultivation (e.g., Yang, 2008 ; Allaire, 2015 ; Shin et al., 2015 ; Huang et al., 2017 ). The findings of the systematic review showed that the critical thinking-embedded activities using ICT tools were more effective than face-to-face activities in developing students' critical thinking ( Guiller et al., 2008 ; Adam and Manson, 2014 ; Eftekhari et al., 2016 ). However, students' prescriptions of the learning design or critical thinking development have not been fully addressed in the literature.

Blended Learning Environment

The concept of blended learning has been defined by several researchers and scholars. For instance, Singh and Reed (2001) defined blended learning as a learning program where more than one delivery mode is being used to optimize the learning outcome and cost of program delivery. According to Thorne (2003) , blended learning is a way of “meeting the challenges of tailoring learning and development to the needs of individuals by integrating the innovative and technological advances offered by online learning with the interaction and participation offered in the best of traditional learning” (p. 2). The above definitions indicate that blended learning can combine the advantages of both traditional face-to-face learning and e-learning and avoid the drawbacks of the two learning modes. The effectiveness of blended learning has been demonstrated by many studies, for example, the findings of a meta-analysis have shown that blended learning brings more positive impact on students learning than online and face-to-face learning ( BatdÄ, 2014 ). Despite the merits of blended learning itself, the effectiveness is determined by the proper design. How to achieve the equilibrium between e-learning and face-to-face modes is crucial to the success of the blended learning environment ( Osguthorpe and Graham, 2003 ).

This study applied the PST model developed by Wang (2008) as the framework for the environment design. As Kirschner et al. (2004) pointed out, an educational system is a unique combination of pedagogical, social, and technological components. PST model thus consists of three key components: pedagogy, social interaction, and technology. According to Wang (2008) , the pedagogical design involves the selection of appropriate content, activities, and the way to use the resources; the social design refers to the construction of a safe and comfortable environment where learners can share and communicate; the technical design provides learners with a technical space of availability, easy access and attractiveness. In any learning environment, the three components play different roles. The technical design offers a basic condition for pedagogical and social design, while the pedagogical and social design is considered as the most important factor that influences the effectiveness of learning ( Wang, 2008 ).

Perceptions of Blended Learning Environment

It has been acknowledged that students' perceptions and satisfaction are important for determining the quality of blended learning environment ( Naaj et al., 2012 ). Studies have been conducted to examine students' views regarding a blended learning environment and factors influencing it. For example, Bendania (2011) study found that students hold positive attitudes toward the blended learning environment and the influencing factors mainly include experience, confidence, enjoyment, usefulness, intention to use, motivation, and whether students had ICT skills. The positive view was also reported in the study done by Akkoyunlu and Yilmaz (2006) , and it was found to be closely related to students' participation in the online discussion forum. Findings from other studies (e.g., Dziuban et al., 2006 ; Owston et al., 2006 ) also revealed students' positive attitudes toward the blended learning environment, and the satisfaction could be attributed to features like flexibility, convenience, reduced travel time, and face-to-face interaction. Some studies, however, reported some negative perceptions of the blended learning environment. For example, the results of the study of Smyth et al. (2012) showed that the delayed feedback from the teacher and poor connectivity of the internet were perceived as major drawbacks of the environment. In another study conducted by Stracke (2007) , lack of reciprocity between traditional and online modes, no use of printed books for reading and writing, and use of the computer as a medium of instruction was considered as major reasons for students withdraw from the blended course. These findings indicate that students' negative attitudes toward the blended learning environment mainly come from the inadequate design ( Sagarra and Zapata, 2008 ).

The review of the above studies indicates that applying ICT tools to cultivate critical thinking has gained much popularity and produced positive results. Few studies, however, focus on students' perceptions of a learning environment designed to promote critical thinking despite the fact that many studies have been conducted to explore students' perceptions of a blended learning environment in general. Therefore, the purpose of the current research is to investigate students' perceptions of a blended learning environment with the orientation of critical thinking development.

Research Design

Research questions.

By adopting the mixed method, this study aims to answer the following two questions:

1. What are students' perceptions of the blended learning environment to promote critical thinking?

2. How do students perceive the impact of the blended learning environment on the development of their critical thinking?

Context and Participants

The study was carried out in the course of Practical English Writing which is a branch of the comprehensive English course for first-year non-English majors at a Normal University in mainland China. The 6-week course adopted a mixed learning mode of classroom face-to-face and online learning. The face-to-face class ran once a week and each class was 90 min. The e-learning tasks were assigned either before or after the class. Six independent learning centers with networked computers were available for students to use and the whole campus was covered with Wi-Fi signal.

The participants of the study involved a total of 90 non-English major students (33 males and 57 females) aging from 18 to 20 in 2020. The students were allocated into classes of Level A after the placement test of English proficiency, which means their English was about higher intermediate level. Adopting the International Critical Thinking Reading and Writing Test ( Paul and Elder, 2006 ), which was developed from Paul and Elders' thinking model, the study assessed students' critical thinking level at the beginning of the course and found that the students' overall critical thinking level was at the lower medium level. But their information literacy level was sufficient to cope with the online platform and the software in the blended learning environment. Before the implementation of the course, the instructor informed the students about the study, and the consent forms were signed by the students.

Environment Design

For the learning environment to achieve the purpose of developing learners' critical thinking, its structural components should be designed to provide favorable conditions for critical thinking cultivation. A systematic review conducted by Lu (2018) has identified a series of favoring conditions that could promote the students' critical thinking, which include (a) critical thinking as one of the teaching objectives, (b) tasks involving the operation of ideas, (c) authentic context, (d) rich and diversified resources, (e) interaction and collaboration, (f) scaffolding and guidance, (g) communicative tools. These conditions were mapped to the design of the components of the PST learning environment model and the designing strategies were generalized from the instruction practice to guide the detailed design of the environmental components.

Pedagogical Design

In terms of the pedagogical design, the thinking skills that can be cultivated were first decided according to the particular learning content. Aiming at promoting the thinking skills, the learning tasks which mostly introduced problems in the “real” context and involve the operation of ideas were designed. Furthermore, rich and diversified resources were provided to the students. The specific strategies of pedagogical design are listed in Table 1 .

Table 1 . Strategies for pedagogical design.

When designing the learning objectives of the activities, the basic concepts and frameworks of critical thinking were introduced to the students, making them aware of its meaning and significace. Furthermore, students were informed of the thinking skills targeted and their importance. When students associated the thinking skills with the tasks, they would try to use the skills to accomplish them.

In order to enable tasks to involve more operations of ideas, writing, discussion, and evaluation activities were given the priority to provide more opportunities for students to communicate with each other and reflect upon their ideas. Besides, the topics of these activities were chosen to induce more collision of ideas. For example, in learning to write complaint letters, students were assigned the roles of customers who made the complaints and the managers who responded to the complaints. In such an activity, students could realize the existence of different perspectives and think more adequately and deeply.

The creation of a relatively real context drew on the following two strategies: One is to provide sufficient details. In the case of the job application writing, details such as the information about the potential employer were provided to the students so that they could consider themselves as “real” potential employees. The other strategy is to create interesting situations. The contexts described were usually attractive to the students, which helped arouse students' interest in completing the tasks.

With the purpose of collecting sufficient and diversified resources, both traditional and online media were included. Since the materials in the textbook are rather limited, the relevant online resources would make complementation for students to have sufficient resources to deal with. To meet the multi-angle nature of resources, the information collected came from different positions and perspectives. For instance, the students were introduced to the websites both for job hunting and recruitment so that they could read information from the perspectives of both employers and potential employees. To help students conduct resource searches by themselves, online resources such as the Online Writing Lab of Purdue University were presented to them to conduct searches. The search was usually directed by a clear question or a problem, and students needed to accurately identify the target source. Some search engines were also introduced to the students, enabling them to compare and select the relevant resources. Students needed to first define what their search objectives were, then assess the search and query results one by one, and finally synthesize the resources to make a reasonable decision.

Social Design

With the purpose of cultivating students' critical thinking in the environment, interactions and collaborations of different types were stressed in the design (see Table 2 ). Furthermore, the scaffold and guidance from the teacher and the peer were designed to provide support to the students.

Table 2 . Strategies for social design.

In designing interaction and collaboration-rich community, the strategies were applied to target both student-student and student-teacher communities. In terms of student-student community, students were grouped according to their levels and the requirements of the activities. Specifically, in a demanding task, students of different academic levels were grouped to ensure the implementation. In a relatively free discussion, students were grouped according to their own will so that they could feel more comfortable sharing their ideas. Also, various types of interactions such as information exchange, discussion, debate were designed. With the change of partners, roles, and tasks, different critical thinking skills were trained. As for the student-teacher community, the student-teacher communication was facilitated through various forms of teacher-student interaction, such as teachers' feedback, office hour, and communications on Tencent QQ, which were necessary to keep students on the right track of developing thinking skills. With various opportunities of communicating with the teacher, students would not feel powerless or frustrated when facing difficult tasks, thus ensuring the achievement of the learning objectives.

Four strategies were employed when designing the scaffolding and guidance. First, the process of thinking was highlighted. When the focus fell on critical thinking processes such as establishing viewpoints, making assumptions, and evaluating information, students had examples to follow when they conducted these activities independently. Second, the role of peers was given full play. In many cases, the demonstration of peers was more direct and effective for the students to develop critical thinking skills. Third, the teacher consciously created a “democratic” classroom and online atmosphere, where students could express their opinions without fearing judgment from the “authority” or other people. Fourth, the teacher established awarding incentives to encourage students to take the initiative to meet challenges and develop thinking. For example, if one student's feedback to others' work was deeper and more thorough, the instructor gave the student more marks and demonstrated the work to the whole class with their permission.

Technological Design

Moodle (Modular Object-Oriented Dynamic Learning Environment) was the main platform of the e-learning environment. A composition reviewing and grading software TRP (Teaching Resources Platform) was also used to facilitate teachers' grading of the compositions. TRP mainly focuses on the mistakes related to language and grammar, which could help direct teachers' attention to the composition's structure, logic, coherence, and other aspects. In addition, Tencent QQ, a social networking software frequently used by students, was selected to send messages and notices to students.

As shown in Table 3 , both synchronous and asynchronous instruments were applied to provide sufficient communication among students in designing communicative tools. When designing the synchronous instruments, the instructor used the Tencent QQ, which could conveniently support the simultaneous real-time communication between learners and encourage group members to fully communicate with each other. The discussion board of Moodle was used as asynchronous tools, and sufficient time was given to the students to respond to other people's opinions or solve problems. The students could use the time to find information, consult others and translate complex ideas into words.

Table 3 . Strategies for technological design.

Research Instruments

Learning environment questionnaire.

The questionnaire adapted from the Web-Based Learning Environment Instrument (WEBLEI) was used to elicit the information of students' perception of the learning environment. The original WEBLEI questionnaire was first created and subsequently modified by Chang and Fisher for investigating online learning environments in University settings. The primary purpose of the questionnaire was to capture “students' perception of web-based learning environments” ( Chang and Fisher, 2003 , p. 9). The questions in the WEBLEI questionnaire are able to cover the three elements of the PST learning environment model. The researcher modified the questionnaire according to the context of the current study. The Cronbach alpha coefficients indicated the acceptable reliability of the modified questionnaire (see Table 4 ).

Table 4 . Cronbach alpha coefficients for modified WEBLEI.

In order to explore students' perceived improvement of critical thinking and the in-depth reasons behind students' perceptions of the learning environment and critical thinking instruction, interviews were conducted after the administration of the adapted WEBLEI questionnaire. Eight students were randomly chosen and invited to the interview one by one. The interviews lasted about 30 min and were audio-recorded with the participants' approval.

Data Analysis

Both quantitative and qualitative data were collected for this study. In terms of quantitative analysis, descriptive statistics were used to describe the means, standard deviations. As for qualitative data, the recordings of the interviews were transcribed for content analysis. The content about the perceptions of the environment was categorized with the outline of the learning environment components. Regarding the development of students' critical thinking, the “elements of thinking” from Paul and Elder's thinking model formed the framework for data analysis. The relevant script was examined and coded according to the framework by the researcher and her collegue to generalize the aspects of critical thinking improvement.

Results and Discussion

Students' perceptions of the environment, students' perception of the pedagogical design.

The means and standard deviation scores of students' perception of the pedagogical design are listed in Table 5 . The overall mean score was 3.86 (SD = 0.79), suggesting that students were generally satisfied with the pedagogical design. Item 1 (M = 3.98, SD = 0.80) (The learning objectives are clearly stated), Item 4 ( M = 3.93, SD = 0.83) (Expectations of assignments are clearly stated), and Item 5 (M = 4, SD = 1.00) (Activities are planned carefully) got particularly high scores, which indicates that students were aware of the careful design of the activities, content, and context.

Table 5 . Students' Perceptions of the Environment.

The students' positive attitude toward the pedagogical design was also revealed from the interview, in which they expressed their satisfaction with the design of tasks and contexts. For example, Student A expressed that the course was designed in the way that they needed to “find solutions to the problems” by themselves most of the time and he also enjoyed the discussions in class. Student C recognized the relative authentic contexts of the tasks, which helped her devote herself to the tasks. She mentioned that in learning to write a CV, the teacher asked the students to imagine the situation in which they were about to graduate and hunt a job. “I felt the topic was very relevant to me, so I was motivated to do this task well.” She told the interviewer.

Apart from the positive opinions, some students expressed their concern about the pedagogical design. For example, Student H said, “The online learning added to our workload. Sometimes I was scared of all the online assignments we had to finish after class.” And student G had difficulty adapting to this learning approach. “It seemed that we were learning by ourselves. I am not sure whether I have learned enough knowledge. I would rather learn how to write from the teacher.”

Students' Perception of Social Design

As seen from Table 5 , the overall mean score of the social design was 3.90 ( M = 0.82), indicating students' generally positive attitude toward the social design. The data gathered from the students' interviews also suggested that students were satisfied with the social design. For example, student B mentioned that she always received encouragement and help when dealing with difficult tasks. Item 11 ( M = 4.07, SD = 0.65) (Other students respond promptly to my request), Item 12 ( M = 4.09, SD = 0.91) (The teachers give me quick comments on my work) and Item 14 ( M = 4.07, SD = 0.58) (I was supported by a positive attitude from my teacher and my classmates) scored higher than Item 9 ( M = 3.47, SD = 1.01) (I can ask my teacher what I do not understand) and Item 10 ( M = 3.79, SD = 0.78) (I can ask other classmates what I do not understand). This finding reveals that in the environment, both students and teachers responded to others promptly, but students had considerations when they needed to consult others.

When asked the reason for this, the students suggested that the teacher and the environment did provide them with the opportunity to seek help, but sometimes they felt reluctant to trouble others. Student E mentioned when he found something he failed to understand, he would prefer to figure it out by himself first and then seek help from the teacher and classmates. He told the interviewer: “I thought the teacher was busy, and my classmates were also busy, so I would prefer to figure it out by myself.”

Students' Perceptions of Technical Design

As for the technical design (see Table 5 ), the average score is 3.73 (SD = 0.85), which suggests that the environment provided relatively sufficient technological support to the students. Item 16 ( M = 3.93, SD = 0.92) (The online material is available at locations suitable for me) and Item 19 ( M = 4, SD = 0.97) (I decide when I want to learn) got higher scores, which indicates that students could enjoy the convenience of “anywhere” and “anytime” in the learning environment.

This positive attitude was demonstrated in the interview data collected from Student F who expressed his appreciation for the freedom and the sense of control brought by asynchronous discussion. He said, “I could finish the task at the time that is convenient for me as long as I did not miss the deadline. I like it.”

One thing worth noticing is that the mean score of Item 20 (Using blended learning allowed me to explore the interest of my own) is 3.18 (SD = 0.68), which falls toward the middle of the 1–5 scale. This score reveals that students did not think the resources of the blended learning environment play an important role in exploring their own areas of interest. In the interview, student D expressed that he did not find the resources very interesting, for the range of the topics was rather limited, and he was not attracted by the resources provided.

In sum, students' ratings on different dimensions of the questionnaire suggest that students perceived the productiveness of the learning environment in a generally positive way. This result is consistent with the studies exploring students' perceptions of the blending learning environment in general (e.g., Akkoyunlu and Yilmaz, 2006 ; Dziuban et al., 2006 ; Owston et al., 2006 ; Bendania, 2011 ; Wang and Huang, 2018 ). In the study conducted by Wang and Huang (2018) , a blended environment was also constructed from the pedagogical, social, and technical perspectives. The findings of the study reveal that students are generally positive toward the design of the learning environment. This may suggest that students would perceive the learning environment positively if the elements of the blended learning environment are carefully designed. Despite the generally positive attitudes toward the learning environment, some students expressed their concern about the workload and adaptation to the way of learning in the interview. In study Stracke (2007) , the way of learning was also found to make the students withdraw from the blended course. The findings indicate that some students may need more time to adapt to more student-centered learning.

Students' Perceived Impact of the Blended Learning Environment on the Development of Their Critical Thinking

Drawing mainly on Paul and Elder's framework of thinking elements, the following themes emerged as to the students' perceived improvement of critical thinking after data analysis and are elucidated through students' quotations.

Gaining a Deeper Understanding of the Concept of “Critical Thinking”

In the interview, students talked about their improvement in understanding the concept of critical thinking. For example, Student D expressed that the environment helped him clarify the concept of critical thinking. He used to consider the concept as closely related to “criticizing” because of its Chinese translation and came to realize that it was closer to the concept of “rational thinking.”

Some students also expressed that the course helped them realize the importance of critical thinking. As the teacher clearly informed the students of the specific critical thinking skills each task aimed to cultivate, students realized that “critical thinking is not an abstract concept, but concrete ways of guiding people to solve problems” (Student B).

Using Facts and Evidence to Support One's Own Opinion

In the interview, students also talked about the change they experienced when forming and supporting their opinion. They started to recognize the importance of facts and evidence in their writing. Student E told the interviewer that he learned that supporting ideas were very important to make one's opinion accepted. He said: “In accomplishing the writing tasks of the course, I gradually learned to provide arguments with further explanations, examples and,… maybe some data.”

Some students also suggested that facts and evidence were important for them to convince others in the discussions. Student B said: “In the past when someone disagreed with me, I usually felt sad and angry. I would either remain silent or quarrel with them. In this course, I learned that if I wanted others to accept my opinion, I needed to convince them with evidence such as facts and information.” She also felt excited that her well-presented opinions were accepted several times during the discussion with her team members.

Thinking From Multiple Perspectives

Another perceived effect is thinking from multiple perspectives, which was mentioned by many students. For example, Student A described how a particular activity helped him recognize the importance of different perspectives and how his own writing benefited from a particular activity in the course. “The teacher asked some of us to play the role of employer and I was assigned this role. When I thought from the employer's perspective, I knew what kind of employee I needed… When I wrote my job application letter, I had a very clear idea what to include in my letter.” (Student A) Student F also mentioned that recognizing different perspectives helped him finish writing the complaint letter well. According to him, he not only mentioned the dissatisfaction in the complaint letter but also stated the potential negative impact on the company to which he sent the letter.

Exploring and Clarifying the Purpose Behind the Texts or Behaviors

The interviewees also mentioned that they learned to explore and clarify the purpose behind the texts or behaviors. Some students explained how they started to consider purpose as an important component in their writing. Student H told the interviewer that when the teacher started to teach a new genre, she always asked the students to discuss under what circumstances they could meet or use this type of writing, and why they needed it in the daily life. “In this way, I understand that there should be a clear purpose behind each writing. And… and when I tried to finish my own writing task, I also put the writing purpose into my consideration.” said Student H.

Some students also told the interviewer that they gradually learned to avoid distraction and stick to the purpose when they conducted a discussion. According to student G, the students tended to talk about irrelevant things when they had discussions at the beginning of the course. With the instructors' constant reminding, they could realize whether they strayed from the point and returned to the right track in time at the end of the semester.

In summary, the data from the interview suggest that students could perceive their critical thinking development in different thinking dimensions. Furthermore, according to the students' opinion, their development in critical thinking was also manifested in their writing and even transferred to other activities. As for the promoting factors of the development, the students recognized the importance of learning environment design, especially the pedagogical design and the social design. For example, students attributed their deeper understanding of the concept to the instructor's deliberate introduction of critical thinking and focus on the development of thinking skills in the activity design. Also, they believed that the teachers' guidance and peers' scaffold enabled them to realize the importance of multiple perspectives. These factors were also found to promote students' critical thinking in the systematic review conducted by Chou et al. (2018) . This suggests that designing the elements of the learning environment to provide favorable conditions for critical thinking development could bring positive effects.

Limitations and Implications

This study proposed the construction of a blended learning environment to promote critical thinking in terms of pedagogical, social, and technical design and explored students' perceptions of the environment design and their perceived impact on the improvement of critical thinking. The results of the study suggests that students are generally satisfied with the design of the learning environment, and students considered the learning environment helpful in improving critical thinking. Even though the study made a contribution to the instructional design aiming at critical thinking promotion in a blended learning environment, some limitations should be duly noted. First, because the participants of the study were 90 students in the same University, the relative homogeneity of the context may present a possible connection with the result. Therefore, replication is recommended with larger and more diverse samples. Second, the study was not able to present the relationship between environmental design and critical thinking development quantitively. Further study could focus on the correlation between design strategies and the improvement of specific thinking skills, or the predictive capability of elements design for the promotion of critical thinking.

This study also has some implications for critical thinking cultivation in the instruction of specific disciplines. On the one hand, the cultivation of students' critical thinking requires the detailed design of the blended learning environment. Special attention needs to be paid to pedagogical, social, and technical design covering factors such as learning objectives, student interaction, and ICT tools. On the other hand, students' troubles and challenges such as the extra workload and emotional factors should be taken into consideration when designing the learning environment.

Data Availability Statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author/s.

Ethics Statement

The studies involving human participants were reviewed and approved by School of Foreign Languages, Northeast Normal University. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

DL designed and implemented the learning environment, collected and analyzed the data, and wrote the article.

Conflict of Interest

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Adam, A., and Manson, T. M. (2014). Using a pseudoscience activity to teach critical thinking. Teach. Psychol. 41, 130–134. doi: 10.1177/0098628314530343

CrossRef Full Text | Google Scholar

Akkoyunlu, B., and Yilmaz, S. M. (2006). A study on students' views on blended learning environment. Turk. Online J. Distance Educ. 7, 43–56. doi: 10.17718/TOJDE.25211

Allaire, J. L. (2015). Assessing critical thinking outcomes of dental hygiene students utilizing virtual patient simulation: a mixed methods study. J. Dent. Educ. 79, 1082–1092. doi: 10.1002/j.0022-0337.2015.79.9.tb06002.x

PubMed Abstract | CrossRef Full Text | Google Scholar

BatdÄ, V. (2014). The effect of blended learning environments on academic success of students: a meta-analysis study. Cankiri Karatekin Univ. J. Inst. Soc. Sci. 5, 287–302. Retrieved from: https://dergipark.org.tr/en/pub/jiss/issue/25892/272867 (accessed June 11, 2021).

Bendania, A. (2011). Teaching and learning online: King Fahd University of Petroleum and Minerals (KFUPM) Saudi Arabia, case study. Int. J. Arts Sci. 4, 223–241. Retrieved from: https://www.academia.edu/5017206/INSTRUCTORS_AND_LEARNERS_ATTITUDES_TOWARD_TEACHING_AND_LEARNING_ONLINE_KING_FAHD_UNIVERSITY_OF_PETROLEUM_AND_MINERALS_KFUPM_SAUDI_ARABIA_CASE_STUDY (accessed June 11, 2021).

Google Scholar

Butchart, S., Bigelow, J., Oppy, G., Korb, K., and Gold, I. (2009). Improving critical thinking using web-based argument mapping exercises with automated feedback. Australas. J. Educ. Technol. 25, 268–291. doi: 10.14742/ajet.1154

Chang, V., and Fisher, D. (2003). “The validation and application of a new learning environment instrument for online learning in higher education,” in Technology-Rich Learning Environments: A Future Perspective , eds M. S. Khine and D. Fisher (Singapore: World Scientific Publishing Co. Pte. Ltd.), 1–20.

Cheong, C. M., and Cheung, W. S. (2008). Online discussion and critical thinking skills: a case study in a Singapore secondary school. Australas. J. Educ. Technol. 24, 556–573. doi: 10.14742/ajet.1191

Chou, T. L., Wu, J. J., and Tsai, C. C. (2018). Research trends and features of critical thinking studies in E-Learning environments: a review. J. Educ. Comput. Res. 57, 1038–1077. doi: 10.1177/0735633118774350

de Leng, B. A., Dolmans, D. H., Jo bsis, R., Muijtjens, A. M., and van der Vleuten, C. P. (2009). Exploration of an e-learning model to foster critical thinking on basic science concepts during work placements. Comput. Educ. 53, 1–13. doi: 10.1016/j.compedu.2008.12.012

Dziuban, C., Hartman, J., Juge, F., Moskal, P., and Sorg, S. (2006). “Blended learning enters the mainstream,” in Handbook of Blended Learning: Global Perspectives, Local Designs , eds C. J. Bonk and C. R. Graham (San Francisco, CA: Pfeiffer), 195–206. Retrieved from: https://www.researchgate.net/publication/284688507_Blended_learning_enters_the_mainstream (accessed June 11, 2021).

Eftekhari, M., Sotoudehnama, E., and Marandi, S. S. (2016). Computer-aided argument mapping in an EFL setting: does technology precede traditional paper and pencil approach in developing critical thinking? Educ. Technol. Res. Dev. 64, 339–357. doi: 10.1007/s11423-016-9431-z

Elder, L., and Paul, R. (1994). Critical thinking: why we must transform our teaching. J. Dev. Educ. 18, 34–35.

Ennis, R. H. (1996). Critical Thinking . New York, NY: Prentice Hall.

Fisher, A., and Scriven, M. (1997). Critical Thinking: Its Definition and Assessment . Norwich: Center for Research in Critical Thinking.

Gilbert, P. K., and Dabbagh, N. (2005). How to structure online discussion of meaningful discourse: a case study. Br. J. Educ. Technol. 36, 5–18. doi: 10.1111/j.1467-8535.2005.00434.x

Guiller, J., Durndell, A., and Ross, A. (2008). Peer interaction and critical thinking: face-to-face or online discussion? Learn. Instruct. 18, 187–200. doi: 10.1016/j.learninstruc.2007.03.001

Huang, T. C., Jeng, Y. L., Hsiao, K. L., and Tsai, B. R. (2017). SNS collaborative learning design: enhancing critical thinking for human computer interface design. Univ. Access Inf. Soc. 16, 303–312. doi: 10.1007/s10209-016-0458-z

Kim, K., and Bonk, C. J. (2006). The future of online teaching and learning in higher education: the survey says. Educ. Q. 29, 22–30. Retrieved from: https://er.educause.edu/articles/2006/1/the-future-of-online-teaching-and-learning-in-higher-education-the-survey-says (accessed June 11, 2021).

Kirschner, P., Strijbos, J. W., Kreijns, K., and Beers, P. J. (2004). Designing electronic collaborative learning environments. Educ. Technol. Res. Dev. 52, 47–66. doi: 10.1007/BF02504675

Lu, D. (2018). Research on the design and application of blended learning environment with the orientation of critical thinking: a case of college practical English writing course (Unpublished doctoral dissertation), Northeast Normal University, Changchun, China.

Naaj, M. A., Nachouki, M., and Ankit, A. (2012). Evaluating student satisfaction with blended learning in a gender-segregated environment. J. Inf. Technol. Educ. Res. 11, 185–200. doi: 10.28945/1692

Osguthorpe, R. T., and Graham, C. R. (2003). Blended learning environments: definitions and directions. Q. Rev. Distance Educ. 4, 227–233. Retrieved from: https://web.b.ebscohost.com/ehost/pdfviewer/pdfviewer?vid=0&sid=84d5540c-01b3-4b86-8c21-962ff82af437%40sessionmgr103 (accessed June 11, 2021).

Owston, R. D., Garrison, D. R., and Cook, K. (2006). “Blended learning at Canadian universities: issues and practices,” in The Handbook of Blended Learning: Global Perspectives, Local Designs , eds C. J. Bonk and C. R. Graham (San Francisco, CA: Pfeiffer), 338–350.

Paul, R. (1992). Critical thinking: what, why and how? New Dir. Community Coll. 1992, 3–24. doi: 10.1002/cc.36819927703

Paul, R., and Elder, L. (1999). Critical thinking: teaching students to seek the logic of things. J. Dev. Educ. 23, 34–36.

Paul, R., and Elder, L. (2006). The International Critical Thinking Reading and Writing Test . Dillon Beach, CA: The Foundation for Critical Thinking.

Rosen, Y., and Tager, M. (2014). Making student thinking visible through a concept map in computer-based assessment of critical thinking. J. Educ. Comput. Res. 50, 249–270. doi: 10.2190/EC.50.2.f

Sagarra, N., and Zapata, G. C. (2008). Blending classroom instruction with online homework: a study of student perceptions of computer-assisted L2 learning. ReCALL 20, 208–224. doi: 10.1017/S0958344008000621

Shamir, A., Zion, M., and Spector_Levi, O. (2008). Peer tutoring, metacognitive processes and multimedia problem-based learning: the effect of mediation training on critical thinking. J. Sci. Educ. Technol. 17, 384–398. doi: 10.1007/s10956-008-9108-4

Shin, H., Ma, H., Park, J., Ji, E. S., and Kim, D. H. (2015). The effect of simulation courseware on critical thinking in undergraduate nursing students: multi-site pre-post study. Nurse Educ. Today 35, 537–542. doi: 10.1016/j.nedt.2014.12.004

Singh, H. (2003). Building effective blended learning programs. Educ. Technol. 43, 51–54. Retrieved from: http://www.jstor.org/stable/44428863 (accessed June 11, 2021).

Singh, H., and Reed, C. (2001). A White Paper: Achieving Success With Blended Learning . Retrieved from: http://www.leerbeleving.nl/wbts/wbt2014/blend-ce.pdf (accessed June 11, 2021).

Smyth, S., Houghton, C., Cooney, A., and Casey, D. (2012). Students' experiences of blended learning across a range of postgraduate programmes. Nurse Educ. Today 32, 464–468. doi: 10.1016/j.nedt.2011.05.014

Stracke, E. (2007). A road to understanding: a qualitative study into why learners drop out of a blended language learning (BLL) environment. ReCALL 19, 57–78. doi: 10.1017/S0958344007000511

Thorne, K. (2003). Blended Learning: How to Integrate Online and Traditional Learning . London: Kogan Page.

Van Gelder, T., and Bulker, A. (2000). “Reason! improving informal reasoning skills,” in Proceedings of the Australian Computers in Education Conference (Melbourne, VIC). Retrieved from: https://pdfs.semanticscholar.org/ce84/ec799ae2dc15d56939fd0a6e46123e88112e.pdf (accessed April 20, 2020).

Wang, Q., and Huang, C. (2018). Pedagogical, social and technical designs of a blended synchronous learning environment. Br. J. Educ. Technol. 49, 451–462. doi: 10.1111/bjet.12558

Wang, Q. Y. (2008). A generic model for guiding the integration of ICT into teaching and learning. Innov. Educ. Teach. Int. 45, 411–419. doi: 10.1080/14703290802377307

Watson, J. (2008). Blended Learning: The Convergence of Online and Face-to-Face Education. North American Council for Online Learning report . Retrieved from: https://files.eric.ed.gov/fulltext/ED509636.pdf (accessed June 11, 2021).

Yang, Y. T. C. (2008). A catalyst for teaching critical thinking in a large University class in Taiwan: asynchronous online discussions with the facilitation of teaching assistants. Educ. Technol. Res. Dev. 56, 241–264. doi: 10.1007/s11423-007-9054-5

Yang, Y. T. C., and Chang, C. H. (2013). Empowering students through digital game authorship: enhancing concentration, critical thinking, and academic achievement. Comput. Educ. 68, 334–344. doi: 10.1016/j.compedu.2013.05.023

Yang, Y. T. C., and Chou, H. A. (2008). Beyond critical thinking skills: investigating the relationship between critical thinking skills and dispositions through different online instructional strategies. Br. J. Educ. Technol. 39, 666–684. doi: 10.1111/j.1467-8535.2007.00767.x

Yeh, Y. C. (2009). Integrating e-learning into the direct-instruction model to enhance the effectiveness of critical-thinking instruction. Instruct. Sci. 37, 185–203. doi: 10.1007/s11251-007-9048-z

Yen, J. -C., and Lee, C. -Y. (2011). Exploring problem solving patterns and their impact on learning achievement in a blended learning environment. Comput. Educ. 56, 138–145. doi: 10.1016/j.compedu.2010.08.012

Yukawa, J. (2006). Co-reflection in online learning: collaborative critical thinking as narrative. Int. J. Comput. Suppor. Collab. Learn. 1, 203–228. doi: 10.1007/s11412-006-8994-9

Keywords: students' perceptions, blended learning environment, critical thinking, design, survey

Citation: Lu D (2021) Students' Perceptions of a Blended Learning Environment to Promote Critical Thinking. Front. Psychol. 12:696845. doi: 10.3389/fpsyg.2021.696845

Received: 18 April 2021; Accepted: 31 May 2021; Published: 25 June 2021.

Reviewed by:

Copyright © 2021 Lu. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Dan Lu, lud090@nenu.edu.cn

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

• Research article
• Open access
• Published: 15 February 2018

Blended learning: the new normal and emerging technologies

• Charles Dziuban 1 ,
• Charles R. Graham 2 ,
• Patsy D. Moskal   ORCID: orcid.org/0000-0001-6376-839X 1 ,
• Anders Norberg 3 &
• Nicole Sicilia 1  

International Journal of Educational Technology in Higher Education volume  15 , Article number:  3 ( 2018 ) Cite this article

553k Accesses

362 Citations

118 Altmetric

Metrics details

This study addressed several outcomes, implications, and possible future directions for blended learning (BL) in higher education in a world where information communication technologies (ICTs) increasingly communicate with each other. In considering effectiveness, the authors contend that BL coalesces around access, success, and students’ perception of their learning environments. Success and withdrawal rates for face-to-face and online courses are compared to those for BL as they interact with minority status. Investigation of student perception about course excellence revealed the existence of robust if-then decision rules for determining how students evaluate their educational experiences. Those rules were independent of course modality, perceived content relevance, and expected grade. The authors conclude that although blended learning preceded modern instructional technologies, its evolution will be inextricably bound to contemporary information communication technologies that are approximating some aspects of human thought processes.

Introduction

Blended learning and research issues.

Blended learning (BL), or the integration of face-to-face and online instruction (Graham 2013 ), is widely adopted across higher education with some scholars referring to it as the “new traditional model” (Ross and Gage 2006 , p. 167) or the “new normal” in course delivery (Norberg et al. 2011 , p. 207). However, tracking the accurate extent of its growth has been challenging because of definitional ambiguity (Oliver and Trigwell 2005 ), combined with institutions’ inability to track an innovative practice, that in many instances has emerged organically. One early nationwide study sponsored by the Sloan Consortium (now the Online Learning Consortium) found that 65.2% of participating institutions of higher education (IHEs) offered blended (also termed hybrid ) courses (Allen and Seaman 2003 ). A 2008 study, commissioned by the U.S. Department of Education to explore distance education in the U.S., defined BL as “a combination of online and in-class instruction with reduced in-class seat time for students ” (Lewis and Parsad 2008 , p. 1, emphasis added). Using this definition, the study found that 35% of higher education institutions offered blended courses, and that 12% of the 12.2 million documented distance education enrollments were in blended courses.

The 2017 New Media Consortium Horizon Report found that blended learning designs were one of the short term forces driving technology adoption in higher education in the next 1–2 years (Adams Becker et al. 2017 ). Also, blended learning is one of the key issues in teaching and learning in the EDUCAUSE Learning Initiative’s 2017 annual survey of higher education (EDUCAUSE 2017 ). As institutions begin to examine BL instruction, there is a growing research interest in exploring the implications for both faculty and students. This modality is creating a community of practice built on a singular and pervasive research question, “How is blended learning impacting the teaching and learning environment?” That question continues to gain traction as investigators study the complexities of how BL interacts with cognitive, affective, and behavioral components of student behavior, and examine its transformation potential for the academy. Those issues are so compelling that several volumes have been dedicated to assembling the research on how blended learning can be better understood (Dziuban et al. 2016 ; Picciano et al. 2014 ; Picciano and Dziuban 2007 ; Bonk and Graham 2007 ; Kitchenham 2011 ; Jean-François 2013 ; Garrison and Vaughan 2013 ) and at least one organization, the Online Learning Consortium, sponsored an annual conference solely dedicated to blended learning at all levels of education and training (2004–2015). These initiatives address blended learning in a wide variety of situations. For instance, the contexts range over K-12 education, industrial and military training, conceptual frameworks, transformational potential, authentic assessment, and new research models. Further, many of these resources address students’ access, success, withdrawal, and perception of the degree to which blended learning provides an effective learning environment.

Currently the United States faces a widening educational gap between our underserved student population and those communities with greater financial and technological resources (Williams 2016 ). Equal access to education is a critical need, one that is particularly important for those in our underserved communities. Can blended learning help increase access thereby alleviating some of the issues faced by our lower income students while resulting in improved educational equality? Although most indicators suggest “yes” (Dziuban et al. 2004 ), it seems that, at the moment, the answer is still “to be determined.” Quality education presents a challenge, evidenced by many definitions of what constitutes its fundamental components (Pirsig 1974 ; Arum et al. 2016 ). Although progress has been made by initiatives, such as, Quality Matters ( 2016 ), the OLC OSCQR Course Design Review Scorecard developed by Open SUNY (Open SUNY n.d. ), the Quality Scorecard for Blended Learning Programs (Online Learning Consortium n.d. ), and SERVQUAL (Alhabeeb 2015 ), the issue is by no means resolved. Generally, we still make quality education a perceptual phenomenon where we ascribe that attribute to a course, educational program, or idea, but struggle with precisely why we reached that decision. Searle ( 2015 ), summarizes the problem concisely arguing that quality does not exist independently, but is entirely observer dependent. Pirsig ( 1974 ) in his iconic volume on the nature of quality frames the context this way,

“There is such thing as Quality, but that as soon as you try to define it, something goes haywire. You can’t do it” (p. 91).

Therefore, attempting to formulate a semantic definition of quality education with syntax-based metrics results in what O’Neil (O'Neil 2017 ) terms surrogate models that are rough approximations and oversimplified. Further, the derived metrics tend to morph into goals or benchmarks, losing their original measurement properties (Goodhart 1975 ).

Information communication technologies in society and education

Blended learning forces us to consider the characteristics of digital technology, in general, and information communication technologies (ICTs), more specifically. Floridi ( 2014 ) suggests an answer proffered by Alan Turing: that digital ICTs can process information on their own, in some sense just as humans and other biological life. ICTs can also communicate information to each other, without human intervention, but as linked processes designed by humans. We have evolved to the point where humans are not always “in the loop” of technology, but should be “on the loop” (Floridi 2014 , p. 30), designing and adapting the process. We perceive our world more and more in informational terms, and not primarily as physical entities (Floridi 2008 ). Increasingly, the educational world is dominated by information and our economies rest primarily on that asset. So our world is also blended, and it is blended so much that we hardly see the individual components of the blend any longer. Floridi ( 2014 ) argues that the world has become an “infosphere” (like biosphere) where we live as “inforgs.” What is real for us is shifting from the physical and unchangeable to those things with which we can interact.

Floridi also helps us to identify the next blend in education, involving ICTs, or specialized artificial intelligence (Floridi 2014 , 25; Norberg 2017 , 65). Learning analytics, adaptive learning, calibrated peer review, and automated essay scoring (Balfour 2013 ) are advanced processes that, provided they are good interfaces, can work well with the teacher— allowing him or her to concentrate on human attributes such as being caring, creative, and engaging in problem-solving. This can, of course, as with all technical advancements, be used to save resources and augment the role of the teacher. For instance, if artificial intelligence can be used to work along with teachers, allowing them more time for personal feedback and mentoring with students, then, we will have made a transformational breakthrough. The Edinburg University manifesto for teaching online says bravely, “Automation need not impoverish education – we welcome our robot colleagues” (Bayne et al. 2016 ). If used wisely, they will teach us more about ourselves, and about what is truly human in education. This emerging blend will also affect curricular and policy questions, such as the what? and what for? The new normal for education will be in perpetual flux. Floridi’s ( 2014 ) philosophy offers us tools to understand and be in control and not just sit by and watch what happens. In many respects, he has addressed the new normal for blended learning.

Literature of blended learning

A number of investigators have assembled a comprehensive agenda of transformative and innovative research issues for blended learning that have the potential to enhance effectiveness (Garrison and Kanuka 2004 ; Picciano 2009 ). Generally, research has found that BL results in improvement in student success and satisfaction, (Dziuban and Moskal 2011 ; Dziuban et al. 2011 ; Means et al. 2013 ) as well as an improvement in students’ sense of community (Rovai and Jordan 2004 ) when compared with face-to-face courses. Those who have been most successful at blended learning initiatives stress the importance of institutional support for course redesign and planning (Moskal et al. 2013 ; Dringus and Seagull 2015 ; Picciano 2009 ; Tynan et al. 2015 ). The evolving research questions found in the literature are long and demanding, with varied definitions of what constitutes “blended learning,” facilitating the need for continued and in-depth research on instructional models and support needed to maximize achievement and success (Dringus and Seagull 2015 ; Bloemer and Swan 2015 ).

Educational access

The lack of access to educational technologies and innovations (sometimes termed the digital divide) continues to be a challenge with novel educational technologies (Fairlie 2004 ; Jones et al. 2009 ). One of the promises of online technologies is that they can increase access to nontraditional and underserved students by bringing a host of educational resources and experiences to those who may have limited access to on-campus-only higher education. A 2010 U.S. report shows that students with low socioeconomic status are less likely to obtain higher levels of postsecondary education (Aud et al. 2010 ). However, the increasing availability of distance education has provided educational opportunities to millions (Lewis and Parsad 2008 ; Allen et al. 2016 ). Additionally, an emphasis on open educational resources (OER) in recent years has resulted in significant cost reductions without diminishing student performance outcomes (Robinson et al. 2014 ; Fischer et al. 2015 ; Hilton et al. 2016 ).

Unfortunately, the benefits of access may not be experienced evenly across demographic groups. A 2015 study found that Hispanic and Black STEM majors were significantly less likely to take online courses even when controlling for academic preparation, socioeconomic status (SES), citizenship, and English as a second language (ESL) status (Wladis et al. 2015 ). Also, questions have been raised about whether the additional access afforded by online technologies has actually resulted in improved outcomes for underserved populations. A distance education report in California found that all ethnic minorities (except Asian/Pacific Islanders) completed distance education courses at a lower rate than the ethnic majority (California Community Colleges Chancellor’s Office 2013 ). Shea and Bidjerano ( 2014 , 2016 ) found that African American community college students who took distance education courses completed degrees at significantly lower rates than those who did not take distance education courses. On the other hand, a study of success factors in K-12 online learning found that for ethnic minorities, only 1 out of 15 courses had significant gaps in student test scores (Liu and Cavanaugh 2011 ). More research needs to be conducted, examining access and success rates for different populations, when it comes to learning in different modalities, including fully online and blended learning environments.

Framing a treatment effect

Over the last decade, there have been at least five meta-analyses that have addressed the impact of blended learning environments and its relationship to learning effectiveness (Zhao et al. 2005 ; Sitzmann et al. 2006 ; Bernard et al. 2009 ; Means et al. 2010 , 2013 ; Bernard et al. 2014 ). Each of these studies has found small to moderate positive effect sizes in favor of blended learning when compared to fully online or traditional face-to-face environments. However, there are several considerations inherent in these studies that impact our understanding the generalizability of outcomes.

Dziuban and colleagues (Dziuban et al. 2015 ) analyzed the meta-analyses conducted by Means and her colleagues (Means et al. 2013 ; Means et al. 2010 ), concluding that their methods were impressive as evidenced by exhaustive study inclusion criteria and the use of scale-free effect size indices. The conclusion, in both papers, was that there was a modest difference in multiple outcome measures for courses featuring online modalities—in particular, blended courses. However, with blended learning especially, there are some concerns with these kinds of studies. First, the effect sizes are based on the linear hypothesis testing model with the underlying assumption that the treatment and the error terms are uncorrelated, indicating that there is nothing else going on in the blending that might confound the results. Although the blended learning articles (Means et al. 2010 ) were carefully vetted, the assumption of independence is tenuous at best so that these meta-analysis studies must be interpreted with extreme caution.

There is an additional concern with blended learning as well. Blends are not equivalent because of the manner on which they are configured. For instance, a careful reading of the sources used in the Means, et al. papers will identify, at minimum, the following blending techniques: laboratory assessments, online instruction, e-mail, class web sites, computer laboratories, mapping and scaffolding tools, computer clusters, interactive presentations and e-mail, handwriting capture, evidence-based practice, electronic portfolios, learning management systems, and virtual apparatuses. These are not equivalent ways in which to configure courses, and such nonequivalence constitutes the confounding we describe. We argue here that, in actuality, blended learning is a general construct in the form of a boundary object (Star and Griesemer 1989 ) rather than a treatment effect in the statistical sense. That is, an idea or concept that can support a community of practice, but is weakly defined fostering disagreement in the general group. Conversely, it is stronger in individual constituencies. For instance, content disciplines (i.e. education, rhetoric, optics, mathematics, and philosophy) formulate a more precise definition because of commonly embraced teaching and learning principles. Quite simply, the situation is more complicated than that, as Leonard Smith ( 2007 ) says after Tolstoy,

“All linear models resemble each other, each non nonlinear system is unique in its own way” (p. 33).

This by no means invalidates these studies, but effect size associated with blended learning should be interpreted with caution where the impact is evaluated within a particular learning context.

Study objectives

This study addressed student access by examining success and withdrawal rates in the blended learning courses by comparing them to face-to-face and online modalities over an extended time period at the University of Central Florida. Further, the investigators sought to assess the differences in those success and withdrawal rates with the minority status of students. Secondly, the investigators examined the student end-of-course ratings of blended learning and other modalities by attempting to develop robust if-then decision rules about what characteristics of classes and instructors lead students to assign an “excellent” value to their educational experience. Because of the high stakes nature of these student ratings toward faculty promotion, awards, and tenure, they act as a surrogate measure for instructional quality. Next, the investigators determined the conditional probabilities for students conforming to the identified rule cross-referenced by expected grade, the degree to which they desired to take the course, and course modality.

Student grades by course modality were recoded into a binary variable with C or higher assigned a value of 1, and remaining values a 0. This was a declassification process that sacrificed some specificity but compensated for confirmation bias associated with disparate departmental policies regarding grade assignment. At the measurement level this was an “on track to graduation index” for students. Withdrawal was similarly coded by the presence or absence of its occurrence. In each case, the percentage of students succeeding or withdrawing from blended, online or face-to-face courses was calculated by minority and non-minority status for the fall 2014 through fall 2015 semesters.

Next, a classification and regression tree (CART) analysis (Brieman et al. 1984 ) was performed on the student end-of-course evaluation protocol ( Appendix 1 ). The dependent measure was a binary variable indicating whether or not a student assigned an overall rating of excellent to his or her course experience. The independent measures in the study were: the remaining eight rating items on the protocol, college membership, and course level (lower undergraduate, upper undergraduate, and graduate). Decision trees are efficient procedures for achieving effective solutions in studies such as this because with missing values imputation may be avoided with procedures such as floating methods and the surrogate formation (Brieman et al. 1984 , Olshen et al. 1995 ). For example, a logistic regression method cannot efficiently handle all variables under consideration. There are 10 independent variables involved here; one variable has three levels, another has nine, and eight have five levels each. This means the logistic regression model must incorporate more than 50 dummy variables and an excessively large number of two-way interactions. However, the decision-tree method can perform this analysis very efficiently, permitting the investigator to consider higher order interactions. Even more importantly, decision trees represent appropriate methods in this situation because many of the variables are ordinally scaled. Although numerical values can be assigned to each category, those values are not unique. However, decision trees incorporate the ordinal component of the variables to obtain a solution. The rules derived from decision trees have an if-then structure that is readily understandable. The accuracy of these rules can be assessed with percentages of correct classification or odds-ratios that are easily understood. The procedure produces tree-like rule structures that predict outcomes.

The model-building procedure for predicting overall instructor rating

For this study, the investigators used the CART method (Brieman et al. 1984 ) executed with SPSS 23 (IBM Corp 2015 ). Because of its strong variance-sharing tendencies with the other variables, the dependent measure for the analysis was the rating on the item Overall Rating of the Instructor , with the previously mentioned indicator variables (college, course level, and the remaining 8 questions) on the instrument. Tree methods are recursive, and bisect data into subgroups called nodes or leaves. CART analysis bases itself on: data splitting, pruning, and homogeneous assessment.

Splitting the data into two (binary) subsets comprises the first stage of the process. CART continues to split the data until the frequencies in each subset are either very small or all observations in a subset belong to one category (e.g., all observations in a subset have the same rating). Usually the growing stage results in too many terminate nodes for the model to be useful. CART solves this problem using pruning methods that reduce the dimensionality of the system.

The final stage of the analysis involves assessing homogeneousness in growing and pruning the tree. One way to accomplish this is to compute the misclassification rates. For example, a rule that produces a .95 probability that an instructor will receive an excellent rating has an associated error of 5.0%.

Implications for using decision trees

Although decision-tree techniques are effective for analyzing datasets such as this, the reader should be aware of certain limitations. For example, since trees use ranks to analyze both ordinal and interval variables, information can be lost. However, the most serious weakness of decision tree analysis is that the results can be unstable because small initial variations can lead to substantially different solutions.

For this study model, these problems were addressed with the k-fold cross-validation process. Initially the dataset was partitioned randomly into 10 subsets with an approximately equal number of records in each subset. Each cohort is used as a test partition, and the remaining subsets are combined to complete the function. This produces 10 models that are all trained on different subsets of the original dataset and where each has been used as the test partition one time only.

Although computationally dense, CART was selected as the analysis model for a number of reasons— primarily because it provides easily interpretable rules that readers will be able evaluate in their particular contexts. Unlike many other multivariate procedures that are even more sensitive to initial estimates and require a good deal of statistical sophistication for interpretation, CART has an intuitive resonance with researcher consumers. The overriding objective of our choice of analysis methods was to facilitate readers’ concentration on our outcomes rather than having to rely on our interpretation of the results.

Institution-level evaluation: Success and withdrawal

The University of Central Florida (UCF) began a longitudinal impact study of their online and blended courses at the start of the distributed learning initiative in 1996. The collection of similar data across multiple semesters and academic years has allowed UCF to monitor trends, assess any issues that may arise, and provide continual support for both faculty and students across varying demographics. Table  1 illustrates the overall success rates in blended, online and face-to-face courses, while also reporting their variability across minority and non-minority demographics.

While success (A, B, or C grade) is not a direct reflection of learning outcomes, this overview does provide an institutional level indication of progress and possible issues of concern. BL has a slight advantage when looking at overall success and withdrawal rates. This varies by discipline and course, but generally UCF’s blended modality has evolved to be the best of both worlds, providing an opportunity for optimizing face-to-face instruction through the effective use of online components. These gains hold true across minority status. Reducing on-ground time also addresses issues that impact both students and faculty such as parking and time to reach class. In addition, UCF requires faculty to go through faculty development tailored to teaching in either blended or online modalities. This 8-week faculty development course is designed to model blended learning, encouraging faculty to redesign their course and not merely consider blended learning as a means to move face-to-face instructional modules online (Cobb et al. 2012 ; Lowe 2013 ).

Withdrawal (Table  2 ) from classes impedes students’ success and retention and can result in delayed time to degree, incurred excess credit hour fees, or lost scholarships and financial aid. Although grades are only a surrogate measure for learning, they are a strong predictor of college completion. Therefore, the impact of any new innovation on students’ grades should be a component of any evaluation. Once again, the blended modality is competitive and in some cases results in lower overall withdrawal rates than either fully online or face-to-face courses.

The students’ perceptions of their learning environments

Other potentially high-stakes indicators can be measured to determine the impact of an innovation such as blended learning on the academy. For instance, student satisfaction and attitudes can be measured through data collection protocols, including common student ratings, or student perception of instruction instruments. Given that those ratings often impact faculty evaluation, any negative reflection can derail the successful implementation and scaling of an innovation by disenfranchised instructors. In fact, early online and blended courses created a request by the UCF faculty senate to investigate their impact on faculty ratings as compared to face-to-face sections. The UCF Student Perception of Instruction form is released automatically online through the campus web portal near the end of each semester. Students receive a splash page with a link to each course’s form. Faculty receive a scripted email that they can send to students indicating the time period that the ratings form will be available. The forms close at the beginning of finals week. Faculty receive a summary of their results following the semester end.

The instrument used for this study was developed over a ten year period by the faculty senate of the University of Central Florida, recognizing the evolution of multiple course modalities including blended learning. The process involved input from several constituencies on campus (students, faculty, administrators, instructional designers, and others), in attempt to provide useful formative and summative instructional information to the university community. The final instrument was approved by resolution of the senate and, currently, is used across the university. Students’ rating of their classes and instructors comes with considerable controversy and disagreement with researchers aligning themselves on both sides of the issue. Recently, there have been a number of studies criticizing the process (Uttl et al. 2016 ; Boring et al. 2016 ; & Stark and Freishtat 2014 ). In spite of this discussion, a viable alternative has yet to emerge in higher education. So in the foreseeable future, the process is likely to continue. Therefore, with an implied faculty senate mandate this study was initiated by this team of researchers.

Prior to any analysis of the item responses collected in this campus-wide student sample, the psychometric quality (domain sampling) of the information yielded by the instrument was assessed. Initially, the reliability (internal consistency) was derived using coefficient alpha (Cronbach 1951 ). In addition, Guttman ( 1953 ) developed a theorem about item properties that leads to evidence about the quality of one’s data, demonstrating that as the domain sampling properties of items improve, the inverse of the correlation matrix among items will approach a diagonal. Subsequently, Kaiser and Rice ( 1974 ) developed the measure of sampling adequacy (MSA) that is a function of the Guttman Theorem. The index has an upper bound of one with Kaiser offering some decision rules for interpreting the value of MSA. If the value of the index is in the .80 to .99 range, the investigator has evidence of an excellent domain sample. Values in the .70s signal an acceptable result, and those in the .60s indicate data that are unacceptable. Customarily, the MSA has been used for data assessment prior to the application of any dimensionality assessments. Computation of the MSA value gave the investigators a benchmark for the construct validity of the items in this study. This procedure has been recommended by Dziuban and Shirkey ( 1974 ) prior to any latent dimension analysis and was used with the data obtained for this study. The MSA for the current instrument was .98 suggesting excellent domain sampling properties with an associated alpha reliability coefficient of .97 suggesting superior internal consistency. The psychometric properties of the instrument were excellent with both measures.

The online student ratings form presents an electronic data set each semester. These can be merged across time to create a larger data set of completed ratings for every course across each semester. In addition, captured data includes course identification variables including prefix, number, section and semester, department, college, faculty, and class size. The overall rating of effectiveness is used most heavily by departments and faculty in comparing across courses and modalities (Table  3 ).

The finally derived tree (decision rules) included only three variables—survey items that asked students to rate the instructor’s effectiveness at:

Helping students achieve course objectives,

Creating an environment that helps students learn, and

Communicating ideas and information.

None of the demographic variables associated with the courses contributed to the final model. The final rule specifies that if a student assigns an excellent rating to those three items, irrespective of their status on any other condition, the probability is .99 that an instructor will receive an overall rating of excellent. The converse is true as well. A poor rating on all three of those items will lead to a 99% chance of an instructor receiving an overall rating of poor.

Tables  4 , 5 and 6 present a demonstration of the robustness of the CART rule for variables on which it was not developed: expected course grade, desire to take the course and modality.

In each case, irrespective of the marginal probabilities, those students conforming to the rule have a virtually 100% chance of seeing the course as excellent. For instance, 27% of all students expecting to fail assigned an excellent rating to their courses, but when they conformed to the rule the percentage rose to 97%. The same finding is true when students were asked about their desire to take the course with those who strongly disagreed assigning excellent ratings to their courses 26% of the time. However, for those conforming to the rule, that category rose to 92%. When course modality is considered in the marginal sense, blended learning is rated as the preferred choice. However, from Table  6 we can observe that the rule equates student assessment of their learning experiences. If they conform to the rule, they will see excellence.

This study addressed increasingly important issues of student success, withdrawal and perception of the learning environment across multiple course modalities. Arguably these components form the crux of how we will make more effective decisions about how blended learning configures itself in the new normal. The results reported here indicate that blending maintains or increases access for most student cohorts and produces improved success rates for minority and non-minority students alike. In addition, when students express their beliefs about the effectiveness of their learning environments, blended learning enjoys the number one rank. However, upon more thorough analysis of key elements students view as important in their learning, external and demographic variables have minimal impact on those decisions. For example college (i.e. discipline) membership, course level or modality, expected grade or desire to take a particular course have little to do with their course ratings. The characteristics they view as important relate to clear establishment and progress toward course objectives, creating an effective learning environment and the instructors’ effective communication. If in their view those three elements of a course are satisfied they are virtually guaranteed to evaluate their educational experience as excellent irrespective of most other considerations. While end of course rating protocols are summative the three components have clear formative characteristics in that each one is directly related to effective pedagogy and is responsive to faculty development through units such as the faculty center for teaching and learning. We view these results as encouraging because they offer potential for improving the teaching and learning process in an educational environment that increases the pressure to become more responsive to contemporary student lifestyles.

Clearly, in this study we are dealing with complex adaptive systems that feature the emergent property. That is, their primary agents and their interactions comprise an environment that is more than the linear combination of their individual elements. Blending learning, by interacting with almost every aspect of higher education, provides opportunities and challenges that we are not able to fully anticipate.

This pedagogy alters many assumptions about the most effective way to support the educational environment. For instance, blending, like its counterpart active learning, is a personal and individual phenomenon experienced by students. Therefore, it should not be surprising that much of what we have called blended learning is, in reality, blended teaching that reflects pedagogical arrangements. Actually, the best we can do for assessing impact is to use surrogate measures such as success, grades, results of assessment protocols, and student testimony about their learning experiences. Whether or not such devices are valid indicators remains to be determined. We may be well served, however, by changing our mode of inquiry to blended teaching.

Additionally, as Norberg ( 2017 ) points out, blended learning is not new. The modality dates back, at least, to the medieval period when the technology of textbooks was introduced into the classroom where, traditionally, the professor read to the students from the only existing manuscript. Certainly, like modern technologies, books were disruptive because they altered the teaching and learning paradigm. Blended learning might be considered what Johnson describes as a slow hunch (2010). That is, an idea that evolved over a long period of time, achieving what Kaufmann ( 2000 ) describes as the adjacent possible – a realistic next step occurring in many iterations.

The search for a definition for blended learning has been productive, challenging, and, at times, daunting. The definitional continuum is constrained by Oliver and Trigwell ( 2005 ) castigation of the concept for its imprecise vagueness to Sharpe et al.’s ( 2006 ) notion that its definitional latitude enhances contextual relevance. Both extremes alter boundaries such as time, place, presence, learning hierarchies, and space. The disagreement leads us to conclude that Lakoff’s ( 2012 ) idealized cognitive models i.e. arbitrarily derived concepts (of which blended learning might be one) are necessary if we are to function effectively. However, the strong possibility exists that blended learning, like quality, is observer dependent and may not exist outside of our perceptions of the concept. This, of course, circles back to the problem of assuming that blending is a treatment effect for point hypothesis testing and meta-analysis.

Ultimately, in this article, we have tried to consider theoretical concepts and empirical findings about blended learning and their relationship to the new normal as it evolves. Unfortunately, like unresolved chaotic solutions, we cannot be sure that there is an attractor or that it will be the new normal. That being said, it seems clear that blended learning is the harbinger of substantial change in higher education and will become equally impactful in K-12 schooling and industrial training. Blended learning, because of its flexibility, allows us to maximize many positive education functions. If Floridi ( 2014 ) is correct and we are about to live in an environment where we are on the communication loop rather than in it, our educational future is about to change. However, if our results are correct and not over fit to the University of Central Florida and our theoretical speculations have some validity, the future of blended learning should encourage us about the coming changes.

Adams Becker, S., Cummins, M., Davis, A., Freeman, A., Hall Giesinger, C., & Ananthanarayanan, V. (2017). NMC horizon report: 2017 higher Education Edition . Austin: The New Media Consortium.

Google Scholar  

Alhabeeb, A. M. (2015). The quality assessment of the services offered to the students of the College of Education at King Saud University using (SERVQUAL) method. Journal of Education and Practice , 6 (30), 82–93.

Allen, I. E., & Seaman, J. (2003). Sizing the opportunity: The quality and extent of online education in the United States, 2002 and 2003. Retrieved from http://files.eric.ed.gov/fulltext/ED530060.pdf

Allen, I. E., Seaman, J., Poulin, R., & Straut, T. T. (2016). Online report card: Tracking online education in the United States, 1–4. Retrieved from http://onlinelearningsurvey.com/reports/onlinereportcard.pdf

Arum, R., Roksa, J., & Cook, A. (2016). Improving quality in American higher education: Learning outcomes and assessments for the 21st century . San Francisco: Jossey-Bass.

Aud, S., Hussar, W., Planty, M., Snyder, T., Bianco, K., Fox, M. A., & Drake, L. (2010). The condition of education - 2010. Education, 4–29. https://doi.org/10.1037/e492172006-019

Balfour, S. P. (2013). Assessing writing in MOOCs: Automated essay scoring and calibrated peer review. Research and Practice in Assessment , 2013 (8), 40–48.

Bayne, S., Evans, P., Ewins, R.,Knox, J., Lamb, J., McLeod, H., O’Shea, C., Ross, J., Sheail, P. & Sinclair, C, (2016) Manifesto for teaching online. Digital Education at Edinburg University. Retrieved from https://onlineteachingmanifesto.wordpress.com/the-text/

Bernard, R. M., Abrami, P. C., Borokhovski, E., Wade, C. A., Tamim, R. M., Surkes, M. A., & Bethel, E. C. (2009). A meta-analysis of three types of interaction treatments in distance education. Review of Educational Research , 79 (3), 1243–1289. https://doi.org/10.3102/0034654309333844 .

Article   Google Scholar  

Bernard, R. M., Borokhovski, E., Schmid, R. F., Tamim, R. M., & Abrami, P. C. (2014). A meta-analysis of blended learning and technology use in higher education: From the general to the applied. Journal of Computing in Higher Education , 26 (1), 87–122.

Bloemer, W., & Swan, K. (2015). Investigating informal blending at the University of Illinois Springfield. In A. G. Picciano, C. D. Dziuban, & C. R. Graham (Eds.), Blended learning: Research perspectives , (vol. 2, pp. 52–69). New York: Routledge.

Bonk, C. J., & Graham, C. R. (2007). The handbook of blended learning: Global perspectives, local designs . San Francisco: Pfeiffer.

Boring, A., Ottoboni, K., & Stark, P.B. (2016). Student evaluations of teaching (mostly) do not measure teaching effectiveness. EGERA.

Brieman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classification and regression trees . New York: Chapman & Hall.

California Community Colleges Chancellor’s Office. (2013). Distance education report.

Cobb, C., deNoyelles, A., & Lowe, D. (2012). Influence of reduced seat time on satisfaction and perception of course development goals: A case study in faculty development. The Journal of Asynchronous Learning , 16 (2), 85–98.

Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika , 16 (3), 297–334 Retrieved from http://psych.colorado.edu/~carey/courses/psyc5112/readings/alpha_cronbach.pdf .

Article   MATH   Google Scholar  

Dringus, L. P., and A. B. Seagull. 2015. A five-year study of sustaining blended learning initiatives to enhance academic engagement in computer and information sciences campus courses. In Blended learning: Research perspectives. Vol. 2. Edited by A. G. Picciano, C. D. Dziuban, and C. R. Graham, 122-140. New York: Routledge.

Dziuban, C. D., & Shirkey, E. C. (1974). When is a correlation matrix appropriate for factor analysis? Some decision rules. Psychological Bulletin , 81(6), 358. https://doi.org/10.1037/h0036316 .

Dziuban, C., Hartman, J., Cavanagh, T., & Moskal, P. (2011). Blended courses as drivers of institutional transformation. In A. Kitchenham (Ed.), Blended learning across disciplines: Models for implementation , (pp. 17–37). Hershey: IGI Global.

Chapter   Google Scholar  

Dziuban, C., & Moskal, P. (2011). A course is a course is a course: Factor invariance in student evaluation of online, blended and face-to-face learning environments. The Internet and Higher Education , 14 (4), 236–241.

Dziuban, C., Moskal, P., Hermsdorfer, A., DeCantis, G., Norberg, A., & Bradford, G., (2015) A deconstruction of blended learning. Presented at the 11 th annual Sloan-C blended learning conference and workshop

Dziuban, C., Picciano, A. G., Graham, C. R., & Moskal, P. D. (2016). Conducting research in online and blended learning environments: New pedagogical frontiers . New York: Routledge, Taylor & Francis Group.

Dziuban, C. D., Hartman, J. L., & Moskal, P. D. (2004). Blended learning. EDUCAUSE Research Bulletin , 7 , 1–12.

EDUCAUSE. (2017) 2017 key issues in teaching & learning. Retrieved from https://www.EDUCAUSE.edu/eli/initiatives/key-issues-in-teaching-and-learning

Fairlie, R. (2004). Race and the digital divide. The B.E. Journal of Economic Analysis & Policy , 3 (1). https://doi.org/10.2202/1538-0645.1263 .

Fischer, L., Hilton, J., Robinson, T. J., & Wiley, D. (2015). A Multi-institutional Study of the Impact of Open Textbook Adoption on the Learning Outcomes of Post-secondary Students . Journal of Computing in Higher Education. https://doi.org/10.1007/s12528-015-9101-x .

Floridi, L. (2008). A defence of informational structural realism. Synthese , 161 (2), 219–253.

Article   MathSciNet   Google Scholar  

Floridi, L. (2014). The 4th revolution: How the infosphere is reshaping human reality . Oxford: Oxford University Press.

Garrison, D. R., & Vaughan, N. D. (2013). Blended learning in higher education , (1st ed., ). San Francisco: Jossey-Bass Print.

Garrison, D. R., & Kanuka, H. (2004). Blended learning: Uncovering its transformative potential in higher education. The Internet and Higher Education , 7 , 95–105.

Goodhart, C.A.E. (1975). “Problems of monetary management: The U.K. experience.” Papers in Monetary Economics. Reserve Bank of Australia. I.

Graham, C. R. (2013). Emerging practice and research in blended learning. In M. G. Moore (Ed.), Handbook of distance education , (3rd ed., pp. 333–350). New York: Routledge.

Guttman, L. (1953). Image theory for the structure of quantitative variates. Psychometrika , 18 , 277–296.

Article   MathSciNet   MATH   Google Scholar  

Hilton, J., Fischer, L., Wiley, D., & Williams, L. (2016). Maintaining momentum toward graduation: OER and the course throughput rate. International Review of Research in Open and Distance Learning , 17 (6) https://doi.org/10.19173/irrodl.v17i6.2686 .

IBM Corp. Released (2015). IBM SPSS statistics for windows, version 23.0 . Armonk: IBM Corp.

Jean-François, E. (2013). Transcultural blended learning and teaching in postsecondary education . Hershey: Information Science Reference.

Book   Google Scholar  

Jones, S., Johnson-Yale, C., Millermaier, S., & Pérez, F. S. (2009). U.S. college students’ internet use: Race, gender and digital divides. Journal of Computer-Mediated Communication , 14 (2), 244–264 https://doi.org/10.1111/j.1083-6101.2009.01439.x .

Kaiser, H. F., & Rice, J. (1974). Little Jiffy, Mark IV. Journal of Educational and Psychological Measurement , 34(1), 111–117.

Kaufmann, S. (2000). Investigations . New York: Oxford University Press.

Kitchenham, A. (2011). Blended learning across disciplines: Models for implementation . Hershey: Information Science Reference.

Lakoff, G. (2012). Women, fire, and dangerous things: What categories reveal about the mind . Chicago: The University of Chicago Press.

Lewis, L., & Parsad, B. (2008). Distance education at degree-granting postsecondary institutions : 2006–07 (NCES 2009–044) . Washington: Retrieved from http://nces.ed.gov/pubs2009/2009044.pdf .

Liu, F., & Cavanaugh, C. (2011). High enrollment course success factors in virtual school: Factors influencing student academic achievement. International Journal on E-Learning , 10 (4), 393–418.

Lowe, D. (2013). Roadmap of a blended learning model for online faculty development. Invited feature article in Distance Education Report , 17 (6), 1–7.

Means, B., Toyama, Y., Murphy, R., & Baki, M. (2013). The effectiveness of online and blended learning: A meta-analysis of the empirical literature. Teachers College Record , 115 (3), 1–47.

Means, B., Toyama, Y., Murphy, R., Kaia, M., & Jones, K. (2010). Evaluation of evidence-based practices in online learning . Washington: US Department of Education.

Moskal, P., Dziuban, C., & Hartman, J. (2013). Blended learning: A dangerous idea? The Internet and Higher Education , 18 , 15–23.

Norberg, A. (2017). From blended learning to learning onlife: ICTs, time and access in higher education (Doctoral dissertation, Umeå University).

Norberg, A., Dziuban, C. D., & Moskal, P. D. (2011). A time-based blended learning model. On the Horizon , 19 (3), 207–216. https://doi.org/10.1108/10748121111163913 .

Oliver, M., & Trigwell, K. (2005). Can ‘blended learning’ be redeemed? e-Learning , 2 (1), 17–25.

Olshen, Stone , Steinberg , and Colla (1995). CART classification and regression trees. Tree-structured nonparametric data analysis. Statistical algorithms. Salford systems interface and documentation. Salford Systems .

O'Neil, C. (2017). Weapons of math destruction: How big data increases inequality and threatens democracy . Broadway Books.

Online Learning Consortium. The OLC quality scorecard for blended learning programs. Retrieved from https://onlinelearningconsortium.org/consult/olc-quality-scorecard-blended-learning-programs/

Open SUNY. The OSCQR course design review scorecard. Retrieved from https://onlinelearningconsortium.org/consult/oscqr-course-design-review/

Picciano, A. G. (2009). Blending with purpose: The multimodal model. Journal of Asynchronous Learning Networks , 13 (1), 7–18.

Picciano, A. G., Dziuban, C., & Graham, C. R. (2014). Blended learning: Research perspectives , (vol. 2). New York: Routledge.

Picciano, A. G., & Dziuban, C. D. (2007). Blended learning: Research perspectives . Needham: The Sloan Consortium.

Pirsig, R. M. (1974). Zen and the art of motorcycle maintenance: An inquiry into values . New York: Morrow.

Quality Matters. (2016). About Quality Matters. Retrieved from https://www.qualitymatters.org/research

Robinson, T. J., Fischer, L., Wiley, D. A., & Hilton, J. (2014). The Impact of Open Textbooks on Secondary Science Learning Outcomes . Educational Researcher. https://doi.org/10.3102/0013189X14550275 .

Ross, B., & Gage, K. (2006). Global perspectives on blended learning: Insight from WebCT and our customers in higher education. In C. J. Bonk, & C. R. Graham (Eds.), Handbook of blended learning: Global perspectives, local designs , (pp. 155–168). San Francisco: Pfeiffer.

Rovai, A. P., & Jordan, H. M. (2004). Blended learning and sense of community: A comparative analysis with traditional and fully online graduate courses. International Review of Research in Open and Distance Learning , 5 (2), 1–13.

Searle, J. R. (2015). Seeing things as they are: A theory of perception . Chicago: Oxford University Press.

Sharpe, R., Benfield, G., Roberts, G., & Francis, R. (2006). The undergraduate experience of blended learning: A review of UK literature and research. The Higher Education Academy, (October 2006).

Shea, P., & Bidjerano, T. (2014). Does online learning impede degree completion? A national study of community college students. Computers and Education , 75 , 103–111 https://doi.org/10.1016/j.compedu.2014.02.009 .

Shea, P., & Bidjerano, T. (2016). A National Study of differences between distance and non-distance community college students in time to first associate degree attainment, transfer, and dropout. Online Learning , 20 (3), 14–15.

Sitzmann, T., Kraiger, K., Stewart, D., & Wisher, R. (2006). The comparative effectiveness of web-based and classroom instruction: A meta-analysis. Personnel Psychology , 59 (3), 623–664.

Smith, L. A. (2007). Chaos: a very short introduction . Oxford: Oxford University Press.

Star, S. L., & Griesemer, J. R. (1989). Institutional ecology, translations and boundary objects: Amatuers and professionals in Berkely’s Museum of Vertebrate Zoology, 1907-39. Social Studies of Science , 19 (3), 387–420.

Stark, P. & Freishtat, R. (2014). An evaluation of course evaluations. ScienceOpen. Retrieved from https://www.stat.berkeley.edu/~stark/Preprints/evaluations14.pdf .

Tynan, B., Ryan, Y., & Lamont-Mills, A. (2015). Examining workload models in online and blended teaching. British Journal of Educational Technology , 46 (1), 5–15.

Uttl, B., White, C. A., & Gonzalez, D. W. (2016). Meta-analysis of faculty’s teaching effectiveness: Student evaluation of teaching ratings and student learning are not related. Studies in Educational Evaluation , 54 , 22–42.

Williams, J. (2016). College and the new class divide. Inside Higher Ed July 11, 2016.

Wladis, C., Hachey, A. C., & Conway, K. (2015). Which STEM majors enroll in online courses, and why should we care? The impact of ethnicity, gender, and non-traditional student characteristics. Computers and Education , 87 , 285–308 https://doi.org/10.1016/j.compedu.2015.06.010 .

Zhao, Y., Lei, J., Yan, B., Lai, C., & Tan, H. S. (2005). What makes the difference? A practical analysis of research on the effectiveness of distance education. Teachers College Record , 107 (8), 1836–1884. https://doi.org/10.1111/j.1467-9620.2005.00544.x .

Download references

Acknowledgements

The authors acknowledge the contributions of several investigators and course developers from the Center for Distributed Learning at the University of Central Florida, the McKay School of Education at Brigham Young University, and Scholars at Umea University, Sweden. These professionals contributed theoretical and practical ideas to this research project and carefully reviewed earlier versions of this manuscript. The Authors gratefully acknowledge their support and assistance.

Author information

Authors and affiliations.

University of Central Florida, Orlando, Florida, USA

Charles Dziuban, Patsy D. Moskal & Nicole Sicilia

Brigham Young University, Provo, Utah, USA

Charles R. Graham

Campus Skellefteå, Skellefteå, Sweden

Anders Norberg

You can also search for this author in PubMed   Google Scholar

Contributions

The Authors of this article are listed in alphabetical order indicating equal contribution to this article. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Patsy D. Moskal .

Ethics declarations

Competing interests.

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Student Perception of Instruction

Instructions: Please answer each question based on your current class experience. You can provide additional information where indicated.

All responses are anonymous. Responses to these questions are important to help improve the course and how it is taught. Results may be used in personnel decisions. The results will be shared with the instructor after the semester is over.

Please rate the instructor’s effectiveness in the following areas:

Organizing the course:

Excellent b) Very Good c) Good d) Fair e) Poor

Explaining course requirements, grading criteria, and expectations:

Communicating ideas and/or information:

Showing respect and concern for students:

Stimulating interest in the course:

Creating an environment that helps students learn:

Giving useful feedback on course performance:

Helping students achieve course objectives:

Overall, the effectiveness of the instructor in this course was:

What did you like best about the course and/or how the instructor taught it?

What suggestions do you have for improving the course and/or how the instructor taught it?

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Cite this article.

Dziuban, C., Graham, C.R., Moskal, P.D. et al. Blended learning: the new normal and emerging technologies. Int J Educ Technol High Educ 15 , 3 (2018). https://doi.org/10.1186/s41239-017-0087-5

Download citation

Received : 09 October 2017

Accepted : 20 December 2017

Published : 15 February 2018

DOI : https://doi.org/10.1186/s41239-017-0087-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Blended learning
• Higher education
• Student success
• Student perception of instruction

Readiness of teachers for blended learning: A scale development study

• Open access
• Published: 29 May 2024

Cite this article

You have full access to this open access article

• Gülsemin Durmuş Çemçem   ORCID: orcid.org/0000-0001-6894-1489 1 ,
• Özgen Korkmaz   ORCID: orcid.org/0000-0003-4359-5692 2 &
• Volkan Kukul   ORCID: orcid.org/0000-0002-9546-3790 3  

The aim of this study is to create a new scale to assess teachers' readiness for blended learning. There are 317 active teachers volunteering in the study from various educational levels. Exploratory factor analysis was carried out to examine the construct validity of the scale with the data obtained. Following principal component analysis, 6 items were removed from the scale as they spread to different factors, and the remaining 25 items were refactored and grouped into 4 factors. Confirmatory factor analysis was carried out to confirm the factor structures of the scale. The obtained model confirmed the factor structure created in the exploratory factor analysis. Differences between the bottom and top 27% groups were investigated to evaluate item discriminability. For reliability analyses, internal consistency coefficients and stability analyses were performed. A five-point Likert scale with 25 items is used to assess how prepared the teachers are for blended learning. The items are categorized into four factors. The scale's Cronbach alpha coefficient value is 0.943 and McDonald’s ω value is .942. Analyses demonstrate that the scale is a valid and reliable tool for assessing teachers' readiness for blended learning.

Avoid common mistakes on your manuscript.

1 Introduction

Technologies designed to meet different needs are now widely used in education and training. Blended learning (BL) provides the use of methods and techniques that can meet individual learning needs by combining face-to-face learning methods with online distance learning systems (Osguthorpe & Graham, 2003 ). New developments in technology are best practices for learning beyond traditional boundaries to optimize learning outcomes (Singh & Reed, 2001 ). According to Graham ( 2006 ); four factors, which are time, space, content, and the human-machine ratio, affect traditional face to face and computer-assisted distance learning environments. The main purpose was to create the ideal combination through this interaction. In this regard, different BL models have emerged (Graham, 2006 ; Hannon & Macken, 2014 ; Khan, 2005 ; Staker & Horn, 2012 ; Teach Thought Staff, 2019 ). The main purpose of creating these models is to offer different learning activities and different designs so that learners can achieve maximum learning outcomes. BL requires us to consider the features of digital technologies and information and communication technologies in general (Dziuban et al., 2018 ). It is expected that the term BL will become the most common umbrella term to describe the technology usage for educational purposes and this approach will be widely adopted and accepted as the 'new normal' in education due to its widely accepted benefits (Hrastinski, 2019 ). Graham ( 2006 ) suggests that BL can be practiced at all levels of educational process. Additionally, the fact that blended learning provides personalised, flexible and effective learning experiences provides important educational opportunities for the differentiated needs of students with special educational needs (Schenk, 2023 ). In blended learning environments, special education teachers provide students with both personalised and individually paced learning opportunities through virtual learning resources and guidance through face-to-face teacher communication, aiming for maximum efficiency in achieving instructional goals (Rivera, 2017 ; Zavaraki & Schneider, 2019 ).

Wilson and Smilanich ( 2005 ) identified the steps to be followed in designing and implementing blended curricula as 1) identifying the needs, 2) determining the objectives of the program, 3) designing the blended program, 4) creating and coordinating individual learning solutions, 5) implementing the blended program, 6) measuring the results of the program. In step 3 of these designs, the design of the blended program, they stated that there should be a basic definition of each training, factors such as the profile of the learner when choosing the training solutions, the characteristics of the learning content, and a template to help summarize the training program.

Many international and national educational institutions and organizations have incorporated online applications into their training programs, owe to the widespread use of digital technologies. The online and BL guide published by the International Baccalaureate provides examples of content design and assessment steps (Tonbuloğlu & Tonbuloğlu, 2021 ). Cambridge ( 2020 ) organized a study on BL models by bringing together technology experts, education experts and government representatives to discuss the structuring of future education models. In addition, the Ministry of National Education of Türkiye included the qualities of BL in the 2023 Education Vision document at the K-12 level (MEB, 2018 ).

Teachers need to use educational methods and techniques in accordance with the requirements of the age to be effective and efficient in education. In this direction, in order to fulfil the teaching profession effectively and efficiently in accordance with the requirements of the teaching profession, the Ministry of National Education of Türkiye has determined the general competencies of the teaching profession as "personal and professional values - professional development, student recognition, teaching and learning process, teaching and learning process, monitoring and evaluating learning development, school, family and community relations, program and content knowledge" (MEB, 2017 ). In the legislative summaries listed under the title "Education, training and youth", which is one of the priorities of the political and policy agenda of the European Union for the years 2019-2024, it is aimed to achieve the goals set under the scope of education and training framework, such as increasing the quality and efficiency of education, developing creativity, innovation, and entrepreneurship (European Union, n.d. ). In the world and in Türkiye, efforts are being made to provide efficient and effective educational opportunities through new regulations and training in line with the necessities of the age in education. In this regard, the readiness of teachers for modern methods of education and training is important. BL is a learning method that is used to provide the best learning in accordance with the requirements of the age. Therefore, determining teachers' readiness for this learning method is considered important in terms of organizing and implementing educational activities.

This study aims to fill the gap in the literature by measuring teachers' readiness for blended teaching, thus contributing significantly to the field. The identification of teachers' readiness for blended teaching is a crucial aspect of this study.

Cabı and Gülbahar ( 2013 ) measured the effectiveness of blended learning environments. Koç ( 2019 ) created the interaction value scale for blended learning environments, and Mıhçı Türker and Öztürk ( 2022 ) adapted the blended teaching readiness scale for pre-service teachers in Turkish. Tang and Chaw ( 2013 ) conducted a study on university students' attitudes towards blended learning. Yıldız Durak ( 2017 ) adapted the flipped learning readiness scale for Turkish secondary school students. Shakeel et al. ( 2023 ) developed a blended learning readiness scale for university students. Los et al. ( 2021 ) developed the Online and Blended Teaching Readiness Assessment (OBTRA) study for post-secondary teachers. It is important to note that the OBTRA scale measures different characteristics than those included in K-12 blended teaching preparation, as observed by Graham et al. ( 2019 ). The OBTRA scale defines the knowledge, skills, and attitudes necessary for teachers to be effective in blended learning environments. No existing scales in the literature are specifically designed to measure the readiness levels of teachers at all levels of education for developing, implementing, and evaluating blended learning practices. However, it is important to note that measuring teachers' readiness for this innovative learning model is a critical factor in supporting technological transformation in education. Graham et al. ( 2019 ) confidently proposed the development of a more concise and current scale to aid teachers in their preparation for blended teaching. The current scale is deemed impractical due to its excessive number of items, and the high correlation between its second-order factors is seen as a limitation. The new scale is expected to be a significant improvement and better serve the needs of educators. This study will measure the competencies of teachers at all levels of education in accordance with current requirements, as well as their readiness to develop, implement, and evaluate blended learning applications. The results will demonstrate the high level of expertise and capability of teachers in meeting the demands of modern education.

2 Literature review

2.1 blended learning and blended learning models.

It is possible to use online and face to face learning activities together in a BL environment, according to Gülbahar et al. ( 2020 ). The definitions of BL in the literature vary widely. According to Eastman ( 2015 ) BL is a complete set of educational practices that brings together the face-to-face instruction with computer-based learning tools that may be customized and driven by the learner. BL, according to Hockly ( 2018 ), combines the use of online and offline computer technology with face-to-face instruction. BL is identified as a design of modern teaching process by combining distance learning systems and traditional learning processes together (Tonbuloğlu & Tonbuloğlu, 2021 ). According to Horn and Staker ( 2017 ), it is a formal education program where the student is away from home and has at least some elements of time, place, road, and speed control. According to the Oxford Dictionaries definition ( 2023 ); it is a cost-effective method of education that combines the subject to be taught in the classroom with the use of various technologies, including online learning. According to Graham ( 2006 ), it is a type of education that mixes face-to-face instruction with online learning platforms. The definitions in the literature all agree on one thing: BL strives to combine the best elements of the harmony between face-to-face instruction and digital technologies for student-centered learning.

Various strategies have been created to promote efficient and effective learning in a BL method. As shown in Fig. 1 , BL grouped into four separate methods by Horn and Staker ( 2015 ). These models are the rotation, the flexible, the personal blended, and the enriched virtual models. The rotation model is also constituted by the Station, laboratory, flipped classroom, and individual rotation models.

BL models (Horn & Staker, 2015 )

BL systems were categorized into various types by Graham ( 2006 ). These categories are enabling blending, enhancing blending, and altering blending. Online and offline learning, self-paced and collaborative learning, scheduled and unstructured learning, and tailored and off-the-shelf content are all examples of BL, according to Khan ( 2005 ). Three BL methods were illustrated by Hannon and Macken ( 2014 ). These concepts include blended block, blended presentation and interactivity, and entirely online. Interactive blended sessions using online resources, training, seminars, and presentations are examples of blended presentation and interaction. The blended block consists of weekly online training, in-person intensive workshops, and internet resources. The fully online model simulates interaction with online resources and learning activities, including asynchronous online lessons, online group projects, discussion forums, and podcasts of brief courses.

Six common BL models are station rotation, laboratory rotation, distance (also known as enriched virtual), flexible, flipped classroom, and individual rotation, according to Teach Thought Staff ( 2019 ). Project-based, self-directed, internal-external, completed, mastery-based, and various options are some less well-known BL models. A BL model that classifies K–12 BL programs was proposed by Staker and Horn ( 2012 ). This model can be divided into four categories: 1) the rotational model, where students alternate between online and traditional learning methods; 2) the flexible model, where content is initially made available online and students advance in accordance with a schedule customized for them; 3) the self-blending model, where students take one or more online courses to finish traditional courses; and 4) the enriched virtual model, where students split their time between face-to-face and online activities. The main goal in developing these models is to provide a variety of designs with a variety of learning activities so that students can attain the best learning results possible.

3 Methodology

3.1 research design.

The descriptive survey model was used in this study to develop the teachers' readiness scale for BL. The descriptive survey model was used to attempt to describe the validity and reliability of the scale. Karasar ( 2007 ) defines the descriptive survey model as a model that aims to determine the group traits as they are.

3.2 Participants

The research study included 317 teachers from different branches who were actively working in different provinces of Türkiye in 2022-2023. The study group was determined using the convenient sampling method from the non-probability sampling method. The convenience sampling method considers the suitability and willingness of participants to take part in the study. This method accelerates the study as it is easily accessible (Creswell, 2012 /2020: 193). Research participants are 317 teachers. The distribution of the study group by gender, educational status, branch, seniority, teaching level and foreign language level is shown in Table 1 .

3.3 Constituting the item pool

The researchers reviewed the relevant literature and analyzed the scales. When the studies for educators were reviewed (Fidan et al., 2020 ; Korkmaz et al., 2019 ; Şad et al., 2016 ; Toker et al., 2021 ; Yörük & Özçetin, 2021 ), it was found that there are studies on digital literacy of teachers, online learning, or development of digital teaching materials. There are also some studies on BL environments (Cabı & Gülbahar, 2013 ; Koç, 2019 ; Mıhçı Türker & Öztürk, 2022 ; Tang & Chaw, 2013 ). The studies on the readiness for BL with students or teachers at different educational levels were found (Graham et al., 2019 ; Shakeel et al., 2023 ; Yıldız Durak, 2017 ). In this direction, the literature was reviewed, and BL readiness items were written for testing purposes. While writing the items, the item "I have the necessary knowledge to use BL environments" from the scale developed by Koç ( 2019 ) was adapted as "I have the competence to use the BL method" and the item "I have the necessary technological facilities to use BL environments" was adapted and added to the item pool as "I have the necessary technological facilities to use the BL method". After the items were formed, two professors and one associate professor, who have expertise in computer and instructional technologies and have studied on blended learning and distance education, were consulted for content validity. In the prepared form, the experts were requested to evaluate each item using one of the following options "appropriate, should be corrected, should be removed" and to write an explanation for the items they did not find appropriate. The experts' views are shown in Table 2 .

In agreement with expert views, some items were decided to be corrected. Two items were decided to be removed from the scale. In the item pool of the Teachers' Readiness for BL scale, which was prepared in result of the studies, there are 6 items in the design factor, 7 items in the learning-teaching process factor, 11 items in the competence factor, and 7 items in the management factor. The whole scale item pool was determined to be 31 items. In the items, options were given according to the five-point scoring system to measure the levels. These options were arranged as "(1) Strongly Disagree, (2) Disagree, (3) Partially Agree, (4) Agree, (5) Strongly Agree" and scored accordingly.

3.4 Data analyses

KMO and Bartlett analyses were first used for assessing if factor analysis (FA), based on statistical analyses, could be used to identify the scale's construct validity in the data obtained using the scale. The KMO value should be greater than .60 and the value close to 1 demonstrate that the data set is at a sufficient level to perform FA to be able to perform factorization (Büyüköztürk, 2002 ; Field, 2013 ; Russell, 2002 ). Şencan ( 2005 ) formed a KMO value range table to grade the meaning of the values formed in the KMO and Bartlett analysis results and according to this table, while the range of 0.8-0.9 is considered as a good value, values above 0.9 are expressed as excellent level. The values in Bartlett's test demonstrate that the null hypothesis is not accepted at 0.05 level. (Büyüköztürk, 2002 ).

Both exploratory and confirmatory factor analyses were conducted. The scale was factorized using principal component analysis. Factor loadings (FL) were assessed using varimax vertical rotation. The results of the FA required that items with FL below 0.40 and items with a .10 difference between the loadings on two factors were excluded from the analysis. It is expected that the loadings between two factors should be as high as possible and it is recommended that the difference between these values should be at least .10 (Büyüköztürk, 2002 ; Howard, 2016 ). Hinkin (1995) stated that the generally acceptable factor loading in FA studies is 0.40, but items with FL up to 0.30 can also be included in the scales. On the other hand, FL above 0.50 are also considered to be quite good (Büyüköztürk, 2002 ). According to the results of the analyses, the lowest factor loading was calculated as .587 and the highest factor loading was calculated as .834. Accordingly, all the items had excellent FL (Çokluk et al., 2010 ).

According to the completed items and factor structures produced by the exploratory FA, confirmatory FA was performed. To ascertain whether the chosen measurement models are supported by the data, confirmatory FA technique is the preferable analysis technique (Byrne, 2016 ; Gürbüz, 2021 ).

3.5 Ethical authorization of research

The study collected data from teachers who are adult individuals teaching in educational institutions in different cities across Türkiye. As participation was voluntary, no approval from any institution or organization was necessary.

3.6 Findings

3.6.1 findings related to the validity of the scale.

For the validity of the Teachers' Readiness for BL scale, the construct validity analysis, item-total correlation analysis and item discrimination analysis were held, and the findings are shown below.

Findings related to exploratory FA

To determine the construct validity of the Teachers' Readiness for BL scale, Kaiser-Meyer-Oklin (KMO) and Bartlett tests were performed with the data obtained and KMO=0.951; Bartlett test value was determined as χ2=6930.134; sd=465 ( p =0.000). These values showed that FA could be applicable for the 31-item scale.

Firstly, the dimensions of the scale were defined using principal component analysis. The varimax vertical rotation technique was used for this analysis. The results of the analysis show that 6 items whose item loadings were distributed across different factors were excluded from the scale, and then FA was performed again on the remaining 25 items. The Teachers' Readiness for BL scale covers teachers' ability to improve, implement and evaluate BL practices. As a result of the analyses, the final 25 items of the scale were distributed across four factors. In this final version it was concluded that the KMO value of the 25-item scale was 0.943; Bartlett value χ2=5306.341; sd=300; p <0.000. The FL of the items without rotation were found between .515 and .809; the FL of the items rotated using the varimax vertical rotation technique were found between .623 and .834. Table 3 shows the results of the factor analysis conducted on 25 items, indicating the variances explained by the components.

Table 3 shows the factor loadings for each of the four factors. The first factor had a loading of 22.65, the second factor had a loading of 19.86, the third factor had a loading of 16.29, and the fourth factor had a loading of 7.48. The scale's items and factors accounted for 66.30% of the total variance. The factors resulting from the analyses were found to be consistent with the sub-skills identified during the creation of the item pool. However, items created for the identified skills were also distributed across different factors. Specifically, three items originally intended for the competence factor, which was supposed to have 11 items according to the item pool, were instead placed under the teaching and learning process factor. The study collected 5 items under the 'Designing' factor, 10 items under the 'Teaching and Learning Process' factor, 3 items under the 'Competence' factor, and 7 items under the 'Management' factor. The scree accumulation graph used to determine the final number of factors (Fig. 2 ) indicates that the scale has 4 factors.

The results of the analyses regarding the item loadings of the remaining 25 items in terms of factors, the factors' eigenvalues and the variance amount explanations are shown in Table 4 .

As shown in Table 4 , there are 10 items in the teaching and the process of learning factor of the scale, and the FL range from .587 to .760. While the TLP factor’s eigenvalue in whole scale is 11.53, its total effect to variance is 46.1%. There are 7 items in the management factor. The FL of these items ranges from .623 to .834. While factor’s eigen value in the whole scale is 1.92, its effect to the total variance is 7.7%. There are 5 items in the "Design" factor. The FL of the items ranges from .634 to .786. While factor’s eigen value in the whole scale is 1.82, its effect to the total variance is 7.3%. There are 3 items in the factor "Competence". The FL of the items range between .738 and .785. While the factor’s eigenvalue in the whole scale is 1.29, its effect to the total variance is 5.1. As all items in this factor are inverted, they are to be reverse coded.

Findings on confirmatory FA

For confirming the scale's factor structure, the available data was used in a confirmatory FA, which revealed that the scale was grouped into 4 factors in result of the exploratory FA. The result of the confirmatory FA performed with the maximum likelihood technique demonstrate that the value of the 3 items of competence was far from .70. However, it was not removed from the scale on the grounds that there were 3 items in the competence factor and therefore it might negatively affect the content validity. The estimate values of the items were found between .525 and .865.

When the goodness of fit values were analyzed; it is evident that χ2(sd=265, N=325)= 5632.871, p <.0001, CMIN/DF=2.247, RMSEA= 0.064, RMR= 0.031, SRMR=0.457, GFI= 0.868, AGFI= 0.838, CFI= 0.938 and IFI= 0.938. These values show that the goodness of fit indices is compatible at a reasonable level according to the threshold values (Gürbüz, 2021 ). The obtained model reveals that it confirms the factor structure created in the exploratory FA. In this direction, scale’s factorial model is shown in Fig. 3 .

Confirmatory FA diagram of the scale

3.7 Item-factor correlation values

In this part, item-factor correlation values were calculated from the scores gained from the items of the scale and four factors. The level of conformity of each item to the overall purpose was tested. Table 5 shows the item-factor correlation values for each of the items.

As shown in Table 5 item-test correlation coefficients ranged from .715 to .844 for the TLP, between .779 and .874, for the management, from .824 to .879 for the design, and from .709 to .821 for the competence factor. All items show a positive and significant relationship with all factors ( p <0.000). In this regard, each item can be stated to serve both the factor in which it is located and the overall goal of scale.

3.8 Item discrimination

The item scores were ranked from higher to lower and 84 lower and upper groups were formed, including the bottom 27% and the top 27% groups to calculate discrimination power of the scale items. Independent groups t-test values were measured in line with the total scores in lower and upper 27%. Table 6 shows the results of the t-tests and the levels of significance of the discrimination powers.

As shown in Table 6 , the independent sample t-test values for the 25 items, factors and total score in the scale were found to be from 3.614 to 16.175. The t-value for the overall scale was 29.480. The level of each of the differences determined because of the analysis is significant ( p <0.001). According to these values, it is evident that overall scale and each scale item have high discrimination. However, the discrimination level of the competence factor was found to be lower than the other factors.

3.9 Findings on the reliability of the scale

Internal consistency and consistency analyses were held to calculate the scale’s reliability. The findings reached because of the analyses are shown below.

3.10 Internal consistency level

Reliability analyses for the whole scale and its factors were calculated using Cronbach's Alpha and McDonald's ω reliability coefficient. Reliability analysis values for each of the factors and the overall scale are given in Table 7

According to the values shown in Table 7 , the Cronbach's Alpha value of the scale, which includes four factors and a total of 25 items, is .943 and McDonald's ω value is .942. The results of the analyses according to the factors showed that Cronbach's Alpha values ranged between .664 and 928 and McDonald's ω value ranged between .680 and .928. Based on this, it can be stated that consistent measurements were made for the overall scale and each factor.

3.11 Consistency level

The scale's stability has been calculated with test-retest method. Scale’s final version which consists of 25 items was performed to the 21 participants after four weeks. The relationship between the scores gained in result of the two applications was analyzed both for all items as a whole and for each item. Thus, the performance of both the scale as a whole and each item within the scale was analyzed for stable measurement. The findings related to these analyses are shown in Table 8 .

Table 8 demonstrate that the scale items’ correlation coefficients obtained by test-retest method range from .007 to .658. It was seen that three items under the TLP factor, three items under the management factor, one item under the design factor had low correlation level and the correlation coefficient of the sixth item under the management factor was .007 and there was no significant relationship. The other eighteen items were found to have a moderate correlation. The correlation coefficients for the factors ranged between .444 and .616, and the correlation for the total score was .657. Although the stability level of the sixth item under the management factor was low, since the overall stability level of the factor was high, this item was not extracted from the scale not to disturb the content validity of the scale. It can be said that factors’ stability levels and the total score are high.

4 Result and discussion

In the study, a new scale was developed for defining teachers’ readiness levels on BL. The BL readiness scale consists of four factors and 25 items determined as a five-point Likert-type scale. The items in the scale were scaled as (1) Strongly Disagree, (2) Disagree, (3) Partially Agree, (4) Agree, (5) Strongly Agree. The validity of the scale was tested through FA and discriminant properties.

According to factor items’ FL, the factors’ eigenvalues and the variance ratios explained, it can be stated that the scale’s construct validity is appropriate. Exploratory FA demonstrate that the scale was grouped under 4 factors. For validating the determined factor structures, confirmatory FA was used. The results of the confirmatory FA demonstrate that scale model’s observed values revealed the consistency of data at a valid level.

Item-total correlations were calculated to determine the level at which the items in the scale can measure the factors they are included in and the characteristics that are tried to be measured. According to the results, the items and factors in the scale contributed significantly to the purpose of measuring the characteristics that the scale was intended to measure. Additionally, the discrimination levels of the items were determined by analyzing the t-values for the difference between the top 27% groups and the bottom 27% groups. According to the values obtained, the overall scale and the discrimination of each item were found high. However, the level of discrimination of the Competence factor was found lower than that of the other factors. The internal consistency coefficients of the scale were obtained by means of Cronbach's alpha and McDonald’s ω. The obtained values demonstrate that the total and each factor of the scale can provide consistent measurements. The test-retest method was used at different times to test the invariance of the scale over time. According to the correlation of the total score obtained from the scale, it was concluded that it could make consistent measurements with respect to time invariance.

The scale items were formed within the framework of a four-factor structure. Construct validity analyses shows that a four-factor structure emerged in a way to confirm this four-factor structure. These factors are TLP, management, designing and competence. According to the construct validity analyses, it was seen that three items under the competence factor were distributed under the TLP factor. It appears that competences are a concept used to indicate what teachers and prospective teachers can do and this concept is expressed as standards in English-speaking countries (Alan & Güven, 2022 ). In this direction, it was decided to maintain the four-factor structure consisting of TLP, management, designing and competence, which was formed by considering that it would not be wrong to evaluate the competence items within the TLP. The four-factor structure can be explained as follows:

Teaching and learning process

It is the process of including the knowledge, skills, behaviors, and attitudes aimed to be gained by the students in the programs by considering the characteristics of the target group (Gökçe, 2014 ). There are two different conceptions of teaching and learning in education: traditional and constructivist (Chan & Elliott, 2004 ). Because of these understandings, the TLP should be shaped in line with the characteristics of the target group. The quality of teaching is improved by creating a technologically rich TLP (Horton & Horton, 2003 ). Therefore, the efficiency of TLP is one of the important factors affecting the quality of education.

In blended teaching processes, teachers are an important part of the process (Bonk & Graham, 2004 ). From the learner's point of view, it can be seen as a challenging process to manage and take responsibility for one's own work in a flexible working environment independent of time and space in BL (Vaughan, 2007 ; Welker & Berardino, 2005 ). Managing a relational process with learners is important for the effectiveness and efficiency of the course (Moore & Kearsley, 2004 cited in Göksel, 2015 ). For these reasons, the most efficient learning and teaching in BL environments is possible with the correct management of the process.

Designing the instruction is among the most important factors that determine the efficiency of the teaching process. Designing online and face-to-face environments together in the BL process is known as an important factor (Cabı & Gülbahar, 2013 ). Fresen ( 2007 ) stated that instructional design factor is among the success factors of web supported learning.

Proficiency

Processes involve a process with different variables. In this direction, technology competences of instructors and their online environment experiences are important factors affecting the quality of learning environments (Wheeler, 2001 ). Bandura ( 1997 ) states that teachers' perceptions of efficacy are more important than professional content knowledge and teaching efficacy.

Therefore, it can be concluded that a valid and reliable scale exists for determining teachers' readiness levels for blended learning. The scale's results allow teachers to identify areas where they need improvement, and educational institutions can plan effective support and training programs to address these areas of weakness. When teachers face obstacles in adopting the blended learning method due to a lack of technological competencies, application difficulties, or insufficient institutional support, continuous training programs and strengthened technological infrastructure can be helpful. As no measurement tool has been found in the related literature to assess the readiness levels of teachers at all educational levels for creating, implementing, and evaluating blended learning applications, it is believed that the developed tool can make a significant contribution to the literature.

5 Limitations

Exploratory FA was performed with the data gathered right after the scale form was performed to the teachers. Afterwards, confirmatory FA was performed to confirm the scale’s factor structures, which was determined to consist of four factors and 25 items according to FA with the data available without the second application.

Data availability

No additional data is associated with this paper.

Alan, B., & Güven, M. (2022). An international concept confusion: A document analysis on the concepts of competence, competency and competent. Anadolu Journal of Educational Sciences International, 12 (1), 271–293. https://doi.org/10.18039/ajesi.1004416

Article   Google Scholar  

Bandura, A. (1997). Self-efficacy. The exercise of control . W.H. Freeman.

Google Scholar  

Bonk, C. J., & Graham, C. R. (2004). The handbook of blended learning: Global perspectives, local designs . John Wiley & Sons.

Büyüköztürk, Ş. (2002). Sosyal bilimler için veri analizi el kitabi [Data analysis for social sciences handbook] . Pegem Akademi.

Byrne, B. M. (2016). Structural equation modeling with AMOS: Basic concepts, applications, and programming (3nd ed.). Routledge Pub.

Book   Google Scholar  

Cabı, E., & Gülbahar, Y. (2013). A scale development study for assessing the effectiveness of blended learning environments. Pegem Journal of Education & Instruction, 3 (3), 11–26 https://dergipark.org.tr/en/pub/pegegog/issue/22581/241208

Cambridge, (2020). Shaping Up Education Models for the Future in The UAE.  https://www.cambridgeassessment.org.uk/news/shaping-up-education-models-for-the-future-in-the-uae/ . Accessed 10.05.2023.

Chan, K. W., & Elliott, R. G. (2004). Relational analysis of personal epistemology and conceptions about teaching and learning. Teaching and Tteacher Education, 20 (8), 817–831. https://doi.org/10.1016/j.tate.2004.09.002

Çokluk, Ö., Şekercioğlu, G., & Büyüköztürk, Ş. (2010). Sosyal Bilimler Için Çok Değişkenli Istatistik: Spss Ve Lisrel Uygulamaları [Multivariable statistics for social sciences: Spss and Lisrel applications] . Pegem Akademi.

Creswell, J. W. (2012/2020). Educational research: planning, conducting, and evaluating quantitative and qualitative research , 4th Edition. Halil Ekşi Editor. İstanbul: Edam, 481

Dziuban, C., Graham, C. R., Moskal, P. D., Norberg, A., & Sicilia, N. (2018). Blended learning: The new normal and emerging technologies. International Journal of Educational Technology in Higher Education, 15 , 3. https://doi.org/10.1186/s41239-017-0087-5

Eastman, P. (2015). Blended learning design guideline . Office of the State Superintendent of Education Charter School Incubator Initiative.

European Union, n.d. Education, training and youth . Retrieved from https://european-union.europa.eu/priorities-and-actions/actions-topic/education-training-and-youth_en . Accessed 10.05.2023.

Fidan, M., Debbağ, M., & Çukurbasi, B. (2020). Technology proficiency self-assessments of teachers becoming professional in the 21st Century: A scale adaptation study. Pegem Journal of Education and Instruction, 10 (2), 465–492. https://doi.org/10.14527/pegegog.2020.016

Field, A. (2013). Discovering statistics using IBM SPSS Statistics. (4th Edition) (pp. 665–714). Sage.

Fresen, J. (2007). A taxonomy of factors to promote quality web-supported learning. International Journal on E-Learning, 6 (3), 351–362 Chesapeake, VA: AACE https://www.learntechlib.org/primary/p/20885/

Gökçe, F. (2014). Sınıfta Öğrenme ve Öğretme Sürecinin Yönetimi [ Management of Learning and Teaching Process in Classroom] . Pegem Akademi.

Göksel, N. (2015). Uzaktan Eğitim: Çevrimiçi Öğrenmede Sistem Yaklaşımı. Açıköğretim Uygulamaları ve Araştırmaları Dergisi . [Distance education: System approach in online learning]. Journal of Open Education Applications and Research, 1 (1), 129–138 https://hdl.handle.net/11421/24387

Graham, C. R. (2006). Blended learning systems: Definition, current trends, and future directions. In C. J. Bonk & C. R. Graham (Eds.), The Handbook Of Blended Learning: Global Perspectives, Local Designs (pp. 3–21). Pfeiffer http://www.publicationshare.com/graham_intro.pdf

Graham, C. R., Borup, J., Pulham, E., & Larsen, R. (2019). K–12 blended teaching readiness: Model and instrument development. Journal of Research on Technology in Education, 51 (3), 239–258. https://doi.org/10.1080/15391523.2019.1586601

Gülbahar, Y., Kalelioğlu, F., & Afacan Adanır, G. (2020). Harmanlanmış öğrenme. [Blended Learning] . Pegem Akademi.

Gürbüz, S. (2021). Amos ile Yapısal Eşitlik Modellemesi Temel İlkeler ve Uygulamalı Analizler [Structural Equation Modelling with Amos Basic Principles and Applied Analyses] (2nd ed.). Seçkin Publishing.

Hannon, J., & Macken, C. (2014). Blended and online curriculum design toolkit . La Trobe University.

Hockly, N. (2018). Blended Learning. ELT Journal, 72 (1), 97–101. https://doi.org/10.1093/elt/ccx058

Horn, M. B., & Staker, H. (2015). Blended: Using disruptive innovation to improve schools . Published by Jossey-Bass .

Horn, M. B., & Staker, H. (2017). The blended workbook learning to design the schools of our future . Published by Jossey-Bass .

Horton, W., & Horton, K. (2003). E-Learning tools and technologies . Wiley Publishing, Inc..

Howard, M. C. (2016). A review of exploratory fa decisions and overview of current practices: What we are doing and how can we improve? International Journal of Human-Computer Interaction, 32 (1), 51–62. https://doi.org/10.1080/10447318.2015.1087664

Hrastinski, S. (2019). What do we mean by blended learning? TechTrends, 63 (5), 564–569. https://doi.org/10.1007/s11528-019-00375-5

Karasar, N. (2007). Bilimsel araştirma yöntemi [Scientific research method] . Nobel.

Khan, B. H. (2005). Managing e-learning: Design, delivery, implementation, and evaluation . Infomation Science Publishing.

Koç, M. (2019). Preservıce teachers’ beliefs regarding instructional value of blended learning (Master's Thesis) . Ege University.

Korkmaz, Ö., Arıkaya, C., & Altıntaş, Y. (2019). Developing the self-efficacy scale of teachers' digital teaching material development. Turkish Journal of Primary Education, 4 (2), 40–56 https://dergipark.org.tr/en/pub/tujped/issue/50537/609625

Los, R., De Jaeger, A., & Stoesz, B. M. (2021). Development of the Online and Blended Teaching Readiness Assessment (OBTRA). In Frontiers in Education (Vol. 6, p. 673594). Frontiers.

MEB, (2017). Öğretmenlik Mesleği Genel Yeterlikleri. Öğretmen Yetiştirme ve Geliştirme Genel Müdürlüğü. [General Qualifications for Teaching Profession. General Directorate of Teacher Training and Development]. Ankara. Retrieved from https://oygm.meb.gov.tr/meb_iys_dosyalar/2017_12/11115355_YYRETMENLYK_MESLEYY_GENEL_YETERLYKLERY.pdf on 12.04.2023.

MEB, (2018). Millî Eğitim Bakanlığı 2023 eğitim vizyon belgesi [Ministry of National Education 2023 education vision document]. Retrieved from, http://2023vizyonu.meb.gov.tr/doc/2023_EGITIM_VIZYONU.pdf on 12.04.2023.

Mıhçı Türker, P., & Öztürk, M. (2022). The adaptation of the blended teaching readiness instrument to Turkish. Educational Technology Theory and Practice., 12 ( 2 ), 460–479. https://doi.org/10.17943/etku.1084982

Moore, M. G., & Kearsley, G. (2004). Distance education: A systems view (2nd ed.). Wadsworth Publishing.

Osguthorpe, R. T., & Graham, C. R. (2003). Blended Learning Environments: Definitions and Directions. The Quarterly Review of Distance Education, 4 (3), 227–233 https://www.learntechlib.org/p/97576/

Oxford Dictionaries. (2023). Oxford learner’s dictionaries.  https://www.oxfordlearnersdictionaries.com/definition/english/blended-learning?q=blended+learning . Accessed 08.03.2023.

Rivera, J. H. (2017). The Blended Learning Environment: A Viable Alternative for Special Needs Students. Journal of Education and Training Studies, 5 (2), 79–84. https://doi.org/10.11114/jets.v5i2.2125

Russell, D. W. (2002). In search of underlying dimensions: The use (and abuse) of FA in personality and social psychology bulletin. Personality and Social Psychology Bulletin, 28 (12), 1629–1646. https://doi.org/10.1177/014616702237645

Şad, S. N., Alkan, İ., & Koç, A. (2016). İnternet özyeterlilik algisi ölçeğinin öğretmen ve Öğretmen adaylari için türkçe’ye uyarlanmasi [Adaptation of the internet self-efficacy perception scale into Turkish for teachers and pre-service teachers]. In III. International Eurasian Educational Research Congress (pp. 274–279). Muğla Sıtkı Kocman University.

Schenk, M. (2023). Blended Learning Approach for Special Educational Needs (SEN) Students. Acadecraft .  https://www.acadecraft.com/blog/blended-learning-approach-for-special-educational-needs-sen-students

Şencan, H. (2005). Sosyal ve Davranışsal Ölçümlerde Güvenilirlik ve Geçerlilik [Reliability and Validity in Social and Behavioral Measurements] . Seçkin Publishing.

Shakeel, S. I., Haolader, M. F. A., & Sultana, M. S. (2023). Exploring dimensions of blended learning readiness: Validation of scale and assessing blended learning readiness in the context of TVET Bangladesh. Heliyon , e12766. https://doi.org/10.1016/j.heliyon.2022.e12766

Singh, H., & Reed, C. (2001). A white paper: Achieving success with blended learning. Centra Software, 1 , 1–11.

Staker, H., & Horn, M. B. (2012). Classifying K-12 Blended Learning . Innosight Institute.

Tang, C. M., & Chaw, L. Y. (2013). Readiness for blended learning: Understanding attitude of university students. International Journal of Cyber Society and Education, 6 (2), 79–100. https://doi.org/10.7903/ijcse.1086

Teach Thought Staff, (2019). 12 of the Most common types of blended learning. https://www.teachthought.com/learning/12-types-of-blended-learning/ on 12.04.2023

Toker, T., Akgün, E., Cömert, Z., & Edip, S. (2021). Digital competency scale for educators: adaptation, validity and reliability study. National Education Journal, 50 (230), 301–328. https://doi.org/10.37669/milliegitim.801607

Tonbuloğlu, İ., & Tonbuloğlu, B. (2021). Eğitimde dijital dönüşüm harmanlanmiş öğrenme [Digital transformation blended learning in education] (Analysis report: 2021/09) . İLKE İlim Kültür Eğitim Vakfı [Foundation of Science Culture Education]. https://doi.org/10.26414/anr09

Vaughan, N. (2007). Perspectives on blended learning in higher education. International Journal on E-Learning, 6 (1), 81–94.

Welker, J., & Berardino, L. (2005). Blended learning: Understanding the middle ground between traditional classroom and fully online instruction. Journal of Educational Technology Systems, 34 (1), 33–55.

Wheeler, S. (2001). ICT and the changing role of the teacher. Journal of Educational Media, 26 (1), 7–17. https://doi.org/10.1080/1358165010260102

Wilson, D., & Smilanich, E. (2005). The other blended learning: A classroom-centered approach . John Wiley & Sons https://docs.edtechhub.org/lib/Z3R5NFEU

Yıldız Durak, H. (2017). Turkish adaptation of the flipped learning readiness scale for middle school students. Bartin University Journal of Faculty of Education, 6 (3), 1056–1068. https://doi.org/10.14686/buefad.328826

Yörük, T., & Özçetin, S. (2021). Adaptation of online learning self-efficacy scale into Turkish. Kahramanmaraş Sütçü İmam Üniversitesi Sosyal Bilimler Dergisi [Kahramanmaraş Sütçü İmam University Journal of Social Sciences], 18 (3), 1640–1657. https://doi.org/10.33437/ksusbd.983825

Zavaraki, E., & Schneider, D. (2019). Blended Learning Approach for Students with Special Educational Needs: A Systematic. Journal of Education & Social Policy, 6 (3), 1–12. https://doi.org/10.30845/jesp.v6n1p12

Download references

Acknowledgements

I would like to thank the teachers for their help during the data collection process.

Open access funding provided by the Scientific and Technological Research Council of Türkiye (TÜBİTAK). There was no external funding for this research.

Author information

Authors and affiliations.

Department of Computer Education, and Instructional Technology, Amasya University, Science Institute, Amasya, Türkiye

Gülsemin Durmuş Çemçem

Department of Computer Engineering, Amasya University, Engineering Faculty, Amasya, Türkiye

Özgen Korkmaz

Department of Instructional Technology, Amasya University, Education Faculty, Amasya, Türkiye

Volkan Kukul

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Gülsemin Durmuş Çemçem .

Ethics declarations

Competing interests.

There are no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Çemçem, G.D., Korkmaz, Ö. & Kukul, V. Readiness of teachers for blended learning: A scale development study. Educ Inf Technol (2024). https://doi.org/10.1007/s10639-024-12777-x

Download citation

Received : 06 March 2024

Accepted : 07 May 2024

Published : 29 May 2024

DOI : https://doi.org/10.1007/s10639-024-12777-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Teacher qualifications
• Blended learning
• Teacher readiness
• Scale development
• Find a journal
• Publish with us
• Track your research

Addressing the Challenge of Hybrid Learning Environment in Foreign Language Education: Training Lecturers for Blended Teaching Approaches

• Abdirahman Abdi Ali Prince Sattam bin Abdulaziz University
• Khadijah AlQarni King Abdulaziz University
• Hamzah Faleh Migdadi Al-Hussan Group for Education and Training
• Tahani I. Aldosemani Prince Sattam bin Abdulaziz University
• Shadi Majed Alshraah Prince Sattam bin Abdulaziz University

This study investigates the effectiveness of training programs in supporting language lecturers in integrating blended learning methodologies. It underscores the significance of Vygotsky's educational theory and involves 126 university educators through a mixed-methods approach. Interviews and surveys were conducted to explore challenges and solutions. Findings reveal the pivotal role of training in enhancing lecturers' digital literacy and content creation skills and ensuring fair access for all students. Notably, 61.3% of participants recognize the value of training in developing engaging digital content, with 29.88% strongly supporting this perspective. Training is essential for creating multimedia resources and designing online courses to foster student engagement. The study emphasizes the need for tailored training to address diverse technological needs, with 59.74% of respondents acknowledging its importance and 31.53% strongly agreeing. Overall, training empowers lecturers to meet the varying digital accessibility needs of students, including through the development of alternative offline materials and the implementation of asynchronous activities. This research highlights the critical role of training programs in facilitating effective blended learning practices among language lecturers.

Author Biographies

Abdirahman abdi ali, prince sattam bin abdulaziz university.

English Department, Preparatory Year Deanship

Khadijah AlQarni, King Abdulaziz University

English Language Institute

Hamzah Faleh Migdadi, Al-Hussan Group for Education and Training

English Language Department