Thinking Level in Education: A Complete Revision of Anderson’s Taxonomy

1 Jakarta State University, Rawamangun Muka St ., ID-13220 Jakarta, Indonesia, ilmizajuli95@gmail .com 2 Jakarta State University, Rawamangun Muka St ., ID-13220 Jakarta, Indonesia, henita .rahmayanti@unj .ac .id 3 Jakarta State University, Rawamangun Muka St ., ID-13220 Jakarta, Indonesia, agungpurwanto@unj .ac .id 4 Jakarta State University, Rawamangun Muka St ., ID-13220 Jakarta, Indonesia, dianav@unj .ac .id 5 Semarang State University, Sekaran Gunungpati, ID-50229 Semarang, Indonesia, edikurniawan@mail .unnes .ac .id 6 Malang State University, Semarang 5 Malang St ., ID-65145 Malang, Indonesia, ardyanto .tanjung .fis@um .ac .id 7 Tanjungpura University, Jenderal Ahmad Yani St ., ID-78124 Pontianak, Indonesia, ruqiah .gpp@fkip .untan .ac .id 8 Makassar State University, A P Pettarani St ., ID-90222 Makassar, Indonesia, nurlita .pertiwi@unm .ac .id 9 Sultan Idris Education University, MY35900 Tanjong Malim, Malaysia, charanjit@fbk .upsi .edu .my


Introduction
21st century education has changed in various aspects. The changes include technology used in learning up to abilities must be acquired by students. This happens to adapt to the needs of various fields of work and expertise in the 21st century. Students in the 21st century do not only need memory ability but they must be able to identify and have a higher-level problem-solving skill. These abilities must certainly be taught in education (Motallebzadeh et al., 2018;Talmi et al., 2018;Zohar & Agmon, 2018). Morganti et al., 2017;Purwanto et al., 2020;Rahmayanti et al., 2020;Sigit et al., 2019Sigit et al., , 2020So et al., 2019). Schools, higher education institutions, and teacher training sectors have equal roles to nurture the development of thinking skills among students. Teaching higher order thinking skills become meaningful when there is an incorporation of critical and creative thinking in activities prepared for the students. Activities prepared by educators help to guide and teach the students to think critically and analytically. Tanujaya (2016) developed an instrument that measures higher order thinking skills among students. The instrument aimed to measure students' mastery of higher-order thinking at every level of education. The study findings showed that students were not able to understand and master higher-order thinking skills. This was due to the fact that students failed to structure thinking skills. The higher-order thinking skills are a crucial aspect of the teaching and learning process. There is a dire need to develop strategies to foster higher-order thinking skills by the teachers to the students so that they can delve into in-depth understanding of cognitive development and systems of classifying thought processes. The low level of the skill is the result of teacher's lack of attention to such thinking skills (Tanujaya, 2016). For students to master thinking skills, teachers must incorporate thinking skills exercises in all subjects taught. Students with the skill of applying higher order thinking skills are able to perform with excellence. Undoubtedly, teachers must be aware of the importance of infusing HOTS for students to be able to perform with distinction. Therefore, teachers can effectively promote higher order thinking if they have an in-depth understanding of the stages of cognitive development.
Based on this description, it is urgent to carry out a comprehensive and complete revision of Anderson's Taxonomy. The new taxonomy can be used for all subjects at school or university level. Furthermore, the new taxonomy can be used for the wider community in the context of social education. This research will revise the thinking level of Anderson's taxonomy as a whole in the LOTS and HOTS domains to be used in all subjects. The new taxonomy is a novelty in the 21st century education and is imperative to carry out. This is because the new taxonomy is more contextual to measure the ability of students in education in a rapidly developing technological era in the 21st century. Therefore, this study aimed to revise the Anderson's taxonomy to a new taxonomy and implement it in classroom learning.

Research design
This study used a descriptive method with a survey technique. Descriptive research is a method that facilitates the researchers to describe in detail each score obtained by students. This will simplify the descriptive data analysis. Descriptive research is conducted to measure a variable in more detail Rahmayanti et al., 2019). Research using descriptive methods is usually carried out using instruments in the form of test questions, observation sheets, interviews, or questionnaires given to respondents (Bigirwa et al., 2020;Bilasa & Taspinar, 2020;Bodzin et al., 2020). The advantage of descriptive research is that the data collected will be more accurate because it focuses on describing the data.

Sample and Data collection
The study samples were taken from various cities in Indonesia. Samples were selected randomly using simple random sampling. The number of sample was 1030 people that comprised 334 university students, 211 Junior High School (JHS) students, and 485 Senior High School (SHS) students. Samples were randomly selected from various schools and universities in various cities in Indonesia.

Analyzing Data
The data analyzed were in the form of descriptive data. The data were analyzed using Microsoft Excel and Statistical Package for the Social Sciences (SPSS). The data presented in tabular form to make it easier to describe students' knowledge scores for each level. This was intended so that the details of each item and indicator can be described clearly. The categorization of the students' knowledge scores is presented in Table 1.

Thinking Level and Indicators
The thinking levels developed in the study were in accordance with previous findings, six thinking levels were formulated as a new taxonomy in this research, namely: Identify, Compare, Implement, Criticize Problem, Solve Problem, and Develop innovation (see Table 2). Indicators used in this study were the developed knowledge indicators. The indicators were based on the revised thinking level (new taxonomy). As for the new thinking level (TL) that has been revised from the old version consisted of (TL-1) Identify, (TL-2) Compare, (TL-3) Implement, (TL-4) Criticize problem, (TL-5) Solve Problem, and (TL-6) develop innovation. Detail comparison of the thinking level between Anderson's version and the revised version (new taxonomy) is described in Table 2. The changes from the Anderson's taxonomy are depicted in a diagram in Figure 1. It could be seen that there were changes in the LOTS and HOTS. There was a cross (an increase and decrease in levels in the HOTS domain) which was adapted from previous HOTSEP research (Ichsan & Rahmayanti, 2020)especially to solve environmental problems when COVID-19 pandemic. This was to develop a new level of thinking, namely Higher Order Thinking Skills of Environmental Problem (HOTSEP). In the HOTSEP's research, three thinking levels had been revised: criticize environmental problem, solve environmental problem, and develop innovation about the environment. All revisions made to HOTSEP research were in the higher thinking level. The results of the HOTSEP study were adapted and became a thinking level in the new taxonomy in this study, namely criticize problem, solve problem, and develop innovation (see Figure 1).
As regards knowledge measurement using the new taxonomy, the instruments used were test questions in the form of multiple-choice, filling in, essays, or other forms that were still relevant to measuring student knowledge. To facilitate the preparation of test questions, the study also developed operational verbs. The function of these operational verbs is to distinguish the characteristics of each question made at various levels. The operational verbs can be modified according to the context of the learning being carried out. Details of the operational verbs can be seen in Table 3.

Instruments
The instruments used were distributed online using Google Form. The instruments used in the study consisted of six types, namely: Environmental Knowledge for University (EKU), Natural Science Knowledge for University (NSKU), Social Science Knowledge for University (SSKU), Environmental Knowledge for School (EKS), Natural Science Knowledge for School (NSKS), and Social Science Knowledge for School (SSKS). The six instruments were developed based on the thinking level in the new taxonomy. The number of items for each instrument was 12 items. The EKU instrument is elaborated in Table 4.

Table 4
Indicators of the EKU instrument

Thinking level Indicator Item
Identify (TL-1) Identifying various types of environmental disasters. 1, 2 Compare (TL-2) Comparing the different characteristics of natural and non-natural disasters in the environment.

9, 10
Develop Innovation (TL-6) Developing innovations to solve problems in support of environmental disaster mitigation programs.

11, 12
Details of the NSKU instrument can be seen in Table 5. The NSKU instrument was developed with general knowledge coverage for all students. Items made are relatively more general that various students from various backgrounds can answer them.

Thinking level Indicator Item
Identify (TL-1) Identifying various physical and chemical factors that affect human, animal, and plants.

1, 2
Compare (TL-2) Comparing the differences between physical and chemical factors in human, animal, and plants.

3, 4
Implement (TL-3) Implementing physics and chemistry concepts in everyday life related to human, animal, and plants.

7, 8
Solve Problem (TL-5) Solving problems around them using the concepts of physics, chemistry, and biology.

9, 10
Develop Innovation (TL-6) Developing innovations to solve problems and support efforts to conserve nature based on various concepts of physics, chemistry, and biology.

11, 12
The SSKU instrument made for the university level focuses more on the overall concept of social sciences from the aspects of geography, sociology, and economics. These various aspects were included to measure student knowledge. In more detail, the indicators for this instrument can be seen in Table 6.

Table 6
Indicators of the SSKU Instrument

Thinking level Indicator Item
Identify (TL-1) Identifying various social events that occur in the community.

1, 2
Compare (TL-2) Comparing the differences between social, economic and geographic factors in a society.

3, 4
Implement (TL-3) Implementing the concepts of sociology, economics, and geography that have been studied.

5, 6
Criticize problem (TL-4) Criticizing various social and economic problems caused by geographic factors.

7, 8
Solve Problem (TL-5) Solving problems around them using the concepts of economics, sociology, and geography that have been studied.

9, 10
Develop Innovation (TL-6) Developing innovations to solve social problems with various concepts that have been studied.

11, 12
The EKS instrument was made for the secondary school level. The indicators developed for EKS focused more on an environmental pollution problem. It was used to measure students' knowledge in dealing with various pollution problems that occur in their neighborhoods. The EKS instrument showed in Table 7. The NSKS instrument was made to measure students' knowledge of various natural science concepts. The NSKS instrument was made by adapting various contextual natural phenomena to be discussed. It was intended so that students can use various concepts of natural science to solve problems in everyday life. Details of the NSKS instrument can be seen in Table 8. Implement (TL-3) Implementing the concept of a healthy lifestyle for organ health.

5, 6
Criticize problem (TL-4) Criticizing students who do not want to exercise. 7, 8 Solve Problem (TL-5) Solving problems using the concepts of physics, chemistry, and biology.

9, 10
Develop Innovation (TL-6) Developing simple program ideas based on natural science principles.

11, 12
Meanwhile, the SSKS instrument was developed to measure students' knowledge in social sciences. The instrument consisted of 12 items covering a variety of general knowledge in the fields of geography, sociology, and economics. Detail of the SSKS instrument can be seen in Table 9.

Table 9
Indicators of the SSKS instrument

Thinking Level Indicators Item
Identify (TL-1) Identifying the names of cities and counties based on geographic location.

7, 8
Solve Problem (TL-5) Solving problems about students who don't want to socialize.

9, 10
Develop Innovation (TL-6) Developing innovations to be able to socialize among students outside the region.

Validity and Reliability
The validity test was carried out using SPSS with the Pearson Product Moment Correlation technique. The results showed that the instruments were categorized as valid for all items. The results of the validity test can be seen in Table 10. The reliability value of each instrument was EKU (0.58), NSKU (0.74), SSKU (0.87), EKS (0.74), NSKS (0.63), and SSKS (0.59). The reliability results of the instrument showed that all instruments can be used to measure student knowledge. The results of this reliability calculation also showed that the instrument can be used in subsequent studies.

Results
The results indicated that the EKU score was still in a very low category. It suggested that students' knowledge from TL-1 to TL-6 must be improved. It was especially in TL-6 which is related to developing innovative programs to overcome flood problems. Many students had not been able to create innovative programs and tended to create commonly conducted standard programs. It can be considered as not innovative. Details of the results can be seen in Table 11. Regarding the NSKU score, it indicated that students in general had not been able to understand the concept of natural sciences as a whole. It was evidenced by a very low knowledge score. The lowest score obtained on TL-6 that is related to innovation during the Coronavirus Diseases 2019 (COVID-19) pandemic to address the problem of scarce clean water supplies. It can be seen more clearly in Table 12. Students' knowledge in terms of social science obtained the lowest score at TL-6. This showed that students had not been able to contribute in the form of innovation to solve the problem of lack of employment in the surrounding community. Students who study various social sciences should have ideas and innovations to solve this problem. The SSKU scores can be seen in Table 13. The results of the EKS measurement implied that the overall student score was still in a very low category for understanding various environmental concepts. The lowest score on EKS was found on items with TL-6 that is related to innovation for environmentally friendly campaign in the wider community. The results can be seen in Table 14. The NSKS scores obtained by JHS and SHS students were also in a very low category. It could be assumed that natural science learning in schools has not run optimally. The lowest score from the NSKS was found in item 12 regarding making innovative ideas to invite the public to maintain cleanliness. The detail of NSKS scores can be seen in Table 15. The SSKS instrument is made with a wide range of social knowledge. The goal is that students can think to solve social problems from various aspects of science. The lowest score from SSKS was on items related to implementation (TL-3) as regards the application of social sanctions. In addition, a low score also was obtained by developing innovation (TL-6) that is related to developing innovative ideas so that students between regions can interact online. The details of the SSKS score can be seen in Table 16. The study results as a whole stated that the knowledge of the college students and students was still in the very low category. Therefore, efforts to develop various innovations in education must be increased. The innovation should be aimed at improving the LOTS and HOTS domains. Figure 2 describes the learning process to improve knowledge.

Figure 2
Learning Process to Improve LOTS and HOTS

Discussion
Students' abilities in the LOTS need to be improved. This is because LOTS is a basic ability for someone to understand more complicated matters. LOTS is important because it is an initial ability to improve HOTS (Fitzpatrick & Schulz, 2015;Istiyono et al., 2020;Yeung, 2015). Students who have sufficient LOTS will be able to identify various things around them. Learning at various levels must make the LOTS stronger so that students find it easier to practice HOTS skills. Both abilities must be enhanced by carrying out various innovations in technology-based education to accommodate various abilities of the 21st century (Boholano, 2017;Cukurova & Bennett, 2018;Quieng et al., 2015;Sigit et al., 2020;Urbani et al., 2017).
The LOTS ability in Anderson's Taxonomy consists of 3 skills, namely: remember, understand, and apply. The ability was less accurate to measure students' abilities in the 21st century. In this century, students no longer remember the learning topics, but they must also be able to identify various things around them. The ability to identify is much more important and contextual than just remembering. Additionally, the ability to understand turns into the ability to compare. It is easier to measure students' ability to "compare" than to "understand" as understanding is a very broad yet less operational aspect. The final LOTS level is the ability to apply that has changed into implementing various concepts. The change in thinking skills in the LOTS was following the needs of students who can be contextual in understanding various contextual topics (Cronje et al., 2011;Mendhurwar & Mishra, 2018;Paristiowati et al., 2019;Sahronih et al., 2019).
Once students have mastered the LOTS abilities, their HOTS abilities begin to be trained at the same time. This was important because HOTS ability is the ability to solve a problem and develop innovation to solve the problem (Elfeky, 2018;Husamah et al., 2018;Tanujaya et al., 2017;Vidergor, 2018). The HOTS domains of Anderson's Taxonomy consist of: analyze, evaluate, and create. The domains had changed into criticize problems, solve problems, and develop innovation. The changes were adapted from the HOTSEP taxonomy in previous studies (Ichsan & Rahmayanti, 2020). The difference was that changes in the HOTS domain in the new taxonomy were more general for all subject topics instead of merely the environment topic. The change in the HOTS was important because students in the 21st century must be more critical in responding to various problems that occur (Anagun, 2018;Lee, 2016;Smith, 2014;Suwono et al., 2017).
The change from Analyze (TL-4) to Solve Problem (TL-5) that increased the level from level 4 to 5 was due to the need to not merely analyze a problem. Students in the 21st century also require problem-solving skills. In addition, students' ability to analyze will be easier to measure in the solve problem aspect. It has to do with students' ability to analyze that was hard to show in a concrete or contextual form. The score given for the ability to analyze was also difficult because it was more subjective to assess a person's ability to analyze. The second change was in evaluate (TL-5) to criticize problem (TL-4) which has a decreased level. This was because it is hard to measure students' ability to evaluate. Students must be able to criticize a problem. The ability to criticize a problem will be easier to measure than the ability to evaluate something. The last change was in Create (TL-6) that became develop innovation (TL-6). The level remained at the highest level. The change occurred because students in the 21st century are not only required to create something but develop an innovation. Students not only create standard work but must be accompanied by innovative ideas. These innovative ideas are very important to form creativity and originality in a work (Cheng et al., 2016;Lince, 2016;Ritter & Mostert, 2017).
Education must be strengthened in studying various topics of environment, natural sciences, and social and other sciences. This research showed that it is necessary to perform various educational innovations in terms of learning. The low knowledge score indicates that learning media, learning strategies, learning models, student worksheets and various learning tools need to be developed. Innovations must be made especially to improve students' HOTS in learning on various topics. The topics of environment, natural sciences, and social sciences are the most common topics and must be mastered by all students. Knowledge of the environment will make students better understand and grasp the various problems of environmental damage (Arnon et al., 2015;Goldman et al., 2017;Kinslow et al., 2018). Likewise, various concepts of natural sciences and social sciences need to be mastered to fit the 21st-century educational context (Calado et al., 2018;Dani, 2011;Sarkar & Corrigan, 2014). This new taxonomy can also be implemented in various subjects such as citizenship, language, engineering, mathematics, sports, arts, and other subjects at the school to university level.

Conclusion
Based on the results, a new taxonomy has been developed and a conclusion could be drawn that student's knowledge scores are all still in the very low category. The new taxonomy measures student knowledge more accurate and contextual according to the needs of the 21st century. Students in the 21st century are expected not only to create something but also products or ideas that are developed to be original and innovative. It indicated the need for innovation in education. Besides, efforts are also needed to develop various learning tools, learning models, learning strategies, learning media, teaching materials, and other equipment.

Suggestions and Limitation
The limitation of this study includes the sample used that was limited to measure environmental knowledge, natural sciences knowledge, and social sciences knowledge. Meanwhile, measurements for other subjects have not been carried out using the new taxonomy. The suggestion of this study is to measure knowledge of other subjects/learning topics using a new taxonomy. Moreover, it is necessary to develop various educational innovations to strengthen student knowledge in various subjects and at various levels. socialinių mokslų žinios mokykloje ( jaunesniųjų 33,92 ir vyresniųjų 36,00). Nustatyta, kad studentų ir mokinių žinias dar reikia tobulinti įvairiomis švietimo naujovėmis. Tyrimo metu buvo sukurta nauja taksonomija, o jos įgyvendinimas parodė, kad studentų ir mokinių žinių balas vis dar labai žemas, matuojant pagal mąstymo lygį naujoje taksonomijoje, todėl studentų ir mokinių balą reikia gerinti diegiant įvairias švietimo naujoves.