A Study of the Validity and Reliability of the Preschool School Climate Scale (PSCS)

Annotation. In this study, it is aimed to psychometrically evaluate the Preschool Climate Scale (PSCS), which will evaluate the school climate characteristics of preschool education institutions. Both explanatory and confirmatory factor analyzes were performed in the sample of teachers and administrators (N=910). High internal consistency values were obtained from the structure consisting of 53 items and 6 subscales. Similar values were confirmed at different times and samples after testing. The results revealed that the PSCS is a valid and consistent scale. factors of the scale (subscale) and Cronbach alpha values are as follows: (a) Principal support, accessibility and competence (.91), (b) Colleague solidarity (.91), (c) Positive social development practices (.90), (d) Student behaviours (.90), (e) Teacher effectiveness (.88), (f) Enjoyment of teaching (.70), (g) Parent participation (.75). In this study, the “student behaviors” sub-dimension of the mentioned scale was used to examine criteria-dependent validity.


Introduction
The preschool education institutions support children's cognitive development and they are the places where children establish positive social relationships, gain independence, develop emotionally, behaviorally, and cognitively (Cohen et al., 2009). The effect of the experiences gained at preschool transfer at the latter stages of education (Peisner-Feinberg et al., 2001). In addition, an increasing number of studies draw attention to the school climate in the multifaceted development of children and the well-being of both teachers and students (Grazia & Molinari, 2019). For this reason, the school climate which expresses the physical environment and social dynamics of the institutions (Hayne et al., 1997), is a subject that must be dwelled upon acutely particularly in the preschool period.
The school climate defines the characteristics of a school, and the characteristics of the social, physical, and psychological settings in which it exists (Hoy & Tarter, 1997). The relationship between children, parents, teachers, school principals, and other personnel in the school, which affects the school environment, the structure of the organization, and the adopted learning-teaching practices are all within the school climate (Cohen et al., 2009;Ruiz-Hernandez et al., 2021;Thapa et al., 2013). In addition to these, the rules expected to be followed at school, the beliefs, values, and goals of individuals who affect the school climate are also related to the school climate (Grazia & Molinari, 2019). As can be understood from these definitions, the concept of school climate is multidimensional and has a complex structure. For this reason, there are some difficulties in measuring the school climate, which includes a complex and wide structure (Grazia & Molinari, 2019).
The school climate is one of the basic elements which determines whether children's development and learning will be positive or negative. The preschool education institutions are the settings where children experience their first formal education and are quite valuable in the sense that they form the basis of later education stages (Adams, 2008;Aguilar & Tansini, 2012). To determine the educational institutions which have a negative school climate and take precautions about them, the school climate of preschools must be evaluated. Nevertheless, the measurement tools which will evaluate the quality of preschool climate extensively are limited (Dongying & Yue, 2020).

Theoretical framework
The school climate points out the personality or character of school life (Cohen et al., 2009). In other words, the school climate is how people at school feel when at school, or the 'feeling' of the school (Tableman, 2004;Welsh, 2000). It reflects the values, aims, and norms representing the educational and social responsibilities of the school, and the quality of the interaction of children, teachers, parents, and school personnel (National School Climate Council, 2007). While the school climate expresses the quality of inter-school relationships, it also includes the interpersonal, organizational, and academic dimensions (Loukas et al., 2006).
When the relevant literature is examined, it is seen that there is no consensus on the definition of school climate. This may be since the school climate literature is not very old and this concept has a multidimensional and complex structure. One of the preliminary studies on the conceptualization of school climate was done by Halpin and Croft (1962). The definitions of "school climate" made since then can be evaluated under three headings according to their focus. It is seen that the first of these definitions is organization-institution oriented (Cohen et al., 2009;Hoy & Miskel, 1987). E.g., Cohen et al. (2009) define school climate as the personality or character of school life. The second group of definitions seems to center individuals (Litwin & Stringer, 1968;Haynes et al., 1997;Peterson et al., 2001;Thapa et al., 2013). E.g,, Peterson et al. (2001) define school climate as the feelings of its members about the school environment. The third group of definitions is relationship and value-oriented (Loukas et al., Harton, 2006;NSCC, 2007). E.g., According to The National School Climate Council (2007), school climate is people's experiences, norms, goals, interpersonal relationships, teacher-learning experiences, and organizational structures within the school. Based on this information, it can be said that school climate is a very broad concept with many variables and includes many perspectives.
School climate is an organizational structure that includes many elements that affect and are affected by it. These are school principals and vice-principals, teachers, other school personnel (cook, cleaning staff, security, etc), children, and families (Kurt & Çalık, 2010). Although these seem to be separate from each other, they directly and indirectly affect each other and the climate of the institution. In addition to these, school climate consists of many dimensions. For example, Thapa et al. (2013) divided school climate into five dimensions. These are safety (rules and norms, physical security, social-emotional security), relationships (respect for differences, social support, leadership, children's ethnicity, etc.), teaching and learning (support for academic learning, social-emotional learning, ethics and citizenship learning, etc.), institutional environment (physical structure, resources, supports) and school development process. Lowenstein et al. (2015) also divided school climate into four dimensions. These are safety, adult support, classroom processes, and school-classroom features. NSCC (2017), on the other hand, divided the school climate into 13 dimensions. These are rules and norms, physical security and social-emotional security under the theme of security; supporting learning and social and community learning under the theme of teaching and learning; respect for differences, social support -adults and social support -students under the theme of interpersonal relations; school engagement-responsibility and the physical environment under the theme of the institutional environment; social media under the theme of social media and leadership and professional relations under the theme of employees. In summary, school climate includes school administrators, teachers, children, families, the relations between them, education and training, safety, and the physical environment of the institution.
According to the bioecology hypothesis of Bronfenbrenner (1977), the school children's close surroundings reflect the microsystem, and the relationships hereby reflect the mesosystem. These systems are in constant interaction. The people's development hereby is affected by the activities, roles, and interpersonal relationships (Bronfenbrenner, 1977). When the people and institutions establish strong and supportive connections with the child, they have the power to support the child's development in a positive way (Trawick-Smith, 2013). On the one hand, all the individuals who are present in the school system directly or indirectly and the relationship patterns affect the school climate. In his theory, Epstein (2001) emphasizes that home and school form an 'overlapping effect area' on children's development and academic success. In addition, he stated that the schools which have practices that encourage the strong cooperation of the school, family, and community development the children's academic and social skills thanks to the consistent connections they establish between home and school settings (Epstein, 2001). On the other hand, the administrators (Cohen et al., 2009), the administrative attitude (Büte & Balcı, 2010;Karademir & Ören, 2020), the relationships between teachers (Littrell et al., 1994), and the relationships between children (Thapa et al., 2013) are also determinants of the quality of school climate.
The positive school climate has many positive reflections on children and young people. It affects the students' self-respect (Hoge et al., 1990), and emotional and mental health (Way et al., 2007) in a positive way. In addition, there is a positive relationship between the positive school climate and the students' positive perception of self (Reynolds et al., 1980, cited in Thapa et. al., 2013. There is an adverse relationship between the students' drug addiction and psychiatric problems (LaRusso et al., 2008). The positive school climate increases the motivation for learning and helps to decrease the disadvantages of academic success which arise due to socioeconomic status (Thapa et. al., 2013), the negative sides of violence and aggressiveness, abuse, and rape (Attar-Schwarts, 2009;Thapa et. al., 2013). It has a protective effect on young people's development of positive life and learning (Ortega, Sánchez, Ortega-Rivera, & Viejo, 2011). Furthermore, there is a meaningful relationship between the positive school climate and children's academic and personal development and wellbeing (Haaht et al., 2005). The negative school climate causes low job satisfaction, low creativity, low satisfaction, and low adaptation and paves the way for frustration response (Welsh, 2000). As a result, the positive school climate supports positive behaviors and the negative school climate supports negative behaviors. The settings with negative school climate are strict and oppressive places where the people in the organization are not appreciated and have communication problems (Varol, 1989). It has been indicated that bullying students being educated in those settings have a negative perception of the school climate compared to those who do not bully (Kartal & Bilgin, 2009).
It is thought that determining the quality of the school climate is necessary for many respects. The following statement is included in article 28.2 of the Convention on the Rights of the Child: "States Parties shall take all appropriate measures to ensure that school discipline is administered in a manner consistent with the child's human dignity and conformity with the present Convention." As can be seen in the aforementioned statement, countries that have accepted this convention have the responsibility of protecting the best interests of children and ensuring equality of opportunity in education. For children to benefit better from formal education outcomes, measures should be taken to improve the school environment. One of the ways we can minimize the disadvantages of children living in disadvantaged areas due to their inability to access quality education opportunities is to correctly evaluate the school climate. Because determining the quality of educational institutions helps to identify the wrong points there.
The preschool period is especially valuable as it forms the basis of all other levels. For this reason, it is necessary to determine the quality of pre-school education institutions in order to make early interventions. When the literature is examined, it has been determined that there are some evaluation tools to determine the climate of schools (Booren & Handy, 2009;Cushing et al., 2003;Furlong et al., 1991;Haynes et al., 1994;Yen & Ferrara, 1997;etc.). However, it is seen that almost all of these measurement tools have been developed for primary school level and beyond (Hoy et al., 1991;Spier, 2015). However, although pre-school education institutions have some organizational similarities with educational institutions at other levels, they also have many different features. From this point of view, it is clear that the measurement tools in the literature will be insufficient for a comprehensive evaluation of pre-school education institutions. For this reason, the validity and reliability studies of the PSCS, which was developed to evaluate the school climate of preschool education institutions, are included in this study.

The Present Study
The present study aims to validate the Preschool School Climate Scale (PSCS) for preschool teachers and school administrators. In particular, the purposes of the current study were (a) to test whether the scale is valid and reliable or not and (b) to confirm the scale in a different group.

Participants
School climate includes all the variables that directly and indirectly affect the school system and the relationship between them. From this point of view, it can be said that defining the school climate from a single point of view will offer a limited perspective. Therefore, the participants of this research are school administrators and teachers working in public kindergartens in forty-two provinces of Turkey. The sample of the study was limited to teachers and school administrators because the literacy levels of the personnel working in the schools included in the sample were quite low and some of them were illiterate. 66 of these participants are kindergarten principals and vice-principals, and 844 are preschool teachers. This study was conducted with a total of 910 participants. 854 of the participants are women (94%) and 56 are men (6%). 455 of the participants (50%) are in 20-29 age range, 426 (47%) are in 30-39 age range, 20 (2%) are in 40-49 age range and 9 are (1%) above 50.

Preschool School Climate Scale (PSCS)
PSCS was developed within the scope of the current study. This measurement tool consists of 53 items and 6 sub-dimensions. Response alternatives utilized a 5-point format, never (scored 1) to always (scored 5). PSCS covers teachers and administrators working in pre-school education institutions. The dimensions that constitute the scale are as follows: (1) Adult attitude and relationships: This subscale consists of 13 items. It contains items on understanding the relationship of all adults in the kindergarten system (teachers, administrators, security guards, cooks, cleaning staff, etc.) with children, their families, and each other. This dimension, unlike other dimensions, includes all adults in the school. (e.g., The adults in our school cooperate to improve the school).
(2) Administrative atmosphere: This subscale consists of 11 items. These items are aimed at understanding the attitudes of kindergarten administrators (principal and assistant principal) towards teachers, school personnel, children, and families, as well as their contribution to the school climate. (e.g., The administrators of our school value the teachers).
(3) Peer relationships and the position of the child: This subscale consists of 7 items. These items are aimed at understanding the place of children in the kindergarten climate and the quality of children's relationships with each other. (e.g., The children in our school solve the problems arising from conflicts peacefully).
(4) Teacher attitude: This subscale consists of 9 items. These items are aimed at understanding the attitudes of kindergarten teachers towards children, their families, and their approaches towards school-family cooperation. (e.g.,Teachers in our school are keen on school-family cooperation and family participation).
(5) Physical environment: There are 7 items in this subscale. With these items, it is aimed to obtain information about the cleaning, heating, lighting, materials, and safety of kindergartens. (e.g., Our school is safe for the children).
(6) Curriculum essentials: This dimension consists of 6 items. These items are aimed at obtaining information about the points considered during the planning and implementation of the curriculum used in kindergarten and their implementation. (e.g., We plan our practices of the program by considering the children's development characteristics).

School Climate Scale
School Climate Scale was developed by Canlı, Demirtaş, and Özer (2008)  In this study, the 'leadership and interaction' sub-dimension was referred to examine the validity of the criteria dependent variable. There are six items in this dimension (e.g., 5th item: "Our school principal expresses what he expects from the teachers openly"). The factor loads of the items are between .772 and .645.

Revised School Climate Teacher Survey
Revised School Climate Teacher Survey is the shortened version of the School Climate Teacher Survey developed by Liu, Ding, Berkowitz, and Bier (2014). This scale adapted into Turkish by Yılmaz and Demir (2016) (.75). In this study, the "student behaviors" sub-dimension of the mentioned scale was used to examine criteria-dependent validity.

Procedure
The development process of the Preschool School Climate Scale started with a comprehensive examination of the related literature. As a result of the examinations, it has been realized that the family dimension is mostly excluded from the school climate. However, the family is considered an integral part of the school, especially in the preschool period. For this reason, the scope of a school climate that includes the family was determined and the scale items were formed accordingly. In this process, various measurement tools which measure the school climate regarding different levels of education were also utilized. The measurement tools utilized are The Comprehensive School Climate Inventory (CSCI) (National School Climate Center, 2019), Teacher-student Relationship Scale (Ası & Karabay, 2018), Shortened School Climate Scale for Elementary School Students (Sözer, Yılmaz, & Kasa Ayten, 2018), Extensive School Climate Assessment Scale (Acarbey, 2006), Multi-Dimensional School Climate Perception Scale for Secondary School Students (Karan, 2012), and School Climate Scale (Çalık & Kurt, 2010). The researcher created a 105-items PSCS trial form in the frame of these measurement tools and related academic structure.
The created trial form was presented to an expert in search of remarks. In the frame of the feedbacks received from the experts, necessary improvements of items regarding grammar, spelling, and content were done. Afterward, the form was implemented to 4 teachers and 2 administrators within the pilot study. Thus, the PSCS was given its final draft form which consists of 105 items, and the implementations started.
The data were derived in three stages. In the first stage, the data were derived from 509 people to be able to carry out the confirmatory factor analyses through the same sample and the exploratory factor analysis of the first form with 105 items. In the second stage, 291 participants were reached to be able to test the accuracy of the structure derived as a result of the first test within various samples and time. Finally, 110 people have reached to test the criteria dependent validity for some dimensions of PSCS. The data were collected through the implementation of the printed version of the scale directly to the subjects by the researcher herself and the implementation of the digital version prepared through Google Forms to subjects in preschool teacher groups on WhatsApp, Instagram, and Facebook.

PSCS' Structural Validity
Certain precautions were taken to increase the structure validity. First of all, the prospective items which were prepared based on the literature were analyzed and examined through a pilot study. Afterward, EFA was put into practice in the wake of the first data collection procedure. While deciding on the derived factor number, the scree plot was analyzed and the latent value was determined as >8.0 because of the plateau that originated. The conceptual meaningfulness of the items set in the factors was also evaluated. On the other hand, CFA was implemented to the data used in EFA. Besides, the data were collected once again to confirm the structure at various times and samples and CFA was repeated. The fact that the data derived from both CFA's were compatible with each other revealed that the structure was valid. Finally, the correlation between some subscales of PSCS and subscales related to other scales was analyzed and high positive values were calculated. In sum, all of the procedures carried out confirmed the structure validity.

PSCS' Reliability
The reliability of PSCS and subscales was calculated by using Cronbach alpha reliability coefficient (Cronbach, 1951). Nunnaly (1978) suggested that this value be α≥0.70. It was calculated as α≥0.903 in PSCS.

PSCS' Concurrent and Predictive Validity
To examine the validity of consistency and regression, the relation between two dimensions of PSCS and two different subscales related to them was analyzed. The analysis was carried out by using the calculation of Pearson correlation.

Explanatory factor analysis (EFA)
Two indicators were analyzed to determine if the sample was compatible with the analysis before the exploratory factor analysis. The Kaiser-Meyer-Olkin measure of sampling adequacy index was .941, and Bartlett's test of sphericity was significant, X2(df: 1378, N = 509) = 13241,892 p<.0001, indicating this sample and correlation matrix were appropriate for such an analysis.
The exploratory factor analysis started with 105 items. When the analysis results were examined, 21 factors were determined the Eigenvalue of which is higher than 1. Yet, when the scree plot and variance values and items were analyzed, the researcher decided to develop a structure with 6 factors. The first factor (adult attitude and relationships) was calculates as 15.98 percent; second factor (administrative atmosphere) 14.01 percent; third factor (teacher attitude) 10.7 percent; fourth factor (physical environment) 8.827 percent; fifth factor (peer relationships and the position of the child) 8.77 percent; and sixth factor (curriculum essentials) 8.77 percent. It is seen that the explanatory power of this six-dimension structure is 66.6 percent.
In the item analysis carried out after the Varimax rotation procedure, 52 items the factor load value of which is below .32, and those included in more than one factor were excluded from the scale gradually. Finally, a structure with 53 items and 6 factors was obtained. These emerging factors are named after the information obtained from the literature and the contents of the factors (see Table 1).

Confirmatory factor analysis (CFA)
The accuracy of the six-factor structure was tested by using the IBM SPSS AMOS 26 Graphics program. Another sample group with 291 people was formed to confirm the structure that emerged from EFA in a different sample. The CFA values derived from the analysis of data that were collected from these people are presented in Table 2. The obtained values are as follows: It was indicated that (CMIN/DF) value which points out Chi-square (X2) fairness consistency is 2.060. Kline (2005) states that if the value is ≤ 2.5, this points out "perfect consistency". RMSEA was calculated as. 073. If RMSEA is ≤0.08 it is clear that this is "good consistency" (Hooper et al., 2008). NFI value was calculated as .760; GFI as .676 and CFI as .859. These values indicate "acceptable consistency" (Hayashi, Thomas, Moriya, Rettner, & Parkin, 2008). On the other hand, it is seen that SRMR value is .016. If this value is ≤.05, this points out "perfect consistency" (Brown, 2006). In sum, it is seen that the obtained CFA values confirm the structure that emerged in EFA in a different sample.

Descriptive and inferential statistics
When the descriptive statistics of the sample are analyzed, the average of six subscales are as follows: (1)  When the scale was evaluated in terms of reliability, the Cronbach alpha values of the subdimensions were calculated as follows:(1) adult attitude and relationships is .95, (2) administrative atmosphere is .94, (3) peer relationships and the position of the child is .90, (4) program practices are .91 and total scale value is .96. Cronbach alpha values are above 90. The total correlation of the items ranges from .58 to .83 (Table 1). It can be inferred that the scale meets the requirement of reliability at a high level.

PSCS' concurrent and predictive validity
To test the concurrent and predictive validity of PSCS, all the subdimensions of PSCS, leadership and interaction subdimension of School Climate Scale, and Pearson correlation coefficient of student behaviors subdimensions of Revised School Climate Teacher Scale were calculated (Table 3). When Table 3 is reviewed, it is indicated that the administrative atmosphere dimension of PSCS and leadership and interaction subdimension of School Climate Scale; peer relations dimension and student behaviors subdimensions of Revised School Climate Teacher Scale have positive and meaningful relationships. Whereas it is obvious that there is not a meaningful relationship between the other subdimensions. The present relation between PSCS and subdimensions of other scales support our hypothesis. From this point of view, it is seen that PSCS meets the requirement of concurrent and predictive validity.

Discussion and Conclusions
This study was conducted to test the validity and reliability of the Preschool School Climate Scale (PSCS). The Cronbach alpha values derived from PSCS which consists of 6 factors and 53 items that emerged in exploratory factor analysis indicate that the reliability is at a high level.
The data used in exploratory factor analysis and some of the DFA values conducted first (x2 / df, RMSEA, SRMR, CFI) are excellent adaptive values. It is seen that NFI and CFI values take lower values. The common characteristic of these values is that both values are affected by the size of the sample. The fact that the sample size is not big enough makes it harder for these values to approach 1, which is the ideal value (Hayashi et al., 2008).
To determine whether the PSCS is sufficient to determine the school climate, its compatibility with other scales developed on the same subject was examined. In this evaluation, it is seen that there is a positive and significant relationship between the "administrative atmosphere" dimension of the PSCS and the "leadership and interaction" dimensions of the School Climate Scale. Both dimensions include questions about the administrative practices of school principals and their relationship with other personnel. In addition, it was determined that the "peer relationships and the child's place" subscale of PSCS and the "student behavior" subscale of the Revised School Climate Scale were positively and significantly related. When the items of both subscales were examined, it was determined that they were aimed at determining the relationships among children. The findings show that PSCS works in harmony with other measurement tools accepted in the literature and measures following its purpose.
It is determined that the secondary DFA values derived from a different sample group are quite parallel to primary DFA results. This situation reveals that the factor structure of PSCS supports the structure that emerged in AFA. Since a positive and strong relationship between some subdimensions of PSCS and other scales was indicated, it can be said that PSCS is a valid and reliable measurement tool.
Undoubtedly, the research has some limitations. First, the online collection of data resulted in limited information on the characteristics of existing schools. In future studies, researchers can obtain more comprehensive information by asking more detailed questions about the characteristics of schools. Second, during the development of the scale, data could not be collected from children due to their age and developmental characteristics. This situation caused the information to be obtained about the school climate to contain a limited perspective. To reduce the impact of this limitation, the application of data collection from families will enrich the data. On the other hand, preschool administrators and teachers from 42 of 81 provinces and schools of different socioeconomic levels in Turkey were reached in the study. The aforementioned situation is considered to be a strong aspect of the research. Because the inclusiveness of the sample is high. However, to strengthen this view, it would be useful to verify the measurement tool by using it in different samples. Another limitation is that the scale is applied only to teachers, school principals, and vice-principals. School personnel and families can also participate in the measurement to obtain more comprehensive information about the school climate in future studies.

Article Language
This article is written in American English.