What is a quantitative observation you might make in a school cafeteria?
Systematic classroom observation is a quantitative method of measuring classroom behaviors from direct observations that specifies both the events or behaviors that are to be observed and how they are to be recorded. Generally, the data that is collected from this procedure focuses on the frequency with which specific behaviors or types of behavior occurred in the classroom and measures their duration. There are several elements that are common to most observational systems.a purpose for the observationoperational definitions of all the observed behaviorstraining procedures for observersa specific observational focusa settinga unit of timean observation schedulea method to record the dataa method to process and analyze data (Stallings and Mohlman, pp. 469-471)Prior to the use of systematic observational methods, research on effective teaching typically consisted of subjective data based on personal and anecdotal accounts of effective teaching. In order to develop a scientific basis for teaching, researchers began to use the more objective and reliable measures of systematic classroom observation. In the last quarter of the twentieth century, several hundred different observational systems have been developed and used in classrooms. There have similarly been hundreds of studies that have used classroom observation systems since the 1970s.Although there are several types of observational procedures or techniques that have been used to examine effective teaching (e.g., charts, rating scales, checklists, and narrative descriptions), the most widely used procedure or research method has been systematic classroom observation based on interactive coding systems. These interactive coding systems allow the observer to record nearly everything that students and teachers do during a given time interval. These interaction systems are very objective and typically do not require the observer to make any high inferences or judgments about the behaviors they observe in the classroom. In other words, these low-inference observational systems provide specific and easy identifiable behaviors that observers can easily code. Some of the more commonly used observation instruments are the Brophy-Good Dyadic Interaction System, Stallings Observation System, and the Classroom Observation Schedule. They all have been widely used in research studies and in teacher development projects designed to improve classroom instruction.Some of the major strengths of using classroom observation allow educators to do the following: (1) permit researchers to study the processes of education in naturalistic settings; (2) provide more detailed and precise evidence than other data sources; and (3) stimulate change and verify that the change occurred. The descriptions of instructional events that are provided by this method have also been found to lead to improved understanding and better models for improving teaching.A final strength of this research method is that the findings from these observational studies have provided a coherent, well-substantiated knowledge base about effective instruction. Many of the reviews and summaries of the classroom observation research, such as that of Herb Walberg (1991, 1995), have consistently found that a number of classroom behaviors significantly relate to students' academic achievement. Several aspects of classroom instruction such as conducting daily reviews, presenting new material, conducting guided practice, providing feedback and correctives, conducting independent practice, and conducting weekly and monthly reviews have been found to be significantly related to students' academic achievement. In other words, research using systematic classroom observation has provided us with a substantial knowledge base that has helped us understand effective teaching.Purposes of Classroom ObservationClassroom observation has many valid and important educational purposes. This section summarizes three important purposes or areas where systematic classroom observation has been widely used: (1) description of instructional practices; (2) investigation of instructional inequities for different groups of students; and (3) improvement of teachers' classroom instruction based on feedback from individual classroom or school profiles. Description of instructional processes. One of the fundamental purposes of classroom observation research is describing the current status of instructional practices and identifying instructional problems. As Tom Good puts it, "one role of observational research is to describe what takes place in classrooms in order to delineate the complex practical issues that confront practitioners" (p. 337). There have been many observational studies that have been specifically designed to describe specific educational phenomena. Large-scale observational studies such as Ken Sirotnik and Hersh Waxman, Shwu-Yong Huang, and Yolanda Padrón, for example, have examined instructional practices in elementary and secondary schools. Sirotnik examined 1,000 elementary and secondary classrooms and found that there was very little variety in teaching practices across subjects and grades. He found that the majority of class time was spent either with the teacher lecturing to the class or students working on written assignments. Waxman, Huang, and Padrón observed ninety sixth-grade and eighth-grade classrooms from sixteen inner-city middle level schools and found similar results to those of Sirotnik. Students were typically involved in whole-class instruction and not interacting with either their teacher or other students. Students rarely selected their own instructional activities, and they were generally very passive in the classroom, often just watching or listening to the teacher, even though they were found to be on task about 94 percent of the time. The teacher observation results revealed that teachers typically focused on the content of the task or assignment, responded to students' signals, communicated the task's procedures, and checked students' work. Teachers were observed spending very little time interacting with students regarding personal issues, encouraging students to succeed, showing personal regard for students, and showing interest in students' work.Another example of descriptive, observational studies involves the extent to which technology is used in the classroom. Although there have been a large number of studies that have examined technology use in schools, most of these studies have relied on self-report data from administrators, teachers, or students. These types of data, however, are often unreliable and tend to be upwardly biased in the direction of over-reporting the actual amount of technology use. Therefore, it is important to observe the actual extent to which technology is used in classrooms and to look specifically at the technology used in classroom and used by individual students. In one such study, Waxman and Huang (1995) used systematic classroom observation to examine the extent to which computer technology was integrated into the curriculum of 200 elementary and secondary school inner-city classrooms. They found that there was no integration (i.e., use) of computer technology in the elementary school classrooms, and that students were observed working with computers only 2 percent of class time in middle school classrooms. Huang and Waxman (1996) also conducted systematic observations of 1315 middle school students from 220 mathematics classrooms in order to examine the amount of technology used. The descriptive results revealed that students were observed using calculators about 25 percent of class time, but they used computers less than 1 percent of class time in their mathematics classes.Some other uses of descriptive observational studies have been to evaluate programs and more specifically, to evaluate the fidelity or degree of implementation of programs; to examine the extent to which higher-level thought processes are emphasized in schools; and to investigate the extent to which multicultural education is emphasized in urban classrooms. A final important use involves school effectiveness studies, such as Waxman and colleagues 1997 study, where classroom observation data have been used to investigate observable differences between effective and ineffective schools. Waxman and Huang (1997), for example, observed more than 700 students from four effective and four ineffective urban elementary schools that served predominantly African-American students and found that significantly more students from the effective schools were observed working in an individualized setting, interacting with their teacher, and working on written assignments. On the other hand, students from the ineffective schools observed in whole-class settings were found interacting with their teacher, interacting with others, reading, and working with manipulative materials significantly less than students from the effective schools.Investigation of instructional inequities. Several studies, such as that of Elizabeth Fennema and Penelope Peterson, have found that some groups or types of students are treated differently by teachers in classrooms, and that these inequitable patterns of teacher-student interaction in classrooms result in differential learning outcomes for students. There have been many studies, for example, that have found gender imbalances in teachers' interaction patterns in the classroom. Jere Brophy and Tom Good's 1974 review of the research found that consistent sex-related differences exist in the classroom in teachers' interaction patterns. Boys, for example, typically have been found to receive more praise and criticism in the classroom than girls. They also found that teachers have more behavioral, procedural, and academic interactions with boys than girls. Boys have also been found to ask more questions in the classroom, and teachers have been found to ask boys more questions. Good and his colleagues (1987,1988) have also conducted several observational studies that examined why low-achieving students in secondary schools ask fewer questions than high-achieving students. They also found that students from an upper-middle-class elementary school asked more questions than students from lower-middle-class schools.Other studies have looked at both sex-and ethnic-related differences in the classroom. Hart examined the relationship between teacher-students interaction and mathematics achievement by race and sex. She found the following differences: (1) white and black male students had more classroom interactions than students from other groups; (2) a disparity in the type of interaction between white and black students; and (3) boys were involved in more public interactions with teachers than girls. In other words, it appears that patterns of teacher-student interaction may not only be influenced by the sex of the student, but also by the ethnicity of the student.Padrón, Waxman, and Huang observed student behavior differences between resilient (i.e., successful) and nonresilient (i.e., less successful) elementary school students from low socioeconomic backgrounds. They found resilient students spent significantly more time interacting with teachers for instructional purposes, whereas nonresilient students spent more time interacting with other students for social or personal purposes. Resilient students were also observed watching or listening significantly more often than nonresilient students, whereas the latter were observed more often not attending to task. The percentage of time that resilient students were observed on task (85%) was much higher than that of nonresilient students (61%). The magnitude of these differences was both statistically and educationally significant and illustrates the instructional inequities that exist within classrooms.Improvement of teaching practices. Research using observational methods has yielded important information that has practical implications for the improvement of teaching practices. One of the traditional problems hindering teachers' classroom instruction has been the lack of valid and accurate information that teachers could use in order to facilitate their professional growth. Many teachers, even experienced ones, are not always aware of the nature of their interactions with individual students. Consequently, one of the most important purposes of systematic classroom observation is to improve teachers' classroom instruction. Feedback from individual classroom profiles derived from systematic observations has been found to help teachers understand their own strengths and weaknesses, and have consequently enabled them to significantly improve their instruction. Through feedback, teachers can become aware of how their classroom functions and thus bring about changes they desire. This process typically involves having trained observers systematically observe teachers and their students in their classrooms and later providing teachers with information about their instruction in clinical sessions. This approach is based on the assumption that teachers value accurate information that they can use to improve their instruction.There is growing evidence that feedback from systematic observations can be used to improve teaching. Several studies, such as that of Jane Stallings, have found that teachers could positively change their attitude and behaviors toward pupils after receiving feedback from classroom observations. Good and Brophy's 1974 "treatment study" exemplifies this type of research. In that study, teachers were given feedback based on forty hours of classroom observation. As a result of this "one-shot" interview where feedback was given, teachers' interaction patterns changed, and their attitudes toward individual students changed, too. Stallings, Howard Ebmeier, Good, and Good and Douglas Grouws have utilized similar strategies in other projects. In those studies, teachers were presented with individual feedback regarding their classroom instruction and then were found to change their behavior in desirable ways. All these studies have found that teachers can improve their classroom instruction given appropriate feedback and suggestions for improvement.The overall findings from these studies suggest that feedback from classroom observations is a viable and effective mechanism for providing teachers with the information they need about their classroom behavior. This feedback is intended to create an "imbalance" in teachers' perceptions of their own behaviors. This imbalance exists whenever teachers find out that their attitudes or perceptions of their teaching differ from that of trained observers. Teachers in such a state of "imbalance" are motivated to do something about their behavior in order to restore themselves to a balanced condition. A similar notion is that self-awareness increases teachers' control of their actions and the possibility that they will modify them. In 1995 Waxman, Huang, and Padrón provided schoolwide feedback to middle school teachers that compared their school profile on classroom instructional behaviors to an overall districtwide average of these same behaviors. Feedback from these profiles was used to stimulate dialogue and discussion about instructional strengths and weaknesses in the school. The profiles also helped initiate discussion about specific instructional areas that needed to be improved in the school. It should be pointed out that these profiles provided some guidelines for practice, and they were not attempts to tell teachers what to do. These profiles provide teachers with concepts and criteria that they can use to reflect about their own teaching. The feedback session was not viewed as one where research findings should be applied into specific rules or guidelines for teachers to follow. Rather, the observational feedback was intended to be used as a guide for teachers with which they and their colleagues could reflect about their practices on their own and decide what action to take. Professional services and university courses are some of the possibilities that teachers could choose if they wanted to continue to collaborate with the researchers in order to help them improve their instruction. In summary, the use of feedback from classroom observations appears to be a potent strategy that can improve instructional behaviors in specific classrooms and schools.Limitations of Classroom ObservationThere have also been several criticisms and cautions related to the use of structured observation techniques, according to Sara Delamont and David Hamilton. The criticisms and limitations of using structured observation techniques are categorized into three subsections: (1) theoretical and epistemological criticisms; (2) methodological concerns; and (3) pragmatic concerns. This section also includes a brief discussion of the implications of classroom observation and some new directions. Theoretical and epistemological criticisms. Although observational research has produced a substantial body of important findings that can lead to improved teaching practices, there is still a lack of consensus or lack of confidence regarding the research. There have been many theoretical and epistemological criticisms of classroom observational, process-product research such as that of Maurice Galton in 1988. Several critics, for example, have argued that this research is devoid of theory and consequently cannot explain why some instructional behaviors impact student outcomes. There are also related concerns about why some variables are selected to be observed at the exclusion of other variables. Because there is no model or theory behind the research, the critics argue that there is no justification for the selection of variables or meaningfulness associated with the interpretation of results. They further argue that the selection of events or behaviors may not be clear to anyone except the observer or instrument developer. In other words, classroom observation research has not dealt with the theoretical assumptions of why a particular style of teaching or set of instructional variables influences student learning.Tom Popkewitz, Robert Tabachnick, and Kenneth Zeichner (1979) state that this research approach has a behaviorist orientation that maintains "it is possible to identify, control, and manipulate specific outcomes of teaching by altering selected aspects of a teacher's overt behavior" (p. 52). They further contend that teaching is viewed, "as the sum of discrete behaviors and a change in one or several of these behaviors is assumed to affect the quality of teaching as a whole" (p. 52). Their most strenuous argument, however, concerns the notion that these teaching behaviors "are often viewed independent of the curricular context with which the techniques are associated" (p. 52). They are concerned that observers generally focus on isolated behaviors, without concern for the preceding and subsequent behaviors that they feel provide the context and meaning of the behavior. Another concern is that most observational systems are generally limited-they can be used only to observe covert behavior that can be quantitatively measured. Furthermore, these observational systems make it difficult to record complex instructional behaviors.Methodological concerns. Most observational techniques have limitations. Some of these concerns or limitations are related to methodological issues that can interfere with the drawing of valid conclusions. One of the primary methodological concerns or source of invalidity that needs to be addressed regarding the use of systematic observational techniques relates to the obtrusiveness of the technique. Observer effects may occur because teachers and students are aware that their behaviors are being observed. The presence of an observer may change teacher or student behaviors, perhaps resulting in reactive effects. Teacher anxiety or teachers performing less well than usual can interfere with the drawing of valid inferences about what normally occurs in the classroom. On the other hand, there is also some evidence that indicates that teachers' instruction may be slightly better than usual when they are being observed. Although some researchers like Donald Medley, Homer Coker, and Robert Soar maintain that observer effects are not serious concerns, the possibility that this threatens the validity and reliability of data collected exists.There are a number of methodological concerns that similarly need to be addressed. The reliability and validity of observational systems is a primary concern. Although many systems report inter-rater agreement or observer accuracy, they do not specify the reliability as it pertains to stability of teacher behavior or on the internal consistency of the scale. Validity is another important concern that needs to be addressed. Construct validity, for example, which focuses on the "theoretical integrity" of the behaviors, is particularly important. Criterion-related validity, or the extent to which the observational measures relate to a criterion measure, is rarely reported, and concurrent validity or the extent to which a particular instrument is related to other instruments is generally missing too.There are other methodological concerns that are related to the actual amount of time that is necessary to obtain a valid observation period, as well as the appropriate number of observations that are required in order to obtain reliable and valid measures of instruction. Similarly, there are a number of methodological concerns related to the analyses of data. Most of these concerns address the issue of what the appropriate level of analysis (e.g., student, the class, or students within class) should be used when analyzing the observation data. Students are nested within classrooms, while classrooms are nested within schools. Prior teacher effectiveness research has often aggregated data to classroom-level analyses that may underestimate the importance of processes within classes because all the within-class variation is lost. Recent analytic developments, such as hierarchical linear modeling (HLM), allow researchers to disentangle these nested effects and investigate hypotheses about the effects of within-and between-school or class factors on classroom instruction or students' perceptions of their learning environments. Advanced statistical models, such as HLM, allow researchers to identify and separate individual effects from group effects, after statistically controlling for other explanatory variables. Such multilevel models can estimate how group-level variables (e.g., characteristics of the classroom or school) influence the way in which individual-level variables (e.g., students' classroom behavior) affect students' achievement.Ads by GoogleAnother concern related to prior classroom observation research is that it has typically been generic (i.e., generalizing across grade levels and content areas), rather than focusing on a given grade level and/or subject area. Similarly, the content of the lesson is often neglected as has the quality of the interaction that is being recorded.Pragmatic concerns. A final category of limitations related to classroom observation are pragmatic concerns that focus on the practicality of conducting observational research. One of the primary pragmatic concerns of observation research is that it is costly to do because it requires extensive training and time. Some training programs for observers, for example, require as much as seven full days of intensive training before the observations are conducted in classrooms. Gaining access to schools and classrooms to conduct observations is another serious concern. Many school districts are reluctant to allow observation of teachers in their schools because they feel it would be too disruptive to the learning environment. Teachers have also been known to dramatically alter their instruction when observers are present in the classroom.Another pragmatic concern relates to the misuse of classroom observation data. Classroom observations can be very useful as a formative evaluation procedure, but should not be used to provide summative decisions, such as those regarding a teacher's dismissal or rehiring. Similarly, classroom observations should not be tied to summative decisions like salary increases. Unfortunately, several school districts and state departments of education have misused observational research and translated findings into specific rules or standards that they have used in developing evaluation instruments. These misuses are more "accidents" of the research, however, rather than problems associated with the "essence" of the research.The previously mentioned criticisms and limitations, however, do not necessarily detract from the value and utility of the observational method. Many of these criticisms are incidental aspects of some observational research. Nate Gage and Margaret Needels and others, for example, have refuted many of these criticisms and have provided several examples of how observation research has contributed to instructional theories. Medley has also argued that the previous methodological limitations of observational research were greatly reduced in the 1990s. He points out, for example, the impact that the laptop computer will have on classroom observation research. In addition to replacing traditional clipboards and stopwatches, the laptop computer will aid the precision and accuracy of researchers in recording events, as well as provide a detailed account of contextual items that occur during the observation.New DirectionsIt is important to point out again that no one data source or methodology will sufficiently answer all critical educational questions. Multiple measures or indicators of instruction are needed to help capture a more comprehensive picture of what goes on in classrooms. Some of the new directions for classroom observation research include the following: (1) combining both qualitative and quantitative methods in observation systems; (2) developing observation instruments that are based on "standards" of pedagogy; (3) using student-centered observation instruments that allow for comparisons between groups of students within the class; and (4) using instruments that assess authentic, interactive instructional practices that have been found to relate to student gains on higher-level cognitive outcomes.
