"The real strength in using a data driven decision making process for school improvement is that numbers are objective. The data just don't lie."
Do you agree or disagree with the statement above? Why or why not?
Subscribe to:
Post Comments (Atom)
37 comments:
I agree that numbers don't lie. However, I can't say that all data providing those numbers is always quality, objective or appropriate.
Everyone knows that you can spin data (statistics) in order to fit the goal that you are trying to accomplish. While numbers are objective, the data that they project is not. There are many factors that are not taken into consideration when projecting data.
I agree with Leslie. Although the numbers do not lie, it does not always tell the "real truth". A student may have performed bad on test because he/she had a personal situation on his/her mind. We have tobe careful when using data to drive decisions.
Brad, I agree with you. We all know from Dr. Brown that there is data on pretty such everything. You can find data to support almost anything you want it to. Therefore, if a school is going to focus on data driven decision making, they need to find an effective way to collect accurate, non-biased data.
The data doesn't lie if we are talking about data beyond just test score data. There is a lot of data about teacher quality, retention, working condition surveys, demographics, etc. Putting together all this data along with test score data can create a snapshot of what is going on at a school. I disagree with the statement if the term data is referring to only test scores. How this data is being used for AYP is unfair (all or nothing). Academic Change, which is used for ABCs seems more fair, but I need to know more. Principals and schools are being judged based on the data. We can resist, or try to use it to our advantage.
I am conflicted on this subject. If we are talking about EOC data results, regardless of the situation, the data does not lie. It is what it is and as administrators we have to look at it, make necessary changes, and move on from there; no ifs, ands, or buts. If we are talking about surveys then the data is interpretative to individual characteristics of the person filling out the survey which can skew the results.
I partly agree with you Marcia that data on test scores is what it is. My point is that we must include other forms of data, including data that is qualitative. Informal surveys may not necessarily be valid or reliable, but perceptions are an important piece of data that cannot be overlooked. Surveys can be scored, analyzed and reveal a great deal of valuable information about a school. I would argue that qualitative data is just as valuable as quantitative data and they both need to be included in forming the SIP.
Nothing is ever simple, is it?
I would add to the discussion by saying that I think data is crucial to school improvement-- to proceed without some validation of what needs fixing other than my perception seems foolhardy to me. On the other hand, the type of data, how data was collected, the interpretation of the data, etc are all aspects to look carefully at to be sure the data is worth building a plan on. You can always revise later if the plan doesn't help--that is part of the experimental process and we aren't perfect.
One way to strengthen data is by triangulation, in other words, finding more than one source of data. For example, I often gave my own reading assessment that was made by a different company than other assessments used by the school. I would look at this, the EOG scores, and the work the student did in class and in seminar to give a final judgment to parents on his/her reading comprehension or growth. If the use of the data is important, I never put stock in just one assessment or one type of data if I can help it. As Lisa mentioned, there are always a few numbers that aren't right--as in the case of the student who was sick during the test.
I would include a good scientist in looking at the data as well. They are so much better at that type of thing.
I believe that data analysis is a hugely important tool when looking at a school. While (like Brad), I believe that data can be "spun" in multiple ways, I believe that even that conversation about interpretation IS A GOOD THING. Debating the meaning of data to determine how our school is doing is a better conversation in my mind than debating how we "feel" about how we are doing as a school.
I think Greg makes a good point here, and we should go further as administrators and ask people to think of all possible interpretations of the data we are looking at. It should also be asked where we got it, who got it and how, etc.
I think looking at a data profile of a school very much helps in thinking about what it needs or doesn't need. It may keep us focused too. I guess we might want to decide what kinds of data we want to include in that profile, though. I hope more than eog results... Is there a good recommendation for what to look at somewhere?
Data analysis is so important that I think school administrators and teachers need to be well-versed in it. But I would like to go one step farther: I think it would be very useful to have a part-time statistician at the school who can run all sorts of statistical analyses that would be useful for both administrators and teachers. For example, I could see the statistician running all sorts of correlations/regressions: English I grades vs. English I EOC scores, benchmark scores vs. EOC scores, SAT math scores vs. physics grades, student's Algebra I grades vs. geometry grades, etc. The correlation coefficients from these types of analyses, that teachers don't have time to do, could yield some very important information.
I believe that data collection skills can determine the effectiveness of its use, and if directed appropriately at the item being considered will provide indisputable information. One thing that a principal warned me of in regrade to data is the "use it to determine patterns, and not use it to pinpoint specific weaknesses." As an evaluative tool it can help guide our work, but I think it must be coupled with other sources of information if change will occur to address those things being looked at.
I agree with those who advocate breaking down the data and looking for relationships, correlations, etc. That is a very good idea and I like the idea of having access to someone who can run those numbers. Perhaps we will get to the point where we can get software that puts all those pieces together and school staff/administrators have access to it. I still disagree with those that are dismissing data about perceptions. What about how students perceive a school? How students perceive his or her learning, teachers, etc... How teachers perceive his or her students, school culture, etc..If we could see patterns in a school, it could be one way to connect some dots regarding student performance. If 60% of a staff says that a school harbors low expectations for students, are we to ignore it or take it seriously? I would argue for the latter. I am not suggesting that we overlook test scores, but we can't overlook the relationships, culture, climate, etc.
Lisa threw out an interesting term: "the real truth".
So what is the real truth? As school administrators, how do we find it out? Do different types of data -- EOG data, survey data, enrollment data, etc. -- represent more or less effective means of ascertaining the real truth?
Does it even exist?
Greg, I like your idea of having a statistician on site to run data. I would caution the way the different types of data analysis is used though. My principal decided to run the number of A,B,C,D, and F given by each teacher in each subject to all of the teachers and it made people VERY uncomfortable. It might have worked better if the principal used that information when talking individually to a teacher instead of everyone in the department/school knowing about it.
How often is a student’s final numerical grade within five points of their end of course test score in any subject? Maybe I lack some skills in teaching but I never used student grade averages in my class as an indicator of how a student would score on the EOC. While I believe data is one of the biggest things an administrator can use to improve their school it is not the only thing. As far as “the real truth” is concerned I believe many people look at data to find ways to improve scores. What administrators should be looking at is how to use data to support and empower teachers, which in return will improve test scores.
Along the same lines, I wonder how often we focus on an issue and analyze existing data for answers instead of working in the reverse to identify an issue and create data capture that directly aligns with our needs. (It sounds like Robin uses both approaches in her classroom.) I sometimes question whether or not we try to gain too many answers from one piece of data ... perhaps this is ok is some instances?!
I agree with Greg, etc. that having a statistics "consultant" on hand would be very helpful in helping a specific school frame data interpretation in a way that addresses school needs. Does this mean that we should "tweak the data?" Absolutely not. But I do think that what a school site does with the numbers depends on their long term objectives. That said, educators need to strongly consider leslie's question if single-source data collection yields an accurate portrait of a school community. I don't think it does.
We recetly gave End-of-the-quarter benchmark tests to all students in grades three through eight. Most of the grade levles did well except grade three. It wasn't that the teachers had not done a great job teaching the material or that the students did not grasp the concepts, but it was their first time taking a bubble test. As an administrator, we have to be congnizant of many factors that might affect data results. The lesson learned in this situation is that our third grade teachers need to teach the students test taking skills.
With the remaining grade levels, we were able to pinpoint areas of weaknesses that grade levels and individual classrooms needed to improve. Data is important and it can drive decision making but, we have to make sure that we think about all the whole child when making these decisions.
There is an old saying, "It's all in the eyes of the beholder." I agree with many of you that anyone can "spin" data to make it fit their agenda. Even though, End of Course tests are culturally biased, they are culturally biased for everyone. There are schools performing at high levels across the country with the same demographics as the schools that are failing. Therefore, if a school is not meeting NCLB standards there must be a reason. EOC test score data paints a picture of a school's strengths and weaknesses. It is important to celebrate the strengths and seek to improve upon the weaknesses through a School Improvement Plan.
In my opinion, an effective principal uses the EOC test score data to initiate a conversation into the reasons for the outcomes. The data provides the topic for discussion, but the ideas and strategies brought forth by teachers in faculty meetings should drive the goals of the School Improvement Plan. Without the numbers, your school has no idea where it needs improvement.
Dr. Graham, I do think it is possible to discover the "real truth" behind the data. One of the greatest resources that administrators have at their hands is the knowledge and creativity of their teachers. If we are talking about EOC data or enrollment data, the effective principal presents the data to his or her faculty and then facilitates a discussion to find out the "real truth" behind the data. Teachers can often come up with alternative solutions that the principal would have never thought of on his or her own.
I also think the Teacher Working Conditions survey results are very revealing. A principal has to reflect on this data to find the "real truth." Once he has an idea of the problems, he has to look in the mirror to figure out what he is going to do differently. Does the school culture need to be improved? Do my teachers enjoy coming to work every day? Do my teachers feel supported by the administrative team? In my opinion, the principal must be reflective and willing to learn from failure in order to be successful. The TWC survey results provide valuable data for every school administrator. The mediocre principals will complain, the effective principals will seek to change and improve.
Yes, Lisa! Hard data must be interpreted in the frame of the whole child. But is it possible?
In the early childhood program, from the very first class I took, the importance of multiple sources of data in assessment to accurately portray the whole child, has been impressed upon me. Especially in early childhood, where the sources of the data cannot speak for themselves, standardized data can often be the single basis for making decisions for very young children and their families. I believe that longitudinal observations and running records are other reliable sources of data that give us more extensive information than just a bubble sheet or a bell curve alone. Major decisions made for most ANYTHING--children, schools,teachers--should be based upon a compliation of MULTIPLE sources of empirical and longitudinal data.
Yes, numbers do provide administrators a tool for creating or implementing programs; however, it is necessary to understand many researchers can create data that can agree or disagree with anything they wish to prove or disprove. The problem is not in the collection of data, but in keeping the data and utilizing the data longer that a few years. This is very difficult when schools have a huge turnaround. Longitudinal studies are, of course, the best studies; however, they are difficult to monitor for administrators.
Though it seems we can pretty much agree on the need to collect data, my concern is on what data collection tool is best. We are forced to look at EOG scores for data and sometimes we can measure “growth” as compared to prior test scores; furthermore, there are even other programs were we can use to manipulate percentages using the data, but are we using the best measurement tools? Currently we use NOVA, other stat programs, or sample studies within our schools or districts, but how do they compare to those with similar sample populations? What tools do other states use that are making successful progress in their districts and how can we implement them, correctly, into our schools? I just want to make sure that I will be using the data for what it is intended, improving student learning from primary grades to the time they graduate.
Sharon, I agree wtih you . We need to make sure that data is being used to improve student learning. I've seen data used to "prove" lots of things that we already knew, prior to having data collected. But if a school really has the students best interest in mind then data can be a very effective tool. The example of Lemon Middle School from our reading is a good example of effective use of data.
I agree with Ryan when he talks about using the data with your staff to find the truths within it, but it is also important to recognize that data can be collected by anyone for any reason. If you set out to prove something, either positive or negative, you can find the numbers to support your thesis - in this way it is sometimes tricky to discern what really is "truthful" and what may be skewed to suit a particular angle. I believe that using it for reflection and to find patterns, data is an effective tool for evaluation, but as the "whole truth" about anything, I believe more than just data is needed.
I believe that while numbers don't lie as it relates to data collection, it may not be that "real truth" Lisa mentioned. Lisa mentioned that sometimes students take test and score badly due to personal siutations and cirucumstances. I do believe in data because it gives teachers and idea of what needs to be focused on.
I agree with Ryan as it relates to Dr. Graham's question regarding the real truth. When handed data, a principal now has to go back to his or her staff and discuss plus and minuses of the data. From that conversation, teachers reveal the real problems (if the data shows any) they are facing and a principal now has the obilgation to find out how they as a school can improve.
I agree Greg there should be a statitican in schools, but I thought that position was already created at the district level? And wouldn't it be more effective if the principal or someone from the administrative staff were trained to be that? What if it was a teachers or teachers from core subjects? So not only could they anaylze the data and report back, but they could also come to the table with strategies teachers could use. Of course I know that teachers already do enough now, but when are we going to stop giving excuses for why we can't doing something because of time and realize we can't afford any more time wasted on saying the "can'ts"...just a thought
I agree with others, sure the data and numbers may not lie, but what about the person putting the spin on those numbers?
I believe that numbers are objective but it's the presentation of the numbers that gets subjective. As a school administrator, you have to be skilled in determining what the data means, how it affects your students and what the next step is. The "real truth" can be approached when multiple assessments are used to gather lots of different data about our students. Then we begin to see the whole picture.
Do you all think a teacher in the school would be impartial enough to also be the statitician.... especially since we believe that the data outcome can be manipulated to prove anything.
Bonita,
I think it all depends!!! If teachers are using the information to inform their own instruction/strategies in the classrooms, yes. But when this data will be shared with those in supervisory positions, I think it becomes a little more complicated. I also think having a teacher serve as a statitician would create many conflicts with the rest of the staff. This means that one teacher would be privy to every other teacher's data. That might lead to the teacher being placed in an awkward position with the rest of the staff.
I agree that one teacher does not need to become the one who is in charge of the data. Currently at my school we use a software program called Think Gate. The data used by this program is generated once a 9 weeks when our students take benchmark tests. Each teacher is privy to their own data. They can manipulate it any way you could imagine. This is helpful to the individual teacher because it allows them to look at their students areas of growth. It then allows them the chance to decide what to re-teach. From there the teacher talks to their grade level peer about the topic and a reteaching plan is created. Our school administration and our CF has access to all of the data and use it to drive discussions at grade level and content meeting. used in this way, having access to data is incredibly helpful.
Numbers can be guided in order to tell ‘the real truth’ we want them to say. We do know that the person collecting the data can be biased for several reasons and direct results in a certain way. It is true though, that if data collecting is sincere, the data can be a wise counselor to the administrator.
Even student’s test results can be influenced by other external problems or facts. These will show negatively on a numbers report that will not take into consideration any other qualitative report facts.
About the comment of a statistician, I feel that sometimes you don’t need an expert, or a research that will check the existence of the obvious. An administrator can see and feel the reality in a school without needing to see a statistical report. This is needed though in order to make comparisons between different schools.
Post a Comment