Thursday, July 9, 2009

Data Driven Decision Making

Data Driven Decision Making

Belinda Murphy Curriculum Evaluation
July 9, 2009 Dr. Jay Dugan

What is it?

Data-driven decision-making (DDDM) is a system of teaching and management practices that enables classroom teachers to obtain more accurate information about their students. It utilizes background information and student assessment data when decisions are made for planning and implementing instructional strategies at the district, school, classroom, and individual student levels. Some educators dislike the idea of DDDM because of its connection with the federal No Child Left Behind Act (NCLB). This is a shame, because numerous school districts across the country are seeing substantial improvements in student achievement and learning as they incorporate data-driven practices. Teachers in these schools are finding that pervasive and intelligent uses of data can significantly improve their instructional interventions for students, increase their feelings of professional fulfillment and job satisfaction, and also re-energize their enthusiasm for teaching.
Teachers must make an important paradigm shift if they are going to incorporate data-driven decision-making into their day-to-day instruction that is dedicated to the achievement of results, with an emphasis on the delivery and process to instructional pedagogy. Teachers and other instructional support staff will need extensive professional development and training in order to adopt data-driven approaches successfully. (McLeod, S., 2007)

Why do we need it?

“School reform is the ultimate goal of school reform laws and the rules, policies, and procedures for implementing them. Federal and many state laws require schools to have school improvement plans and to set goals to improve student achievement of standards. Goals for improvement are based on state and local assessment results and the indicator systems of which they are a part. These results reveal overall learning, conditions that affect information; the school determines what needs to be improved, who needs to improve, and how that improvement might be accomplished.” (ael.org/dbdm)

Essential Concepts

Educators need to understand the differences between NCLB and DDDM. NCLB is about accountability to the federal government for the educational funds sent to states. “Data-driven decision-making is about getting better information into the hands of classroom instructors; Educators should be careful not to reject DDDM practices and principles, which have been shown to have positive impacts on student learning and achievement gaps, because they are angry about federal and state NCLB implementation decisions. Data-driven activities existed in some schools long before NCLB was passed and will continue in many schools regardless of what happens with the federal legislation.” (McLeod, S., 2007)

Data-driven educators should be able to articulate the essential elements of effective data-driven education outlined in the diagram below. The five major elements of data-driven instruction are:

· good baseline data,
· measurable instructional goals,
· frequent formative assessment,
· professional learning communities, and
· focused instructional interventions.

These elements interact to enhance student learning and to inform teacher practice.

When assessing student and school success, teachers must understand the importance of utilizing various indicators and multiple measures. (Bernhardt, 2004). For example, data from a single administration of a statewide mathematics test does not give teachers the specific information they need in order to improve student learning. Measures of student engagement, information from other assessments, previous interventions, and other data are needed for teachers to design appropriate instructional interventions. Likewise, using a single formative assessment to measure students’ mathematics progress is not as reliable as using several different assessments to determine students’ mathematical understanding. Teachers who are data-driven need to be very careful when reviewing the summative assessment data from yearly state tests. They need to understand how and when the data can or can't be helpful.

Principals can support this phase of the DDDM process by helping staff envision what good data-driven education looks like in practice and by helping teachers understand the five essential elements. Building staff and organizational DDDM capacity takes time, just like any other school reform initiative. Principals need to assist district personnel in the creation and implementation of a comprehensive, long-term professional development plan that is designed to ground teachers in the skills they need to be effective data-driven instructors. (McLeod, S., 2007)

Collecting and Analyzing Summative Data

Data-driven school organizations require teachers to utilize data from yearly summative assessments to improve student learning. In order for this to happen, teachers need to be able to get their hands on the data from yearly summative assessments that will help them improve instructional practice. They already proctor those tests; they also should be able to get relevant summative test data out of district data management and analysis systems for baseline analytical and reporting purposes. “Access to the raw data is crucial, because educators invariably want more detailed data, or want data presented in different ways, than paper reports typically provide.” (McLeod, S., 2007)

Teachers should work with their administrators to select key indicators of success for their classrooms, once they have access to good baseline information. In order to do this, they need to be well-versed in assessment literacy concepts so that they can appropriately interpret summative baseline data. Teachers also need to give continuous feedback to administrators about the effectiveness of the data and/or reports that they are receiving.

Principals should check to be sure that the data teachers receive is in a format that can be useful for classroom instruction, is accurate, and is given to them in a timely manner. Principals should also help district personnel in the designing and implementation of data systems that allow for exploration and reporting of raw data. “Most importantly, building-level administrators must actively help teachers identify key indicators of classroom success, appropriately analyze their data, and then turn those data into strategic pedagogical interventions.” (McLeod, S., 2007)

Setting Measurable Goals

Teachers can use specific baseline data to identify learning needs and mastery levels of various classes, individual students, and demographic subgroups, once armed with key summative indicators of classroom success. They can then use that information to set measurable year-end instructional goals. “These goals are often referred to as SMART goals. The acronym stands for Specific, Measurable, Attainable, Results-Oriented, and Time-Bound. An example SMART goal might look something like the following:

The percentage of third grade students scoring at Level 3 or higher on the state mathematics test will increase from 64% in Spring 2004 to 82% in Spring 2005.

Focus areas for improvement
1. Number sense
2. Computation
3. Measurement

Data-driven educators recognize that formalized goal-setting can lead to improved student learning outcomes. All SMART goals created by teachers and administrators should have the following six components (with example language from the SMART goal above):

1. A measurable baseline (64%);
2. A measurable target (82%);
3. A specific time frame (Spring 2004 to Spring 2005);
4. Specificity about what is being assessed (percentage of third grade students scoring at Level 3 or higher);
5. Specificity about the method of assessment (the state mathematics test); and
6. Focus areas that guide future action needed to reach the learning target (number sense, computation, and measurement).

Inclusion of these six components ensures that SMART goals meet the criteria represented by the acronym. SMART goals can be used with common assessments, teacher-made rubrics, and other types of assessments as well as with standardized tests from publishing companies and state departments.” (McLeod, S., 2007)

Teachers should then be able to identify a few key instructional goals each year. Teachers often are overwhelmed by the plethora of learning needs present in their classrooms and must fight the urge to create too many goals. Teacher goal-setting should address instructional areas that are both relevant and strategic. “Evidence from successful data-driven schools shows that strategic focus and success in a couple of key areas commonly carries over and alleviates other instructional and behavioral concerns as well.” (McLeod, S., 2007)

Principals should model and support the goal-setting process. Administrators’ goal statements might focus on factors such as students’ level of engagement with the teaching-learning process, discipline, attendance, and student learning objectives. Organizational goals should be focused on essential school needs and should be mentioned often, especially with parents, students, and faculty. Administrators also should actively assist teachers as they try to create specific, appropriate goals for their students and classrooms.

Collecting and Analyzing Formative Data

Data-driven schools have a good idea of where their students are at the beginning of the year and have measurable, hopefully attainable goals for where they want their students to be at the end of the year. After that a system of formative assessments should be implemented periodically in order to establish benchmarks which will help track the progress of their students during the school year toward those year-end goals. “Simply using baseline data to set measurable year-end goals, without also implementing a system that allows for frequent analysis and adjustment of instructional and organizational practice, is not likely to result in significant improvements in student learning.” (McLeod, S., 2007)

“Effective formative assessment practices, implemented during the school year, have been shown to be a powerful mechanism for improving student learning. Research meta-analyses have shown that good formative assessment has a greater impact on student learning, and on achievement gaps, than any other instructional practice.” (Black & William, 1998)

Teachers need opportunities to meet frequently in order to have collaborative, data-based discussions about student progress in order to realize the instructional power of their formative assessment practices. It is during these meetings, that teachers need to identify specific patterns from the formative data and discuss what this data tells them about students’ progress toward their year-end learning goals. Teachers can then collaboratively identify appropriate instructional interventions that can be implemented during the next instructional period and collectively commit to implementing those interventions. Not only will this have a major impact on student achievement; but also, this type of professional learning community participation allows the teacher to claim ownership resulting in immense satisfaction.

Teachers, who are truly data-driven, utilize their instructional expertise to identify key formative indicators of success that can be used to measure student progress during the school year. They should also use appropriate technologies to collect, organize, analyze, and report that data to administrators, parents, students, and colleagues. “Other key skills of data-driven teachers include knowledge of relevant assessment literacy concepts (in order to appropriately interpret formative assessment data), the ability to engage in root cause analysis to identify appropriate instructional interventions, and the capacity and willingness to work effectively with other staff on shared instructional problems and solutions.”(McLeod, S., 2007)

“Administrators must recognize that the driving engine behind substantial improvements in student learning outcomes is a strong system of formative assessment, coupled with the opportunity for teachers to collaboratively make sense and act upon the formative data they receive. Too many school systems are focusing on summative baseline data because of NCLB and are realizing only later that a primary reason they are not obtaining desired results is because they lack a feedback loop that allows teachers to receive information, before the end of the school year, about the success or failure of their instructional interventions.” (McLeod, S., 2007)

In order to implement creative solutions that give teachers the necessary time to collaboratively analyze and act upon data, principals will need to work with local communities and district administrators. They will also need to train teachers in effective communication and teaming skills.

Making Changes

Data analysis is useless if it does not produce valuable instructional change. Teachers need to use formative and summative assessment data along with the implementation of strategic, focused instructional interventions to improve upon student learning. These interventions should be aligned with the district curriculum and state standards, in keeping with content-specific, developmentally-appropriate best practices. Teachers should be given the opportunity to work with curriculum specialists in their states and districts so that they can identify effective, grade-level instructional practices for their content areas.

Alignment for Results

It is difficult and challenging for administrators and teachers to shift from an existing practice which focuses on process and delivery, to a practice aimed at the achievement of results. Any instructional practice, organizational structure, or school program that hinders student success is reexamined and redesigned. Even successful practices are examined to see if they can be improved. Results-driven educators understand the importance and impact on student learning of continuous and progressive improvement, and recognize that even small improvements add up over time to become large ones. “Ambitious long-term goals like “achieving 100% proficiency” can be disabling rather than motivating. Turning desired outcomes into minute, concrete, short-term goals and then successfully achieving those goals is inherently motivating and can turn organizational inertia into desired progress.” (McLeod, S., 2007)

Instead of teachers individually selecting the direction and content of their professional development plans (PIP), administrators and teachers should work together to ensure that professional development opportunities are aligned to school, district, and student learning needs. Curriculum design and implementation should be adjusted to meet these needs. In results-driven school systems, all programs and processes are designed to facilitate maximum student learning. “If it’s not working, why are we doing it?”

Teachers who have incorporated a results orientation into their instructional practice regularly investigate the data pertaining to failure and/or success of their pedagogy. Successful strategies are adjusted to achieve even greater results and ineffective strategies are discarded. “Data-driven teachers exhibit a constant dissatisfaction with the status quo and continually strive for further improvement, even when already exhibiting high levels of success. These teachers also are willing risk-takers who understand that trying something new and different may be the only path to improved outcomes. A results-oriented school system incessantly asks, at every level of the organization, two questions:

What evidence do we have that what we’re doing is working?, and
How will we respond when we find out that what we’re doing is not working? (DuFour, Eaker, & DuFour, 2005).

Principals in successful data-driven schools ensure that these questions continually guide classroom instruction and organizational decision-making. Data-driven principals also align, and help teachers connect with, necessary resources to facilitate effective educational interventions. Two other important roles of principals are helping teachers “chunk” ambitious long-term objectives into short-term SMART goals and facilitating teachers’ understanding that taking greater responsibility for student learning can result in improved student achievement.” (McLeod, S., 2007)

Conclusion

If educators analyze their instruction and results on a regular basis and adjust it accordingly, student learning will improve. We need to focus on small, quick attainable goals and then build on those goals continually. Reflection about classroom instruction and student learning is paramount if we want our students to show significant improvement and hopefully this will produce maximum results for student achievement. Teachers must be able to participate in professional learning communities and collaboratively identify and implement effective, strategic instructional interventions. (Supovitz & Klein, 2003).

Resources

Bernhardt, V. L. (2004). Data analysis for continuous school improvement (2nd ed.). Larchmont, NY: Eye on Education. [available at http://www.eyeoneducation.com]
Black, P., & Wiliam, D. (1998). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 80(2), 139-148. [available at http://www.pdkintl.org/kappan/kbla9810.htm]
DuFour, R., Eaker, R., & DuFour, R. (Eds.). (2005). On common ground: The power of professional learning communities. Bloomington, IN: National Educational Service. [available at http://www.nesonline.com]
Schmoker, M. (1999). Results: The key to continuous school improvement (2nd ed.). Alexandria, VA: Association for Supervision and Curriculum Development. [particularly pages 1-55; available at http://shop.ascd.org]
Supovitz, J. A., & Klein, V. (2003). Mapping a course for improved student learning: How innovative schools systematically use student performance data to guide improvement. Philadelphia, PA. [available at http://www.cpre.org/Publications/AC-08.pdf

http://www.k12schoolnetworking.org/2007

http://www.accessibletech4all.org

http://www.rand.org/pubs/occasional_papers/OP170

http://www.ael.org/dbdm

http://www.clrn.org/elar/dddm.cfm

http://www.microsoft.com/.../ThoughtLeaders_DDDM_May05.doc


1 comment:

  1. Nice blogging, My review is very good example.
    Lindsay Rosenwald http://www.lindsayrosenwald.info/ Dr. Lindsay Rosenwald is one of the re-known venture capitalists and the hedge fund managers in the world

    ReplyDelete