Sunday, October 21, 2007
NYC Adopts Incentive Compensation Plan
Those interested in this policy topic are advised to visit the National Center for Performance Incentives , a national research and development center for state and local policy at Vanderbilt University’s Peabody College. Under the leadership of Matthew Springer, this center sets out to answer one of the most contested questions in public education today: Do financial incentives for teachers, administrators, and schools affect the quality of teaching and learning? The Center is hosting a national conference on this topic in late February 2008, which promises to be very, very interesting.
Monday, August 20, 2007
Improvement in Student Performance Slows Since NCLB
A link to the study, published in the July issue of Educational Researcher, can be found below.
Wednesday, August 15, 2007
America Competes?
This bill authorizes the President's Math Now proposal, which many have already disparagingly described as 'Reading First does mathematics.' Math Now focuses on the use of research-based instructional practices to prepare students for rigorous math courses in middle and high school, particularly Algebra I and II. Given appropriate allocations, Math Now will be implemented through federal grants that will flow to the states and then to local school districts to improve K-9 instruction.
While this focus on mathematics is certainly needed, I echo the concerns expressed recently by Chester Finn and Diane Ravitch (see link below to Beyond the Basics) that America's economic strength and innovation also spring from students receiving a rich education in other subjects, such as literature, history, and the arts: "America's true competitive edge over the long haul is not its technical prowess but its creativity, its imagination, its inventiveness. And those attributes are best inculcated not by skill-drill but through liberal arts and sciences, liberally defined."
Wednesday, July 25, 2007
Chairman Miller's NCLB Memorandum Link
Mapping State Proficiency Standards onto NAEP Scales
In a recently released report (see link below), the Department's Institute of Education Sciences compared state assessment proficiency percentages to the estimated
percentages of students achieving proficiency with respect to the standards established by the National Assessment of Educational Progress (NAEP). IES found large discrepancies between the two. It attributed this variation to differences in both content standards and student academic achievement from state to state, as well as from differences in the stringency of the standards adopted by the states.
Thursday, July 5, 2007
State Graduation Rates: A Failing Accountability Measure
While statisticians and researchers argue about the best approaches for estimating graduation rates, the public is left to ponder why state governments fail to accurately count the number of students who graduate from their schools each year and what can be done to fix this problem. This paper sets out to explain the reasons for this inaccuracy in graduation rates and to identify future directions for assuring accuracy in reported graduation rates.
At the time when the No Child Left Behind Act of 2001 made reporting of an on-time graduation rate an accountability reporting requirement, few states had data collection systems adequate to produce an accurate rate. The accurate calculation of this rate requires cohort data, or student record data on student progress from grade to grade, data on graduation status, and data on students who transfer in and out of a school, district, or state during their secondary school studies. Given the absence of cohort data, the NCLBA regulations offered the states some flexibility in reporting graduation rates. Specifically, the states were provided with the option of developing their own definition of a graduation rate that “more accurately measures the rate of students who graduate from high school with a regular diploma” (Title I Final Regulations, 2002, section 200.19 (a) (1) (B)), provided that the state did not count drop-outs as transfers. The states individually responded with new proxy graduation rate definitions, most of which were calculated using cross-sectional data.
Independent researchers quickly pointed to flaws in several of these state-created graduation rate definitions that had been approved by the U.S. Department of Education (hereafter referred to as the Department). The Education Trust examined the first round of state graduation rate data reported to the Department for the 2001-02 school year and found that many state graduation rates appeared to diverge significantly from independent estimates. The report writers asserted that many states used questionable graduation rate definitions, which too often had the effect of overinflating graduation rates. North Carolina, for instance, reported an impressive graduation rate of 92.4%. However, when Jay Greene, an independent analyst, applied a cohort definition to enrollment data and diploma counts in the Department’s Common Core of Data, he produced a graduation rate of only 63% for the state.
The problems associated with the North Carolina graduation rate definition are illustrative of the complex problem of graduation rate definitions. The North Carolina graduation rate definition was not based on the percentage of students who entered in the 9th grade and received a degree four years later, but on the percentage of diploma recipients who received their diploma in four years or less. Students who dropped out of high school or transferred were excluded from the calculations altogether. As the Education Trust noted in its 2003 report, this meant that “if only 50% of students who enter 9th grade in North Carolina were to eventually obtain a high school diploma, but every one of those 50% did so in four years or less, then North Carolina would report a “graduation rate” of 100%.” North Carolina’s rate definition clearly produces a misleading graduation rate, yet their graduation rate definition met the NCLBA criteria for state-defined rates and had been approved by the Department.
There are two issues with North Carolina’s graduation rate definition that recur in many states. First, the rate definition uses cross-sectional data, since in 2003, as is the case today, the state of North Carolina does not have a student record data system in place and therefore cannot use cohort data to calculate its graduation rate. Because student record data are not available, the state cannot track student progress from the 9th through the 12th grade to provide an accurate calculation of on-time graduation. Secondly, the cross-sectional data provide annual snapshots of the number of students and dropouts at one point in time in the school year, but they cannot track individual students as they transfer in and out of a school, district, or state during their secondary school education. This failure to account for transfers produces graduation rates that misrepresent the true proportion of students receiving an on-time diploma.
The need to determine accurate graduation rates fueled efforts at the national level to improve the state of educational data systems. Following the published recommendations of a National Center on Education Statistics task force on Graduation, Completion, and Dropout Indicators, the Department directed resources to assist states with the development of student record data systems. The Department launched, for instance, a $50 million program of cooperative agreements to help states develop their student record data systems; initial awards were made to 14 states in November of 2005. The National Governors Association undertook its own measures to improve the quality of graduation rates, developing the Graduation Counts Compact in 2005. Under the Compact, states commit to developing a high-quality, student-level data collection system that tracks students from kindergarten through college. It also includes a four-year cohort graduate rate formula: the number of students who graduated divided by the number of students enrolling in 9th grade for the first time four years earlier—plus the students who joined this class of students (that is, the cohort) and minus the students who left. All 50 states signed the contract and promised to implement the reforms.
However, at present fewer than half of the states have student record data systems that will enable them to follow individual students from pre-kindergarten through high school, and only a handful are using them to calculate a four-year cohort graduation rate (Hoff, 2006). It may take years for other states to get such systems in place. Curran reported that the National Governors Association anticipates that by 2010, 39 states will report a graduation rate using the Compact (cohort) definition. These states will begin reporting only after they have developed four or five years of longitudinal data capable of tracking students’ progress from their first-time entry into the ninth grade through their exit from high school.
It is not sufficient to report on cohort graduation data without taking into account student transfers. As the U.S. Government Accountability Office reported in its 2005 report, No Child Left Behind Act. Education Could Do More to Help States Better Define Graduation Rates and Improve Knowledge about Intervention Strategies, the primary factor affecting the accuracy of graduation rates is student mobility. The American family in the early 21st century is very mobile, presenting a challenge to state and local education agencies to accurately account for student transfers and dropouts.
In order to accurately track students who transfer in and out of a local education agency, a state must have a student record data system that includes exit or “leaver” data. These data are typically entered in the system as codes that identify the reason a student left a particular district or school (Data Quality Campaign, 2006). States with exit data systems in place provide their districts with a set of codes with which to identify reasons for students’ exits, including events such as death, transfer out of state, transfer to a home school, transfer to another country, transfer to a private school, incarceration, General Educational Development (GED) certificate, and hospital-bound.
Since exit code systems are critical to the reliability of graduation rates reported by states, the Department’s National Forum on Education Statistics, a cooperative of state, local, and federal education agencies, produced Accounting for Every Student: A Taxonomy for Standard Student Exit Codes (2006), which outlines the need for an exit code system and provides recommendations on how to establish a classification system of exit codes within the architecture of the student record data system. An exit code system, the Forum argued in this report, can greatly assist states in producing correct and comparable calculations of completion and dropout rates.
However, an exit code system is only as useful as the policies and procedures governing its data. A local educational agency, for instance, without policies and procedures in place to sufficiently identify the reasons for students’ exits from a system, may inaccurately report all students of “unknown” status as dropouts. It may also count students who drop out, return to school, and then drop out again as a dropout each time, inflating the district’s drop-out rate. Alternatively, a district may record students who drop out as transfers before they receive documentation that the student actually enrolled in a different school, thereby artificially lowering the drop-out rate. These situations illustrate the problems posed by insufficient policies and procedures for identifying and tracking student exits.
Identifying students who transfer is important to calculating graduation rates, since a state-defined rate definition under NCLBA, as noted earlier, can measure the rate of students who graduate from high school with a regular diploma provided that the state did not count drop-outs as transfers. State adoption of a standardized procedure for documenting transfers was recommended by the National Governors Association Task Force on State High School Graduation Data, as part of its recommendations to states on how to develop a high-quality, comparable high school graduation measure. Their task force had recommended use of a transcript request or other documentation from a receiving school as the only form of acceptable documentation to record a student departure as a transfer. By default, a student for whom there is no information should be documented as a nongraduate or dropout. However, as of August 2006, few states had established this procedure, although Curran reported that many states were developing checks and audit procedures as their student record systems came online that enabled undocumented student transfers to be identified when the student registered at another school or for students who had transferred out but had not shown up as enrolled elsewhere. This approach, however, does not fully address the issue of transfers to private schools, home schools, or out-of-state schools, which are harder to verify.
The use of proxy graduation rate definitions will soon end as states, with the support of the Department and interested groups such as the National Governors Association and the Data Quality Campaign, complete the process of developing student record data systems. When student record systems come online, it is anticipated that many of the accuracy issues plaguing graduation rate reports will simply disappear. However, the high school graduation rates reported using cohort data will only be accurate if actions are taken now to ensure that policies and procedures are developed and respected in regards to documenting student exits, especially transfers. Ultimately the leadership at the school, district, and state levels is responsible for creating a data-driven culture that makes it a priority to rethink and possibly reorganize how education data is managed throughout the system and increase training and professional development for staff, including managers and users. Only a broad-based commitment to data quality will produce accountability measures that accurately portray the state of education in the U.S.
Wednesday, June 20, 2007
Incentive Pay for Teachers ... New Buzz about an Old Idea
The new report by TeacherSolutions takes a favorable view of one of the profession's most controversial issues--merit pay. Merit pay is certainly not a new idea in American education. Merit pay policies have been formulated, adopted, implemented, and evaluated throughout the twentieth century. Single salary schedules for teachers, typically based on teachers' qualifications and years of services, have dominated teacher compensation systems since the 1920s. But what the NEA won't share is that historically the consideration of merit has been an important part of these single salary schedules since this earliest period.
In a nutshell, pay for performance is a systematic process for measuring teacher behavior or results, and linking these measurements to changes in teacher pay. Performance indicators may include changing classroom behavior, improving professional skills, and producing desired outcomes. A teacher’s knowledge and skills, for instance, may be assessed via testing, certification, and classroom observation. Classroom performance is typically measured by observations such as those done as part of a teacher evaluation system. Instructional outcomes may be assessed via tests of student learning and sometimes other measures such as attendance and graduation rates. Teacher performance can be defined in terms of these outcomes, and pay changes can be linked to outcome measures.
In response to the rise in educational accountability in this country, pay for performance systems have become increasingly sophisticated.
All indications from Washington are that pay for performance policy is here to stay, at least through the Bush Administration. The FY06 federal budget included funding for the Teacher Incentive Fund, a grant program that seeds the development of pay for performance programs, and for a National Research Center on Performance Incentives, established through a $10 million grant to Vanderbilt University from the Department of Education's IES. The President's FY2008 budget request for the Department of Education proposes to expand support for the Teacher Incentive Fund. The National Governors' Association, one of Washington's most respected public policy organizations, has been a vocal supporter of pay for performance policy, which it views as critical for improving teacher quality. Not surprisingly, governors across the country have raised the profile of this issue. As the Education Commission of the States has pointed out, "20 governors outlined teacher compensation as one of their major education issues in their 2005 State of the State addresses, and nine specifically spoke of some type of performance-based or merit pay" (Azordegan, Byrnett, et. al., 2005). Far from being an educational fad, pay for performance policy will have continued and profound effects on the teaching profession for some time to come.
Virginia's Pre-K Initiative...Gathering Steam
We’re going to keep working to give every
Universal pre-kindergarten was a core element of Virginia Governor Kaine’s election platform, and he reiterated this policy position during his election-night speech in 2005. While the governor has not been able to implement his policy of universal pre-K, he has been able to serve as an “issue catalyst” by raising the profile of preschool education in the
One of the Governor's early acts after taking office was to issue Executive Order 7, which established the Start Strong Council. This advisory council has played an important role in policy definition and agenda setting. The council includes approximately 25 leaders from business, schools, the faith community, private child care, Head Start, and state- and local-elected officials. The Council was given several responsibilities, including developing statewide goals and best practices for “expanding opportunities for 4 year olds to access quality pre-kindergarten programs” (Executive Order 7, 2006). The Council’s first report, issued in December 2006, included several policy recommendations to the Governor, including the development of a quality ratings system for preschool settings and a proposal for grant-funded preschool pilots to test innovative program strategies.
The Governor has attempted to build support for his pre-kindergarten policy initiative in other ways as well. The Governor’s Early Childhood Summits, which have been held twice during the past two years, are designed to inform business and community leaders, elected officials, and other stakeholders about the benefits of investing in early childhood education,.
Response to the Governor's pre-kindergarten initiative have been decidedly mixed. For the most part, organizations such as the Virginia Metropolitan League, the Virginia Association of Counties, the Virginia Association of Elementary School Principals, the Virginia Education Association, and the Parent Teacher Association have been very supportive. Other groups, including the association of superintendents and local school boards, have expressed strong opinions about various aspects of the initiative. While these two groups were generally supportive of pre-kindergarten, they have had concerns about the governance structure and the delivery mechanism of the initiative. In addition, while many private providers have been enthusiastic about the Governor’s intention to incorporate both public and private providers for preschool delivery, there have been a few vocal opponents from this constituency. There has been growing interest nationally in early childhood education as a future workforce investment, and state-level economic development and workforce development interest groups have followed suit by demonstrating support for this policy issue.
Virginia's preschool pilot initiative, which is in the starting gate, is considered a critical component of the Governor’s policy setting activities, since the Governor undoubtedly hopes to demonstrate through this experiment the feasibility of the innovative strategies recommended by the Start Strong Council, such as the diverse delivery network (including both public and private partners), incorporating a parent tuition component, and utilizing local school readiness councils to ensure systemic planning at the local level. Each of the pilot sites will receive strong support from the Department of Education, including technical assistance and start-up funds in addition to the per pupil funding, in order to optimize the conditions for success. An external program evaluation will be conducted on all pilots to assess how well their partnerships and other strategies worked, the level of quality in the various public and private learning settings, and the outcomes for the participating children. Formative and summative evaluation data will provide the Start Strong Council and the Governor’s office with critical information to evaluate this policy.
The preschool pilot initiative had a bumpy start earlier this year when the 2007 General Assembly made substantial changes to the budget language for the preschool pilots, reducing the budget by $2.1 million, limiting participation in the pilots to at-risk children, and adding evaluation requirements for selecting providers. Considering this was a mid-biennium budget, however, the initiative should be viewed as successful for having secured any funding at all. I take this to be a positive sign that the preschool policy issue will not only remain on the state agenda but will pick up additional support over time as the legislators and the public realize the effectiveness of pre-kindergarten in preparing children for success in school.
Friday, May 18, 2007
Educating Researchers
"The problem is that the doctoral programs offered and the degrees awarded by education schools are a mishmash. Programs for the preparation of researchers and the education of practitioners generally look very much alike . . .The blurring in purpose of the Ed.D. and Ph.D. leads to a larger problem in the preparation of education researchers. It encourages a
commensurate blurring in the programs to prepare researchers and practitioners. More often than not, they enroll in the same doctoral programs. The result is that practitioners too often receive an education designed for researchers, and future scholars take their course work with
practitioners who have little interest in research or rigorous scholarly studies."
I just finished my first course in my Ph.D. program in educational policy, planning, and leadership. The course, Educational Research, was designed to provide the foundations of research design. There was a handful of Ph.D. students in the class, but the group consisted mainly of Ed.D. students. And the latter group for the most part had absolutely no interest in learning about research design and didn't understand most of the course content. The professor had to revisit topics such as internal validity several times, taking class time away from new topics in order to help these students learn the basics. The course was taught from the perspective of learning about research in order to interpret it (i.e., the practitioner's perspective), not from the perspective of conducting research yourself. These are the frustrations when the two groups of students are indiscriminantly mixed in classes.
Check out Levine's report -- there's a link on this page.
Friday, April 27, 2007
New Report: Performance-Pay for Teachers
Homeland Defense - No Child Left Behind the Lines?
Tuesday, March 20, 2007
The Economics of Early Child Education
At the National Press Club earlier this month, a group of business leaders, economists, and philanthropists launched a modest, $3.1 million, ten-year project to make early education a top U.S. priority. Entitled Partnership for America's Economic Success, the partnership is now in the midst of a two-year research phase, and is commissioning about 15 studies to discern the economic benefits of early education and policy options for public and private financing, among other things.
This project has benefited immensely from the leadership of Susan Urahn and her staff at the Pew Charitable Trusts, which manages and helps fund the Partnership for America’s Economic Success. The Trusts have made considerable contributions to early childhood education. Their State Policy Initiatives program, for instance, has funded rigorous, policy-focused research and supported public education campaigns that demonstrate the value of high-quality preschool for all three- and four-year-olds. The Trusts has invested over $50 million to advance this goal since 2001, launching pre-K campaigns in over a dozen states. And progress has been made: over the past two years, states have increased preschool funding by over $800 million.
Friday, March 2, 2007
"Secondary-Postsecondary Learning Options"
I spent my lunch-hour today listening to an informative webinar on “Secondary-Postsecondary Learning Options,” a catch-phrase coined by the American Youth Policy Forum (www.aypf.org) to describe schools and programs that link secondary education with two- and four-year institutions of higher education and allow high school students to participate in college-level courses. The webinar was presented by the Educational Policy Institute (www.educationalpolicy.org), which excels in disseminating knowledge of educational opportunity policy issues to a wide audience of policymakers and practitioners.
Sunday, February 25, 2007
Who's Who in Education Policy: A Primer by Christopher Swanson
Still Children Left Behind?
Given the tremendous investment in NCLB over the past six years, a positive change in student performance, including a narrowing of the achievement gap between students of different ethnicities and socio-economic backgrounds, would have been expected. But, no. Instead, as the Education Trust pointed out in a quick and dirty analysis published on the day the Department of Education released the report, reading achievement in our nation's high schools has actually declined since 1998 and there is a similar trend in science achievement. And the large achievement gap between students of color and white and Asian students still exists in similar proportions as it did in the pre-NCLB era. Ironically, these findings were released simultaneously with the Department's 2005 High School Transcript Study, which reported, among other things, that more students are enrolling in higher-level courses and the average GPA is on the rise (appoximately a third of a letter grade higher than in 1990).
So, what gives? The Education Trust staff point a finger at disparities in teacher quality, pointing out that high-minority, high-poverty schools have lower teacher quality than their peers. Their point is well-taken, given the research to date that supports the strong relationship between teacher quality and student achievement. There is a growing momentum to address this issue of teacher quality. Look at, for instance, the recent report of the Commission on No Child Left Behind that called for improving the effectiveness of teachers. And the President's FY08 budget included a huge increase in funding for the Teacher Incentive Fund program, which encourages school districts and States to develop "innovative performance-based compensation systems that reward teachers and principals for raising student achievement." Improving teacher quality will be a significant issue in this and coming years.
Recommended: This Week in Education Blog
Saturday, February 24, 2007
Recommendations on High Schools from the Commission on NCLB
The long-awaited report of the Commission on No Child Left Behind was issued yesterday, and it was filled with recommendations on how to improve the controversial law. High schools, previously sheltered from the penetrating reach of NCLB, could, if the commission's recommendations were adopted, face increased accountability. Among the commission's recommendations: to require districts with large concentrations of struggling high schools to develop and implement comprehensive, districtwide high school improvement plans; and to require states to create and implement a 12th grade assessment.
Let me add that NCLB has brought about some positive change. Beyond asserting a level of federal control not hitherto seen, its insistence on measuring the progress of each school, each district, and each state, and ensuring that low-income students, students with disabilities, LEP students, and students from other minority groups are also making adequate progress, is very positive and long overdue. But its tendency to rely on "scientifically based research" and quantitative data to the exclusion of other valid forms of inquiry and its allegiance to the core subjects (really just Math and Language Arts) has turned education into a cold, lifeless shell. Achievement can be measured in other ways, and learning can happen, indeed thrive, beyond math and reading.
Science Education is Where in the President's Budget?
The squabbling in the nation's capitol continues over the President's proposed budget for FY08. Wednesday’s session before the House Committee on Science and Technology was filled with bickering, although the overall funding for the National Science Foundation was slightly increased. At issue was the apparent reduction in funding for science education and particularly in the area of teacher training. Considering the strong association between teacher quality and student achievement, this does at first glance seem to be a wrong move.