Research | Spring 2020 Issue

Strategies for Schools

The challenges of measuring what works — and what does not — in education

By Jean Merl

ASK JOHN ROGERS WHAT WORKS to improve education and what doesn’t, and he will very politely tell you it’s just not that simple.

“Education is a complex enterprise,” said Rogers, director of UCLA’s Institute for Democracy, Education and Access (IDEA). “Yet there is generally a presumption that education problems can be thought of as technical issues.” That, he said, is more appropriate for “making widgets.”

Rogers, a professor at the Graduate School of Education and Information Studies, said education reform often has “implications for equity” that raise questions about who is being served and why. In addition, he said, there is a question of whether the responsibility for improving education should rest with schools or be seen in a broader context of the larger community.

“Which lens are we using?” Rogers asked, during a recent telephone interview. His hobby is photography, and he said he displays some of his photos — a mix of images of protest marches and landscapes of beaches, deserts and forests — in his office at Moore Hall. The lens metaphor comes naturally.

“Telephoto, which focuses on a specific classroom practice?” he asked. “Or wide-angle?”

Fuzziness about basic questions, including equity, responsibility and context — and whether to take the close-up or long view — have caused good intentions to tumble into traps of unintended consequences, often because of a failure to recognize that such questions are important, or that they even apply.

EAs an example, Rogers pointed to the emphasis on raising standardized text scores that was a hallmark of the federal No Child Left Behind Act. That led some educators to “teach to the test” and focus on students with promise while shortchanging those who needed help the most.

Increasingly, he said, generous philanthropic donations to school districts are coming with strings supporting a particular program or a subject matter that pleases the donor but may not always be best for students.

And a growing number of charter schools, which are publicly funded but exempt from major rules governing traditional schools, have shifted away from their beginnings as “lighthouses” showing innovative ways to improve education and have often set up competition for students that can destabilize the very systems they were intended to help.

Charter vs. traditional

IN THE LOS ANGELES UNIFIED SCHOOL DISTRICT, the issue of charter vs. traditional schools has dominated several cycles of bitter, high-spending school board elections. In a 2018 research and policy brief, Rogers noted that more than 160,000 students in kindergarten through 12th grade — more than a fourth of all LAUSD students — were enrolled in charter schools.

He was part of a symposium of business and civic leaders, education experts and others who had joined together two years earlier to help the district. Their report, We Choose All: Research to Inform Public Education in Los Angeles, highlight- ed “emerging evidence from around the country about charters, choice and competition.”

The report articulated shared values. In its introduction, Rogers noted that “it pointed to the need for us to learn more about how different stakeholders in Los Angeles think about these issues.”

He offered this “self-critique” of the effort: “We had one powerful session that brought peo- ple together. What we needed to do was to have a series of conversations. … With ongoing dialogue, everybody is served better.”

New York’s experience

Continuing dialogue is important in two movements that have gained recent momentum: community schools and what is known as “Improvement science.”

Community schools take a wide-angle view of school reform and are intended for impoverished areas. The schools team with nonprofits and other public entities to provide services to students and families dealing with housing instability, neighborhood violence, inadequate health care and other issues that affect the ability to learn.

In a recent Opinion article in the Los Angeles Times, researcher William R. Johnston at the Rand Corp. told about gains he and his colleagues had found during a study of New York City’s community schools initiative. These schools served some of New York’s most disadvantaged families.

The initiative provided, according to need, dental and vaccine clinics, food banks, legal aid and mental health assistance. One elementary campus stayed open at night, on Saturdays and during school breaks to offer sports, English language learning and art classes.

Three years into the program, researchers found a small improvement in math achievement and big gains in student attendance. More elementary and middle school students were passing courses and moving to the next grades on time. High school graduation rates also rose.

L.A. Unified also has created community schools. In a commitment that was part of the agreement to settle last year’s teachers’ strike, the district will transform 30 traditionally organized campuses by the end of the 2020-2021 school year. Each school gets an additional $400,000 in funding.

Still, Johnson wrote, it will take a while to see results, especially in high schools. “It may take years to see the positive effects of such ambitious whole-school reforms.”

The curse of silos

ABEL VALENZUELA JR., director of the UCLA Institute for Research on Labor and Employment, supports the principles of community schools but would like to see them expanded to colleges and universities.

Valenzuela laments “silos” that prevent schools at all levels from addressing issues that affect students, something that community schools try to do.

“I think better coordination between the different educational systems is important,” he said in an interview at his office on Le Conte Avenue, a busy outpost at the edge of the UCLA campus. “But then [also] better coordination [is important] with systems outside of education … systems of work, systems of neighborhoods and housing and exchanges that occur at that level, systems related to government and services.”

Valenzuela, a professor of Chicana/o studies and urban planning, said his work with the undocumented shows how issues outside of academia can affect a student’s success. The undocumented need special help, he said, and sometimes their families do, too.

“If a student’s parent or sibling is separated, their quarter is lost because they’re devastated,” he said. “They’re trying to figure out how to get their family members back, or how to deal with this familial crisis.”

Valenzuela said colleges and universities must become more student-centered and recognize “some of the cleavages that confront students.

“You can’t educate students who are de- pressed, or who are concerned about other things that are floating, engaging inside of their minds and putting them in dark places.”

He said Chancellor Gene D. Block is calling for more attention to student mental health, which Valenzuela said will make UCLA a better place.

Improvement science

A GROWING NUMBER OF CAMPUSES around the country, including some community schools, are using methods based on the growing field of improvement science, not so much a specific innovation as a way of testing the effectiveness of educational programs.

It works this way:
1) Set a particular goal
2) Develop an idea for achieving it
3) Rigorously assess and refine how to accomplish the goal while placing it into the hands of teachers and others working in the schools.

“It’s both a set of methods and tools” to reach a goal — improved reading instruction, for instance — and “a philosophy about how one should do the work of change,” said Louis Gomez, a professor in the Graduate School of Education and Information Services.

Although the concept is not new, it has caught on more broadly within the last decade, Gomez said, largely because of the Carnegie Foundation for the Advancement of Teaching. He said the model can be applied in any kind of school — traditional, organized, charter, community or private.

In recent years, the Carnegie Foundation has highlighted the achievements of a number of schools using improvement science.

»In California’s Central Valley, a network of schools helped significantly more of its fifth graders reach the state standard in math in only two years.

»In Menomonee Falls, WI, improvement science helped a high school advance from a federal designation of “in need of improvement” in 2011 to being named “a top high school in the nation” by U.S. News & World Report just six years later.

»In New York City, New Visions for Public Schools worked with 110,000 students — many living in poverty, some still learning English and others struggling with disabilities — and increased graduation rates by more than 9% and college readiness rates by 16%.

While these schools are on the right track, Gomez said, “Real transformation of education is a multidecade commitment.

“Our work needs to be disciplined both by the sense of urgency that we should all feel about improving the lives of children; but by the same token, we have to understand that it has taken us decades to create the system that we’ve created, and it’s going to take us a certain amount of time to reshape that system to be something better.”

Jean Merl

Jean Merl

Jean Merl worked as an editor and reporter for 37 years at the Los Angeles Times, specializing in local and state government and politics and K-12 education. Since retiring from the Times, she has contributed to the publications of several UCLA entities, including the School of Nursing and the Graduate School of Education and Information Studies. Merl is also a proud Bruin (MA, 1972).

Post navigation

Related

profile

Eli Broad and The Power of Giving

Like few before him, Broad dominates Los Angeles.