moderated by Paul DiPerna

Jason Palmer Interview

Paul DiPerna:

Jason... You earned your PhD in Public Administration at Ohio State about four years ago.

How did you decide to go in this field?

What was your graduate school experience like at Ohio State?

Jason Palmer:

I have a life-long interest in understanding how government works. One of my earliest memories is watching the Iran-Contra Affair unfold on C-SPAN. I felt like I was in the hearing room -- it was an awesome feeling.  I studied psychology as an undergraduate, because I was also fascinated with learning more about how we think, feel, interact, and develop. Also, as a Christian, I believe that we should do whatever we can to improve society and leave the world a better place than how we found it.  By the time I got to graduate school, studying public administration and focusing on social policy just made sense. It allowed me to develop my interests in government and organizations and to fulfill a personal desire to improve the social condition.

Studying at Ohio State, in what is now the John Glenn School of Public Affairs, was very rewarding. The relatively large campus offers an almost infinite amount of resources. In my case, the school offered a curriculum that spanned studies across the policy continuum (design, implementation, evaluation) and encouraged specialization by allowing students to study in other colleges (e.g., Education, Public Health). My committee chair, Mary Marvel, provided the right balance of challenge and support such that I feel I grew as a scholar and student of government. Another professor, Anand Desai, encouraged me to question my assumptions continuously and thus opened my mind to solving problems from a variety of perspectives.  It was and is a great place to study public affairs.

Paul DiPerna:

In 2002, you won the NASPAA (National Association of Schools of Public Affairs and Administration) Annual Dissertation Award.

What was your dissertation about?

Jason Palmer:

Actually I co-won the award. Ann Marie Thomson (Ph.D., Indiana University, Bloomington) also won the award for her dissertation: Collaboration: Meaning and Measurement.

My dissertation, Performance Incentives, Teachers, and Students analyzed the extent to which monetary rewards change behavior of teachers and ultimately affect student test scores. Some states have been actively engaged in trying to raise student performance in recent years, and their legislatures have offered to reward teachers monetarily if they raise student test scores. I wanted to see if the money made any difference. What I found was that the money was inducing teachers to do a lot more testing of students in the weeks and months leading up to the *big test* (to borrow from Nicholas Lemann). But I also found that the increased testing was having only a marginal effect on test scores.

Paul DiPerna:

How has your Ohio State research experience influenced your work at GAO?

Jason Palmer:

It definitely provided me with a glimpse of what it takes to carry a project forward from beginning to end, from conceptualizing and framing a problem to analyzing it to reporting about it. Our work at GAO is designed very similarly, with distinct phases that guide the work.

My graduate school experience also gave me a deeper appreciation for the challenge in analyzing social policy and creating solutions to challenging social problems. To paraphrase Peter Rossi, the expected value of a social program is zero. While this may be true, it is not especially encouraging. But for someone like myself who hates to be told he can't do something, such as find ways to solve challenging problems, it gives a lot of motivation to go above and beyond.

Within any project, there are any number of places where things can go awry. At GAO, the best projects are the ones where the project leader constantly stays on top of the information and process, keeps the end-goal in mind, and keenly anticipates and prevents or quickly solves problems. This is an especially difficult task when exploring social issues, but it definitely makes the work more rewarding when opportunities for improvement can be identified.

Paul DiPerna:

Do you think the innovations behind the Internet and World Wide Web have affected GAO's productivity in recent years?  If so, how?

Jason Palmer:

Absolutely. We are producing higher-quality information with the same or fewer resources, and thus are able to meet the ever-increasing information needs of our client, the Congress, and prudently manage our own budget. The best and most recent example comes from the team I am currently working with..

We're conducting two nationwide surveys of school principals. Years ago, these would have been printed on paper and sent in the mail and responses would have been keyed in to a computer. Follow-up would take months.  Instead, we have contacted all of the principals in our sample through email and given them each a unique log-in and password.  With these, they enter a web portal to complete their survey on-line. When they're done, their responses are automatically captured and uploaded. So, in this particular case, we can analyze their data in real-time and, at any given moment, determine our response rate and need for follow up. More generally, GAO is able to distribute, retrieve, and analyze information more quickly and more reliably than we did just a few years ago, enabling us to provide better information at lower cost.

Paul DiPerna:

I understand that your specialization at GAO is the No Child Left Behind Act (NCLB), which has directly or indirectly affected the lives of many American students, teachers, schools, and districts in elementary and secondary education.

Can you briefly describe the law?  Do you feel there is conclusive research (if any) on the positive and/or negative effects of this 5+ year-old law?

Jason Palmer:

The key aspects of the act are the provisions for school accountability and highly qualified teachers. There are other provisions, but when most people think about NCLB, they think about these two.

I have focused on the school accountability provisions that require states to set content and performance standards in reading, math, and science, to test almost all students in these subjects, and to hold schools accountable for ensuring that students, at a minimum, reach proficiency in these subjects by 2014. Schools that don't make progress toward the goal of having all students proficient by 2014 are subject to an increasing amount of attention from their districts and states, including school choice, tutoring, corrective actions such as curricular reform, and alternative governance.

It is very hard to come up with anything close to conclusive in education research. This is because so many factors that explain proficiency or achievement or academic growth -- key quantitatively-measured outcomes of the educational process -- are poorly understood.  We can talk in groups of variables -- individual, family, social, institutional (classroom, school, school district, state) -- but we are still several years from estimating the effects of variables in these groups at the individual student level with any sort of significant explanatory power. As it is, we simply know too little and disagree about too much.

Paul DiPerna:

This question might be a bit out of your normal line of work.. but do you think there is a digital divide among students today?

My hunch is that the disparity (variance) between schools is not as great as between students/families. So schools may have a significant role and potential to bridge any divide. Any thoughts?

Jason Palmer:

Let me caveat that my response is based on informal observation of the use of technology at schools we have visited for other purposes...

What I can say is that, over the past few years, I have noticed a sort of lag effect, where schools with one or two computers several years ago now have kids coming in with their own laptops, while schools without computers back then have at least one in most classrooms now. So, schools generally are moving in pretty much the same direction, but I don't see any evidence that the gap is closing.

In communities where poverty is high, priorities are still focused on the basic needs: shelter, food, etc.  If technology were infused into poor communities, it would definitely provide the basis to jump start a generation, but to grow it from the bottom up is extremely challenging.  It can't stop with just the hardware and software -- there has to be an infrastructure like continuous training, technical support, and even things we take for granted, like electricity that it rarely interrupted. You know better than I how expensive of an endeavor this is, but we've all benefited from the investments those before us have made.

Leaders in education at every level -- schools, districts, states, and federal -- are one piece of the solution, but money spent on technology means other needs have to be foregone.  While on the one hand it is a matter of priorities, its also a matter of resources -- schools can't do it alone.  They need to leverage community resources, families and foundations, to really make a difference.

Paul DiPerna:

With the upcoming reauthorization debate surround NCLB, can you give us your assessment of the surrounding political environment?

Who are the key players, and what are their perspectives?  What is the timetable?  What topics present potential for conflict, and for cooperation?

Jason Palmer:

There is a lot of bi-partisan interest in NCLB and in making the reauthorization happen.  Policymakers on both sides -- Pres. Bush, Sen. Enzi, and Rep. McKeon for the Republicans and Sen. Kennedy and Rep. Miller for the Democrats -- generally agree that the law has done some good in terms of producing data that practitioners can use to begin to improve instruction, if not also curriculum. The accountability systems states have in place provide, at least within a state, a common language and source of information for parents to determine where their schools may need to improve.

Some standard disagreements aren't going away, including the drive for alternatives to traditional public schools on the right and relatively more funding on the left. But, if past is prologue, the key players will find a way to compromise on these differences, because there is so much more they agree on. Whether it happens this year or not -- when it's due for reauthorization -- remains to be seen.

Paul DiPerna:

I understand that GAO provides white papers and reports to make the political process as informed as possible (and as much as legislators will allow).

How do you and GAO determine the informational needs of members and the public?

Jason Palmer:

GAO accomplishes its mission -- serving the Congress and the American people -- by producing reports with recommendations for improvement in the way federal agencies operate and, at times, by suggesting ways that legislation can be better designed to achieve congressional goals.

These reports are the product of a calculus that consists of (1) adherence to our key values of accountability, integrity, and reliability; (2) a productive relationship with the Congress, often at arms length in order to preserve our independence; and (3) an acute awareness of current and likely future issues that affect the daily lives of the American people.

We have built a structure that is guided by our strategic plan, yet flexible enough to respond to changing priorities or sudden events. Within that structure, we understand key challenges faced by the executive branch and can build upon that history. At the same time, we have personnel systems in place that allow us to shift resources on short notice so we can stay on the cutting edge of any sudden developments, like Hurricanes Katrina and Rita. It is a constant balancing act between capitalizing on resources we have developed over the years, and the need to stay nimble. This need will only increase as our world and its events continue to increase in complexity.

Paul DiPerna:

Can you describe the process that moves from a project's start all the way to publication and/or presentation?

How often do you present your reports and findings?

Jason Palmer:

GAO issues reports and delivers testimony to Congress almost every day, and many days we have multiple reports come out. The process for creating a report is elaborate, with many safeguards built in so as to ensure the quality of the information.

Generally, the work begins with a request from a congressional committee or at the direction of the Comptroller General to address a set of objectives. For example, we were recently asked to describe how states defined the high school graduation rates they used to comply with NCLB and to identify challenges they faced in calculating them. We then do some exploratory work and develop a methodological approach for addressing the objectives. Next, we implement the approach by collecting and analyzing data from a number of sources, data systems, law, policy, and administrative procedures, just to name the primary ones.

Our work also involves site visits to meet with officials or to see physical evidence. For example, we've had several teams in the Gulf Coast area to determine the Hurricanes' effects on residents of those states and localities and what various governments and charities are doing to help. We then draft and develop our findings and determine whether there are opportunities for federal agencies to improve their efforts. If so, we include recommendations.

We send our draft reports to affected agencies for comment and then ultimately issue the report. Often we are asked to testify before Congress on our work and it is our hope that, between the report and testimony, the work we do lead to improvements how our government works.

Paul DiPerna:

If I understand correctly, NCLB has some language addressing virtual schooling. (Virtual schools considered here to be those online schools offering partial or entire instruction, curriculum, or assessment, replacing or supplementing brick-and-mortar schools.)

How does the law treat virtual schools?

Jason Palmer:

The law generally expects that public virtual schools, including charter virtual schools, will achieve the same standards of traditional schools.

Bryan Hassel of Public Impact wrote one of the first pieces on the intersection of the law and the virtual schooling movement. Generally he argues that virtual schools can offer a solution for school districts who face problems finding seats in schools that students want to transfer to (after their school has been identified for improvement).

While these schools offer the convenience of on-line learning at the click of a mouse, there are concerns that students don't get enough face to face interaction. Savvy parents though are likely to compensate for this and make sure their kids are involved in other community activities where they get together with other adults and kids, like field trips, scouting activities, or group music lessons.

Paul DiPerna:

Could you see the law accelerating the growth of the virtual schools sector?

Jason Palmer:

Virtual schools are only going to continue to grow, mostly because of general growth in the sophistication of web services. Growth will also occur because school districts can use virtual schools as an option for students who wish to transfer out of schools identified for improvement.

Paul DiPerna:

What websites would you recommend to young parents for better understanding No Child Left Behind, and for accessing their child's school (public or private), or a school district's information?

Jason Palmer:

The U.S. Department of Education's main page has a lot of parent-friendly information.

The Department also operates the National Center for Education Statistics that has demographic information in its district and school locator.

On the performance side, many if not all states also produce report cards on their schools and school districts. For example, North Carolina has a web tool to allow users to select a school or district report card.

Some have web based tools that allow users to slice and dice information in any number of ways. One example is the Illinois Interactive Report Card where a user can pull up data on test scores, proficiency rates, and whether the school made adequate yearly progress, among other data.

These are just examples, many other states have similar tools. The Great Schools web site allows parents to look up and compare school information as well and allows users to leave comments about the school.

Paul DiPerna:

Great stuff.. Thanks.

I recently found a family of sites -- Public School Review and Private School Review -- that are good on school and district information.. in much the same way as GreatSchools. However there are some contrasts in the types of data presented, as well as site usability.

Ok we're about to wrap things up here..

Do you have any advice for graduate students when it comes to getting socialized in the educational research profession?

Jason Palmer:

The best approach is to work closely with your advisor. He or she knows the community and the issues. Do your homework, but don't hesitate to ask informed questions. Also, attend at least 2-3 conferences a year and take the initiative to let people know what you are working on and ask them about their work too. Find people working on similar ideas and suggest ways you may work together.

Paul DiPerna:

Are there any upcoming projects and/or conferences that particularly excite you

Jason Palmer:

The main thing I am focused on right now is the reauthorization of NCLB. Hearings are underway and it will be very interesting to see what issues are discussed and what changes are proposed. The growing consensus is that the spirit of the law is on target, but they have to deal with some significant technical and implementation issues.

December 1, 2006