Over the summer of 2004 I asked Professor Roger Kaufman of the Economics Department of Smith College and his colleague, Professor Geoffrey Woglom of Amherst College, to help refine the rather crude work I had done about a year before on "overachieving colleges." I was interested in several questions that arose from Professor Kaufman's earlier work, but I especially wanted to know which private undergraduate liberal arts colleges were achieving exceptional results with limited resources. Professor Kaufman, a well-regarded economist, had presented an important paper at the November 2003 conference on the liberal arts at Williams College and published valuable studies on issues relating to higher education. In response to my invitation, he suggested that Professor Geoffrey Woglom of Amherst College, with whom he had co-authored much of his earlier work, collaborate on this project, and I readily agreed.

Given the limited amount of time available we decided to restrict the analysis to the105 liberal arts colleges in the top two tiers of the US News rankings. We all recognize that this is a limited and imperfect ranking, but it includes almost all the colleges that emerged from their previous analysis and dovetailed neatly with their previous work.

The work involved two principal stages. It was essential at first to look more closely at financial resources per student than I had done in my initial inquiry. (I used endowment dollars per full time student; Kaufman and Woglom used net assets per full-time student at the end of the 2000/01 fiscal year, a more complex but surely better indicator of overall financial strength.) The results of both tabulations underline, however, the wide discrepancy of resources available among these colleges even among those of the so-called "second tier." Principia College in Illinois for example, had almost $600,000 in net assets at the end of 2001; three other Illinois colleges: Lake Forest , Augustana, and Knox, each had less than a tenth of this amount, and, as we shall see, several of them seem to be doing very commendable work.

Combining Kaufman and Woglom's paper, prepared at our request, with Table 1 in their previous paper ("Financial Changes and Optimal Spending Rates among Top Liberal Arts Colleges," Review of Higher Education, forthcoming) shows the same discrepancy at the national level. Here are the twelve colleges in these studies with the highest net assets per FTE student at the end of their 2001 fiscal years:

Here are the twelve with the lowest net assets per FTE:

(The median of the wealthiest dozen colleges is approximately ten times that of the least wealthy colleges.)

The question that most interested me, however, was not how much money these colleges had but what they were achieving with it. This is a difficult question because so much of what we commonly use to judge the "quality" of an institution are "inputs" that simply reflect available financial resources. (Faculty student ratios, or library books per student, for example, are largely determined by dollars available per student and tell us little about whether the books are being read or how well faculty and student are interacting.) However, two possible outcome indicators - graduation rates and the percentage of students going on to receive Ph.D.s - seemed worth close examination, since the first, albeit it crude was logically prior to any other outcomes. (If students don't graduate, it is hard to identify other benefits of their time at a college.) The second indicator, admittedly an imperfect one, may reflect the robustness of intellectual life on a campus. (If we ask: Are students "catching fire," this figure can provide a partial answer.)

Kaufman and Woglom performed statistical regressions to help answer some of my questions. Here is what they did (in layman's terms) in studying graduation rates. First, they determined what effect on graduation rates could be ascribed in all the colleges studied to four factors: expenditures per student, percentage of students graduating in the top 10% of their high school classes, whether or not the college is all-women, and the difference between the 75th percentile SAT score and the 25th percentile SAT score of entering classes. These factors, taken together, make it possible to predict in general terms graduation rate. However, when one measures the actual graduation rate at an individual college against these predictors, there are many exceptions.

The results were surprising. To be sure, US News uses similar regression techniques and presents them in one column of its rankings. But Kaufman and Woglom call our attention to a very interesting group of "overperformers" and to some surprising "underperformers" as well.

There are 61 overperformers in graduation rate among the 105 colleges analyzed (Kaufman and Woglom [2004] Table 6 column 1). The positive results begin with:

Thus, Bennington's actual graduation rate is 17.7 percentage points higher than that predicted by Kaufman and Woglom's regression. The list continues down to:

No less striking is the list of underperformers, some of which have relatively high expenditures per student, high percentages of students from the top of their high school classes, and with high SATs, but relatively low graduation rates. These institutions:

Even when Kaufman and Woglom corrected for the fact that students in the northeast tend to graduate at higher rates than their counterparts in the west, Reed, Lewis and Clark and Harvey Mudd still came in with negative numbers.

These figures do not provide, I hasten to add, a ranking of overall "quality" or even a secure measure of the value added by a college. But they do compel us to ask a further set of questions, above all, what is it that accounts for these discrepancies? Are some institutions much more nurturing of their students, while others tell them to "Shape up or ship out!" - and they do? Are the overperformers just easier than the underperformers? That seems to me unlikely when I notice Amherst, Williams and Haverford relatively high up on this list - certainly not "gut" schools - and a comparable institution, Middlebury, quite low. That suggests to me that the answers are more likely to relate more directly to campus cultures than to the ease or difficulty of graduation requirements. We need, I am convinced, to look more closely at what conduces to high graduation rates, and what may make students "vote with their feet."

The picture that emerges from Kaufman and Woglom's regressions relating to graduates from these colleges who go on to win a Ph.D. also raises some important further questions. As might be expected, the SAT score that was the best predictor of eventual Ph.D. rates was the 75th percentile score. Furthermore, the all-women's colleges didn't behave differently in this regression.

The top of their list (Kaufman and Woglom [2004] Table 3, column 2) has some of the expected names:

Then Reed, which ranked so poorly on graduation rates, emerges as an overperformer in Ph.D. production (by 6.8%). The next dozen institutions are:

Again there are underperformers, and some surprises among them:

These include some of the most prestigious colleges in the country. Their relatively low place in this listing is not, I believe, an indication of low quality; it is more likely to reflect the tendency of their graduates to go into professions that do not require the Ph.D. (law and medicine, for example) or other fields such as finance. (Some of these low scores may also be due to what the authors call non-linear effects of the SAT variable.)

These listings are not an answer to "Where should Jane and Johnny go to college?" Kaufman and Woglom were not trying to do what US News claims to be doing, but they do point us to further questions that need to be asked. Are the overperformers doing some things that help their students grow intellectually? Is there a campus culture that takes academic achievement seriously? How do socio-economic status and racial and ethnic identity affect the results? How many students at the various colleges are from "first generation college attending families"? (It strikes me that the conspicuous overperformers in Ph.D. production may include many colleges with high percentages of such "first generation" students.) Finally, it would useful to compare the figures for Liberal Arts Colleges with those from a comparable analysis of other types of institutions: the Ivies, flagship state universities, masters and comprehensive colleges, etc.

For the present purposes of the Teagle Foundation, however, the study by Professors Kaufman and Woglom serves as a reminder not to forget the claims on our attention by colleges to do well both in graduation rates and Ph.D. production rates. There are a baker's dozen of them, listed here in alphabetical order to avoid any hint of ranking:

They must be doing something right!

While I want to be very careful not to exaggerate its significance, the list is a very striking one in several respects. Since Kaufman and Woglom control for expenditures per student (which are likely to be highly correlated with wealth), there is no reason to expect that the wealthy schools should be overperformers, and the institutions on this list are most emphatically not affluent institutions. The wealthiest among them is Bates with net assets slightly under $150,000 per student; the least wealthy is Bennington with only $22,515. In this group St. Olaf has the median net assets per student - $92,214. That is just over a tenth of the net assets at Grinnell, the wealthiest private liberal arts college. These institutions, moreover, are disproportionately church related with both Roman Catholic and Protestant affiliations prominent among them. It would be worth exploring whether there is some special synergy between religious and educational life on these campuses.

-

I hope Professors Kaufman and Woglom will soon publish the research they did under Teagle auspices. In the meantime, click here for a draft of their report (Word). I also plan to have further discussions with the authors to see if their studies can be advanced into other areas - for example, comparisons with institutions in other sectors, or the role of other factors such as ethnicity, financial aid etc. Their study is, we all understand, a preliminary one but it certainly points in interesting directions.

W. R. Connor

13 December 2004

Email This Article