Institutional effectiveness, as a SACS initiative, entails both a theoretical paradigm and a practical apparatus. The paradigm is committed to the view that "institutional quality depends not only on an institution's processes and resources, but also on the institution's successful use of those processes and resources to achieve established goals."[1] The practical apparatus requires, in turn, that accredited institutions of higher learning maintain processes of planning and evaluation that include "the formulation of educational goals consistent with the institution's purpose, . . . procedures for evaluating the extent to which these educational goals are being achieved, and . . . the use of the results of these evaluations to improve institutional programs, services and operations."[2] As an integral whole, this apparatus (goals/assessment/utilization) has been called, sometimes sardonically, "closing the loop." It should not escape notice that, in SACS usage, the metaphor excludes homeostatic processes: loops that return to equilibria--rather than improved programs, services, or operations--are precluded.
Now, there are at least two distinct methodologies--reflecting distinct methodological principles--for conducting evaluation research: goal-bound and goal-free procedures.[3] Broadly speaking, goal-bound evaluation is focused on the relative degree to which a given product effectively meets an antecedently specified goal, while goal-free evaluation measures the effectiveness of a given product exclusively in terms of its actual effects--the goals and motivations of the producer are ignored. Each approach has relative advantages and disadvantages. On the one hand, goal-bound evaluation is ordinarily more cost-effective than goal-free evaluation; on the other hand, measuring effectiveness entirely in terms of the degree to which stated goals are met can have at least two undesirable consequences: (a) since effectiveness is, on this model, inversely proportional to expectations, effectiveness can be raised simply by lowering expectations, and (b) deleterious or otherwise unwanted effects, if any, are left out of account, while unintended benefits, if any, go unnoticed.[4]
The standard model provided by SACS for evaluating institutional effectiveness is unmistakably goal-bound: institutional quality, as noted above, depends on the subsequent achievement of antecedently established goals. Thus, the standard model directs accredited schools to risk the future devaluation of present standards of academic excellence, and to ignore the evaluation of whatever unplanned consequences may result from existing policies and programs. These are, of course, at least for the most part, local concerns, since diligent faculties, once acquainted with the risks inherent in goal-bound evaluation, should be expected to avoid them. There are, nevertheless, more systemic, global threats implicit in the standard model that are not so easily overcome. Before considering these threats, however, three additional contrasts between goal-bound and goal-free evaluation are worth noting.
Firstly, it must be stressed that to advocate goal-free over goal-bound evaluation is not in any way to advocate purposeless pedagogy. To recommend that evaluators ignore the stated goals of a given curriculum is not at all to recommend that instructors ignore those goals. There is, furthermore, no reason to suppose that aimless, unplanned instruction will produce (or will appear to produce) better effects when examined through a goal-free lens than when examined through a goal-bound one.[5]
Secondly, goal-free evaluation is far less disruptive of the ongoing educational activities within a college or university than is goal-bound evaluation. The latter methodology demands months of painstaking prior preparation so as to provide site visitors with an exhaustive matrix of goals and assessment indices; goal-free evaluators--anxious to survey and assess the actual effects of an institution's educational programs--require no such preparation and attendant documentation.
Lastly, unlike the methods used in goal-bound evaluation, goal-free methods are particularly well suited to the continuous evaluation of an educational program during periods of change or growth since these methods are highly tolerant of novelty. In Michael Scriven's phrase, goal-free evaluation is "oriented towards final results, not original rhetoric." [6]
Because Kant's model is developmental (he views enlightenment as a process of intellectual maturation), and because he understands the prospect of general human enlightenment as intrinsically political (intellectual immaturity consists in a willingness to accept external authority in place of one's own reason), his essay remains pertinent to the analysis of contemporary educational systems, philosophies, and institutions. The connection between enlightenment and education, although largely implicit in Kant's essay, can best be seen, mutatis mutandis, through consideration of (a) his diagnosis of unenlightenment, and (b) his prescription for its future cure.
Why is it, Kant asks, that "so great a portion of mankind, after nature has long since discharged them from external direction (naturaliter maiorennes), nevertheless remains under lifelong tutelage, . . ."?[9] His short answer is facile: they are lazy; his long answer, however, is penetrating: an unenlightened population tends to be politically subservient. The unenlightened must, after all, entrust their affairs to the care of their guardians. After the guardians have first made their domestic cattle dumb and have made sure that these placid creatures will not dare take a single step without the harness of the cart to which they are tethered, the guardians then show them the danger which threatens if they try to go alone. Actually, however, this danger is not so great, for by falling a few times they would finally learn to walk alone. But an example of this failure makes them timid and ordinarily frightens them away from all further trials.[10] Thus, "for any single individual to work himself out of the life under tutelage which has become almost his nature is very difficult." [11]
Nevertheless, Kant is supremely sanguine as regards the prospects of future human enlightenment: "But that the public should enlighten itself is more possible [than for isolated individuals]; indeed, if only freedom is granted, enlightenment is almost sure to follow." [12] More specifically, Kant holds that "the public use of reason must always be free, and it alone can bring about enlightenment among men." [13] It is precisely here that Kant's prescription for overcoming the political inertia latent in unenlightened populations brings the connection between enlightenment and education clearly into focus, because by the "public use of reason" Kant understands "the use which a person makes of [reason] as a scholar before the reading public." [14] Inasmuch as it is still the business of colleges and universities to foster scholarship among faculties and students alike, Kant's reflections are easily applied to contemporary concerns.
In particular, one may ask: How can contemporary educators best model the public use of reason for their students while at the same time contributing to the scholarly life of their academic communities. Even if this question cannot be answered from a single perspective (given the diversity of what counts as scholarship on the contemporary scene), one can in any case maintain that the binding of pedagogy to antecedently formulated goals frustrates self-directed thought, and thus frustrates discovery, originality, and creativity in both instructors and students.
The argument is straightforward. College classrooms should exemplify Kant's understanding of human enlightenment, not simply because classrooms should routinely exhibit the public use of reason, but because classroom instruction is specifically designed to nurture collective intellectual development. Here it is not enough that a few isolated individuals should attain release from tutelage, but that the corporate whole should advance in knowledge, understanding, and self- directed thought. Consequently, the value of open inquiry must, in the college classroom, be shared--it must be intimately and vividly connected with the public use of reason. But unless students learn to set their own goals, their inquiries will not be open, nor their thinking self- directed.
Enlightenment, in other words, can neither be modeled nor achieved by instructors whose pedagogy is driven by the exigencies of goal-bound evaluation. Where effectiveness is measured solely against teleological indices, students are not encouraged to question ideas, to think and argue for themselves; rather, they are encouraged to perform just those cognitive tasks whose completion will best demonstrate that their instructors' pedagogical goals have been achieved. Students so educated will naturally become Kant's cattle: unable to fashion their own projects, they will remain objects of tutelage, while their collective intellectual labor is viewed as an "educational outcome," as a means of assessing their instructors, not as a means to their enlightened self-development.
Ultimately, students taught under the SACS initiative will become habituated, by example as well as assignment, to avoid creative, original solutions to problems, to minimize expectation, and to regard their work as valuable only to the degree that it fits an assessment matrix. These, Kant reminds us, are intellectual habits a free people can ill afford to perpetuate.
[Note that for Cicero the artes liberales are connected with the notion of free, unconstrained thought; in the expression institutus liberaliter educatione we find "liberaliter" deriving its meaning from "being a freeman, not a slave" (and this echoes or adumbrates Kant, of course)--the homo liberaliter educatus is a man of good education. But thinking that is regulated by expectation (of fulfilling goals or receiving rewards) is not free.]
D1: An activity is an actualized ability; an ability is a potential activity.
D2: Academic degrees are signs of academic ability.
Explication: The letters after a graduate's cognomen signify that the person named has
acquired some range of intellectual abilities.
A1: Education is not a product but a practice; not an artifact, but an activity.
A2: One primary function of an educational institution is to confer degrees.
A3: Learned abilities are acquired by habituation. .
Proposition 1: An academic degree qualifies someone's ongoing abilities, not someone's completed achievements (Proof: D1 & D2).
Scholium: The criteria for earning an academic degree are applied to one's achievements, but what is being measured is the character of one's abilities.
Proposition 2: It is a primary function of an educational institution to promote the acquisition of intellectual abilities (Proof: D2 & A2).
Lemma 1: Insofar as intellectual abilities are innate, educational institutions are superfluous.
Lemma 2: Insofar as intellectual abilities are learned, it is a primary function of an educational institution to foster learning them.
Proposition 3: It is a primary function of an educational institution to create and maintain contexts wherein students can develop intellectual habits (Proof: A3, P2, L1 & L2).
Scholium: Whether these habits are good or bad is relative to the individual students and faculty whose actual intellectual intercourse is sufficiently sustained over time to generate habits at all.
Corollary: The specific sort of activity that constitutes education is the cultivation of intellectual habits.
Proposition 4: Grades and grading criteria are pedagogically justified only to the degree that they serve as information relative to the acquisition of intellectual habits (Proof: P2 & P3).
Proposition 5: The marks of intellectual ability are ongoing activities, not grades (Proof; D1, A1 & P4).
Proposition 6: Intellectual habits, evaluated relative to the individual can be conceived as means between extremes (Proof: P4 & P5). Scholium: In the following table (intended only as an example), the quality off an intellectual habit is measured as a mean or an extreme, relative to a concrete individual at work on a concrete project.
Deficit Mean Excess ================================================================ Obtuse Open minded Gullible Narrow Comprehensive Depthless Repetitious Creative Indulgent Superficial Profound Incomprehensible Sloppy Accurate Punctilious Boring Elegant Pretentious
Proposition 7: Strictly speaking, one can neither buy nor sell an education, although one can acquire, with diligence, a first-rate education in any institution of higher learning that is committed to the cultivation of sound intellectual habits (Proof: A1, P1, P3 & P6).
[1] Southern Association of Colleges and Schools, Criteria for Accreditation: Commission on Colleges, 8th ed. (Decatur, Georgia: Southern Association of Colleges and Schools, 1992), p.15.
[2] Ibid., p.16.
[3] For an introduction to the distinction between goal-free and goal-bound evaluation methods as well as a thorough review of their respective strengths and weaknesses, see Michael Scriven, "Evaluation Perspectives and Procedures," in W. James Popham, ed., Evaluation in Education: Current Applications (Berkeley, CA: McCutchan Publishing Corporation, 1974).
[4] Consider a simple example. Suppose a computer manufacturer wants to introduce a line of machines that will run both DOS-based and MAC-based software, and that an engineering team, focused squarely on this goal, produces just such a prototype. Goal-based evaluators might deem the project complete, and the manufacturer might initiate a full-scale production run. But further suppose that in order to take advantage of this particular machine's versatility, users must possess a sophisticated understanding of how computational tasks are differentially allocated to distinct regions of volatile memory, and that most potential users with an interest in this sort of versatility will lack the requisite degree of understanding. In such a case, goal-free evaluators could avoid financial catastrophe simply by looking beyond the stated goals of the development project, estimating instead the actual effects full-scale production would be likely to have on the market.
[5] See Scriven, op. cit., p. 62.
[6] Scriven, ibid.
[7] Immanuel Kant, "What is Enlightenment?" in Lewis White Beck, ed., On History (New York: Macmillan, 1989), p. 3.
[8] Ibid.
[9] Ibid.
[10] Ibid., pp. 3-4
[11] Ibid., p. 4
[12] Ibid.
[13] Ibid., p. 5
[14] Ibid.