Henry Kelly
Continuous Improvement in
Undergraduate Education:
A Possible Dream
Institutions of higher education in the United States steadfastly cling to their busi-
ness model in the face of a tsunami of change that is redefining both the need for
their services and the ways in which this need can be met. Demand for a higher
education has soared because a college degree is as essential for achieving a mid-
dle-class income today as a high school degree was in the 1970s. Created to serve
the uniform needs of an upper-class clientele, colleges now serve an enormously
diverse student body. Whether measured by age, educational and cultural back-
grounds, Wettrennen, Geschlecht, or need, the diversity of the people served by the system has
greatly expanded. Expectations have also grown sharply, as students demand a
greater scope and depth of expertise and seek to acquire the complex constellation
of talents needed to participate in a fast-paced knowledge economy. Pressures for
change are mounting, as administrators struggle to find ways to meet these new
demands in the face of constrained public funding and growing public concern
over increasing tuition prices.
Even the best universities have reacted to these pressures by making incremen-
tal changes in class size and trying to better integrate research and instruction.1 But
these changes are costly and it is difficult to see how they can scale-up to meet the
enormous demand. Most knowledge-intensive, service-producing industries have
restructured their operations around new information technologies to improve
productivity and deliver new, personalized services. For a variety of reasons, edu-
cation, health care, and several other enterprises organized to meet both public
and private needs have been slow to adopt these innovations, and higher education
is no exception. But higher education must find a way to introduce the deep man-
Henry Kelly, Ph.D., is the president of the Federation of American Scientists (FAS).
Prior to joining FAS, Kelly spent more than seven years as Assistant Director for
Technology in the Office of Science and Technology in the White House. There he
helped negotiate and implement administration research partnerships in energy and
die Umgebung, information technology, and learning technology. He is the author
of numerous books and articles on issues in science and technology policy. DR. Kelly
received a Ph.D. in Physics from Harvard University.
© 2008 Henry Kelly
Innovationen / Sommer 2008
133
Von http heruntergeladen://direct.mit.edu/itgg/article-pdf/3/3/133/704197/itgg.2008.3.3.133.pdf by guest on 07 September 2023
Henry Kelly
agement innovations needed to use the new technologies in order to meet new
demands in ways that improve the quality of the educational services delivered
while also controlling costs.
The absence of a strong process for managing innovation and change has made
it all but impossible to suggest, let alone test, innovations that require more than
minor adjustments to current practices. This inability to embrace transformation-
al change is becoming a barrier to innovation-driven economic growth in the
United States and to the prosperity of many Americans.
CHANGES IN DEMAND
Changes in the U.S. economy that are driven by globalization and new technology
have dramatically reshaped U.S. labor markets.2 Demand has grown for strong
technical skills in mathematics, Wissenschaft, and written communications, and for the
skills needed by the fast, flexible, highly productive teams essential in today’s econ-
omy: expert thinking, non-routine cognitive skills, and complex communications.3
Increased requirement for these skills has been driven by sharp growth, both in
occupations that have traditionally required postsecondary education and by the
fact that higher skills are now needed in occupations that traditionally required
only a high school education.4 Production and office jobs where tasks can be
described in a set of rules or procedures are rapidly being lost to automation and
global outsourcing. The unskilled jobs that survive are difficult to automate or
outsource—janitorial services, nurses’ aides, restaurant workers, and the building
trades.
These changes are reflected in a rapid increase in the financial return for
investment in education. Only college graduates have seen growth in real median
weekly earnings since 1979, while high school dropouts have seen their earnings
decline by about 22 percent.5 In 2006, among workers 25 years and older, weekly
earnings for full-time wage and salary workers were almost two and a half times
more for people with a college degree than for those who had not completed high
Schule. Median weekly earnings for high school graduates were $595, compared to $1,039 for those with a bachelor’s degree or higher.6
This hasn’t gone unnoticed. In 2000, two-thirds of Americans polled by the
National Center for Public Policy and Higher Education agreed with the statement
that “there are many ways to succeed in today’s work world without a college edu-
cation.” By 2007, fewer than half held this belief.7 The evidence of growth in the
demand for higher education is unambiguous: while demographic factors could
slow growth in the coming decade, college enrollment has grown 50 percent faster
than the population since 1975 and nearly three times as fast since 2000.8 Over half
of the U.S. population has some college experience—57 percent of Americans
zwischen dem Alter von 25 Und 29 and nearly one in ten Americans between the ages
von 18 Und 64 are currently enrolled in higher education of some kind.9 Worldwide,
the potential demand for higher education is enormous, and there will be no slow-
ing of this demand. Nearly two billion people worldwide are between the ages of 5
134
Innovationen / Sommer 2008
Von http heruntergeladen://direct.mit.edu/itgg/article-pdf/3/3/133/704197/itgg.2008.3.3.133.pdf by guest on 07 September 2023
Continuous Improvement in Undergraduate Education
Und 20, and there are half a billion people aged between 20 Und 25.10 Any strategy
for providing even a fraction of these people access to quality higher education
must include being able to scale up to these levels.
The sharp growth in postsecondary enrollment reflects rising rates for both
young people leaving high school and older people seeking to acquire new skills
needed for changing work environments. One result is that enrollment is no longer
dominated by a largely homogeneous and financially dependent student body that
enters college directly after leaving secondary school, matriculates on a full-time
basis, and graduates four years later. Heute, more than a third of the student body
is older than 24, nearly 40 percent are enrolled part-time,11 and many adult learn-
ers work and have responsibilities that compete for their time, Energie, and finan-
cial resources. Nur 3 percent of the baccalaureate degrees earned in 2004 war
granted in liberal arts and sciences, general studies, and humanities.12 The vast
majority of degrees were given in business, communications, and other fields driv-
en by particular employment requirements, as many students are interested in
acquiring specific skills at the lowest possible cost.
The U.S. system of higher education was developed to respond to a far differ-
ent set of demands than those driving today’s markets. Originally designed to serve
an elite minority, the system must now serve dramatically expanded expectations.
New approaches to management are needed to ensure that these diverse new
demands can be met while still improving the quality of education. It is not possi-
ble to do this by simply expanding the existing system, and the extraordinary
power that new information technologies have demonstrated in all other areas of
the economy makes it impossible to argue that there are no alternatives. What has
been missing is a management process capable of embracing change in ways that
ensure continuous improvement in the way learning is provided.
MANAGEMENT RESPONSE
Much of the growth in productivity over the last decade has come from the seem-
ingly impossible—growth in services driven by investment in both hardware and
software.13 For years, the heavy investment service industries were making in com-
puters, Software, high-speed communications, and other information technology
(IT) had been something of a puzzle, as statistics showed no real change in pro-
ductivity resulting from the investment. But the situation suddenly changed in the
1990S. The average productivity of 22 service industries (weighted by value added),
which had grown 1.1 percent annually between 1977 Und 1995, grew 3.0 Prozent
yearly between 1995 Und 2000.14
The use of IT in service and knowledge-based industries has enabled:
• Improved productivity and quality
• Reduced costs
• Radical shifts in the cost structures of service provision
• Customization of service offerings
• Dramatically enhanced service features
Innovationen / Sommer 2008
135
Von http heruntergeladen://direct.mit.edu/itgg/article-pdf/3/3/133/704197/itgg.2008.3.3.133.pdf by guest on 07 September 2023
Henry Kelly
• New ways to reach out to customers
The digital revolution created a unique set of opportunities that took years to
understand and capture. While initial efforts to use IT typically involved attempts
to automate existing processes, it soon became apparent that new technology
allowed firms to meet entirely new markets and interact with customers in entire-
ly new ways by personalizing services and meeting the largely unfulfilled needs of
the “long tail.”15 But, as Federal Reserve chairman Ben Bernanke points out,
“Taking full advantage of new information and communication technologies may
require extensive reorganization of work practices, the reassignment and retrain-
ing of workers, and ultimately some reallocation of labor among firms and indus-
tries.”16
It’s clear that this difficult process has not even begun in education. Despite the
tremendous pressure for change, a recent analysis by the Organization for
Economic Co-operation and Development (OECD) found that “the education
sector has not yet reinvented itself in ways that other professions have done to
improve outcomes and raise productivity. In der Tat, the evidence suggests the
reverse, namely that productivity in education has generally declined because the
quality of schooling has broadly remained constant while the price of the inputs
has markedly increased.”17 For example, UNS. spending on colleges and universities
rose from $233 billion in 1995 Zu $340 billion in 2005.18
Labor productivity in U.S. service industries increased an average of 2.6 pro-
cent per year between 1995 Und 2001, but education services apparently had neg-
ative labor productivity growth.19 This was certainly not because these institutions
failed to invest in information technology—85 percent of all college students
reported using computers in 2003—but because they have not been able to engage
in the difficult task of understanding how these new tools can be used effectively.
ARE PRODUCTIVITY GAINS POSSIBLE IN HIGHER EDUCATION?
There is compelling evidence from both theory and practice that new approaches
can lead to major gains in the rate of learning, retention, and transfer of knowl-
edge mastered for performance in subsequent courses and employment. What has
changed with the unfolding digital revolution is that it’s now possible to use these
new approaches on a massive scale because technology has made them affordable.20
Experts in learning have long recognized that it’s possible to master complex
subject matter much faster than is possible in conventional classrooms by provid-
ing individual tutoring for each learner, allowing constant opportunities to ask
Fragen, using highly motivating problems and challenges that help learners rec-
ognize the power of the ideas, presenting regular opportunities for practice, Und
providing many chances to refresh older skills in new contexts.21 In a landmark
series of studies, Bloom demonstrated that one-on-one tutoring improved student
achievement by two standard deviations over group instruction, the equivalent of
improving the performance of students in the 50th to the 98th percentile.22
136
Innovationen / Sommer 2008
Von http heruntergeladen://direct.mit.edu/itgg/article-pdf/3/3/133/704197/itgg.2008.3.3.133.pdf by guest on 07 September 2023
Continuous Improvement in Undergraduate Education
While cognitive science research has demonstrated that learning improves
when students ask questions, classroom environments are set up for teacher
monologues rather than classroom dialogues. It is well documented that most
learning environments do not stimulate many student questions. According to one
research study, a typical student asks 17 questions per hour in a conventional class-
room and 27 questions per hour in one-on-one tutoring.23
There is significant interest in personalized learning and in how technology
can be used to tailor education to the individual and his or her pace of progress.
Technology can offer continuous monitoring of progress, provide immediate feed-
back, use this information to diagnose performance, and adjust instruction to the
learner’s level of mastery. The technology also has unlimited patience.
A skillfully designed series of challenges accepted as authentic by a learner can
get an individual into a “flow,” making them willing to spend hours mastering the
skills needed to achieve the goal.24 Flow is a state of high interest and sustained
motivation to achieve an objective and continue to make progress. Maintaining
this state requires developing a series of compelling goals that allows players/learn-
ers to advance at a rate that is not too fast to create anxiety and frustration and not
so slow that it leads to boredom. This is extremely difficult to achieve in a conven-
tional classroom environment. The most compelling computer games, Jedoch,
make effective use of this method to capture and hold a player’s attention. Eins
interesting feature of this flow is that the learner is, in effect, constantly being test-
ed but accepting the measurement as a valid test of skill. The key is an ability to
keep learning and keep trying until mastery is achieved.25 This ability to fail, learn,
and try again is the essence of successful computer games26—people are willing to
spend hundreds of hours mastering obscure and usually useless skills.
The new IT tools make it practical and affordable to deliver the kinds of per-
sonalized learning experiences and practical problem-solving that learning theo-
rists have championed for many years.
Military Training
There are clear cases where IT has been successfully used in formal education and
Ausbildung. The experience of military training is unequivocal—new technologies
can increase both the rate at which expertise is mastered and the rate at which
skills demonstrated during instruction are transferred to skills that prove of prac-
tical value in the field. A recent survey of results indicated that the current gener-
ation of instructional technology can cut either training time or cut costs by a
third.27
The U.S. Department of Defense has led the world in using advanced instruc-
tional technology—including games—for decades. The most visible early example
is the flight simulator that has consistently proven its ability to provide productiv-
ity gains and high levels of expertise, including expertise in dealing with danger-
ous situations that could not be practiced in any other way. Multiple studies
demonstrate that 60 minutes using a flight simulator is worth 30 minutes flying an
Innovationen / Sommer 2008
137
Von http heruntergeladen://direct.mit.edu/itgg/article-pdf/3/3/133/704197/itgg.2008.3.3.133.pdf by guest on 07 September 2023
Henry Kelly
airplane in terms of training efficacy.28 Simulators can be used to build expertise in
using and maintaining many different kinds of equipment, from tanks to sonar
devices. Simulated war games are now being played by teams who build not only
technical skills but the communication, trust, and interpersonal skills essential for
effective team operations.
The U.S. Navy has undertaken a systematic reorganization of its training with
startling results. In one well-studied example, they took a course that had a fixed
“seat-time” requirement and converted it into a series of computer-mediated
instructional modules with a clear performance-based test. The result was that
instead of having everyone take the same time to master the subject, the time dif-
ferent students took to meet the performance goal fell into a Gaussian distribution.
Some took longer than the previously fixed course time but, strikingly, the average
student achieved mastery in half the previously prescribed time and some achieved
mastery in 5-10 percent of the time.29
Reacting to the clear impact of the innovative instructional methods, Die
Deputy Secretary of Defense issued a directive, “Implementing the Strategic Plan
for Transforming DoD Training,” instructing the directors of all defense agencies
to implement technology training as quickly as possible.
Led in part by military medical training, technology-based instruction has
demonstrated major successes in training surgeons, anesthesiologists, first respon-
ders, nurses, and other medical professionals.30 Simulators can authentically repro-
duce much of what a surgeon must do in such procedures and provide a valid
measure of skill.31,32 The economic advantages of increasing the productivity and
effectiveness of medical training are clear because of the high price of convention-
al training methods and the expense of medical errors, which kill as many as
93,000 Americans and cost the health care system $37-$50 billion annually.33,34
Examples in Higher Education
The most common opposition to using these examples as evidence that progress
in higher education is possible is that they provide only tools for training and not
the kinds of expansive knowledge expected of a college education. Es gibt, Wie-
immer, several answers to these objections. Erste, the level of expertise that must be
acquired by pilots of advanced aircraft, people trained to repair sophisticated elec-
tronic equipment, endoscopic surgeons, and other medical professionals is
extremely high. Likewise, the combination of physiological and procedural knowl-
edge needed to successfully complete an operation experiencing unexpected com-
plications is also extremely high. Zusätzlich, there are numerous technical disci-
plines in the sciences, engineering, business, and information technology in which
technology-based simulations are promising. Zum Beispiel, an artificial tutor sys-
tem built by Carnegie Mellon and commercialized by Carnegie Learning demon-
strates a nearly threefold gain in solving complex algebra and geometry problems.35
By applying new technologies and reengineering methods, bedeutsam
improvements have been demonstrated in higher educational settings. Der
138
Innovationen / Sommer 2008
Von http heruntergeladen://direct.mit.edu/itgg/article-pdf/3/3/133/704197/itgg.2008.3.3.133.pdf by guest on 07 September 2023
Continuous Improvement in Undergraduate Education
National Center for Academic Transformation (NCAT) has worked with 30 zwei-
and four-year colleges to demonstrate that a course redesign methodology could
achieve significant improvements in quality while reducing costs in higher educa-
tion. Large-enrollment introductory courses were redesigned and supported with
online tutorials and course management; automated assessment of exercises,
quizzes, and tests; continuous computer-based assessment and feedback; course
delivery via the Internet; and the replacement of duplicative lectures, homework,
and tests with collaboratively developed online material. The NCAT approaches
reduced the time faculty and instructional personnel spent on non-academic tasks.
According to NCAT, 25 von 30 course redesign projects showed significant increas-
es in student learning while the other five showed learning outcomes equivalent to
traditional formats. Among the 30 institutions, costs were reduced by 37 Prozent
on average, with cost savings ranging from 20 Prozent zu 70 Prozent.
MANAGING CHANGE
Despite the evidence of successful IT integration into service industries, higher
education has resisted making the structural changes needed to exploit technolo-
gy in innovative ways. The traditional model of education has been called an
industrial or mass-production model. Former U.S. Deputy Secretary of Education
Eugene Hickok said, “As an institution, the schooling system we have is almost per-
fectly organized. It’s evolved over time, almost perfectly organized to sustain and
maintain itself. The way it is put together makes it very difficult to encourage the
kinds of innovation and changes that you have been talking about.”36 This, von
course, is the heart of the problem.
As Matthews points out, “The bedrock assumption that education must take
place in classrooms in which a professor teaches a group of students underlies the
entire organizational framework for higher education—affecting everything from
course accounting and faculty workload to tuition and state funding. But this
assumption is no longer valid, mainly because of advances in information technol-
ogy.”37 Most discussions about innovation in higher education involve tweaking
the existing system by squeezing out costs and waste. A move to more advanced
learning enabled by new technology tools will require a much different approach.
Effective use of technology will, Zum Beispiel, require redefining many of the
roles played by instructional faculty. It is highly likely that the best use of talented
faculty time is not giving lectures. The package of activities now bundled into a
description of a teacher could be unbundled into different roles and specialties.
There will always be a need for inspirational speakers, and for individual counsel-
ing and tutoring. The instructional staff can, Jedoch, spend much more time
working with individuals and small groups if the more routine parts of instruction
are provided by technology that can provide individualized instruction built
around simulations, virtual environments, and the games and challenges that can
be built around them. Scale economies demand that projects involve contributions
from experts around the world. Some faculty could choose to specialize in build-
Innovationen / Sommer 2008
139
Von http heruntergeladen://direct.mit.edu/itgg/article-pdf/3/3/133/704197/itgg.2008.3.3.133.pdf by guest on 07 September 2023
Henry Kelly
ing and maintaining these new systems, while others may elect to provide special-
ized services (such as in-game tutoring). It is possible that the task of building sim-
ulated historic cities, biological simulations, and other virtual worlds will be a new
kind of publication—at a minimum, a new way of organizing knowledge and
developing connections between diverse disciplines.
Obviously there are many unknowns in the way future learning systems will
operate, but it’s a rich field for exploration. Bedauerlicherweise, the kinds of changes
needed to explore them cannot be considered, let alone implemented, given the
constraints of existing university management. Such changes would require an
approach to management that explores changes that involve not just individuals
and small groups but the entire scope of activities that contribute to instruction. Es
is extremely difficult to undertake such a process, given the way institutions of
higher learning are now managed.
No Process for Managing Continuous Improvement, Innovation
Many institutions of higher education lack processes for stimulating ongoing
Innovation, performance improvement, and organizational change.
While the policy solutions advocated by Secretary of Education Margaret
Spellings’s Commission on the Future of Higher Education38 have been widely crit-
icized,39 it would be a mistake to overlook many of the problems it identified. Der
commission found, Zum Beispiel, that American higher education has taken little
advantage of important innovations that would increase institutional capacity,
effectiveness, and productivity, and that it needs to embrace a culture of continu-
ous innovation and quality improvement by developing new pedagogies, curricu-
la, and technologies to improve learning.
Im Gegensatz, most public and private-sector organizations that have successful-
ly undertaken major efforts to improve performance have done so by institution-
alizing a formal process to identify opportunities for improvement and manage
ändern. While the choice of improvement approaches, Werkzeuge, and techniques vary
widely, generic features typically include commitment from the organization’s top
Führer, a determination of the requirements of the organization’s customers or
markets, measurement and analysis to identify opportunities for improvement and
to gauge performance, workforce involvement in the improvement effort, and a
focus on the design of and improvements in work systems.40 In addition, viele
institutions of learning do not continuously identify new learning science and
technologies and evaluate these for application.
Many service businesses have achieved significant savings by focusing on their
core mission and outsourcing other activities to specialized providers. The strate-
gy also has been used successfully by a number of universities.41 It makes perfect
sense to bring in specialists from government, industry, and other fields to teach or
enrich courses—particularly advanced courses that require non-academic per-
spectives. But while the data is thin, it does appear that most universities have in
fact outsourced the teaching of much of the core curriculum—courses that the
140
Innovationen / Sommer 2008
Von http heruntergeladen://direct.mit.edu/itgg/article-pdf/3/3/133/704197/itgg.2008.3.3.133.pdf by guest on 07 September 2023
Continuous Improvement in Undergraduate Education
tenured faculty doesn’t want to teach. Non-tenure-track individuals are now half
of the teaching faculty,42 which effectively means abrogating responsibility for
managing continuous improvement in a large fraction of its courses.
Problems of Measurement
One of the biggest problems with performance improvement in educational insti-
tutions is the absence of any agreed-upon method for measuring what the institu-
tions are delivering. There is probably less information available about the opera-
tion of the $340 billion43 higher education enterprise in the United States than any other major part of the economy. There is, Zum Beispiel, no data on the courses stu- dents are taking,44 let alone information on what value they obtained from these courses. There is essentially no information available on the “value added” by dif- ferent educational institutions that would account both for the performance of students emerging from the institutions and the backgrounds and abilities of stu- dents entering the system. Metrics are essential for any effective system to reward institutions that are doing a better job in instruction. As Massy points out, “Absent direct information on quality, prospective students must use surrogates like selectivity, faculty-student ratios, research prowess, and even price itself.”45 The root problem comes with defining what should be delivered by higher education. Newman, Couturier, and Scurry observe that “almost never do institu- tionen [of higher education] set out a clear statement of what the student is expect- ed to learn in terms of knowledge, intellectual skills, competencies, and attitudes in order to become a well educated graduate.”46 Indeed, faculties are often adverse to attempts to define these goals. Massey notes that “each professor is entitled to his or her system of values and priorities, and no authority should be able to say that one system is better or worse than another.”47 Expertise is notoriously difficult to define, let alone measure, and a key output question rests not just with the end point achieved, but also the value added by any given educational experience. Measuring educational gains directly is difficult if not impossible in many liberal studies and advanced courses that deal with special- ties only a few faculty members have mastered. But this reality should not block efforts to collect the information that is essential for improving the basic and introductory courses taken by most of the 20 million people participating in U.S. higher education, and the large number of courses aimed at providing specific employment skills. The lack of high-quality measurement prevents effective internal management and makes it impossible for ordinary markets to work. In trying to determine which college to attend, students have no practical way to compare the efficacy of competing institutions. They recognize that the prestige of an institution will help them get jobs, Aber, as Massy points out, “prestige doesn’t equate to education qual- ity.”48 A graduate from a prestigious institution may be better qualified because of a diligent admissions process and the ability to associate with other highly talent- Innovationen / Sommer 2008 141 Von http heruntergeladen://direct.mit.edu/itgg/article-pdf/3/3/133/704197/itgg.2008.3.3.133.pdf by guest on 07 September 2023 Henry Kelly ed students, but not necessarily because of quality instruction. Other surrogates, such as the annual college rankings in U.S. News & World Report, are widely denounced by universities that offer no other performance measurements or explanation of how the market for good education should oper- aß. The value of a degree is in fact linked to the degree-granting institution’s pres- tige more than to its demonstrated ability to convey knowledge, and prestige is almost entirely linked to the faculty’s research accomplishments—not its teaching expertise. This creates a double distortion in the market for educational value: stu- dents and the public may place a higher value on prestige than on educational value, and therefore use research-driven prestige as a measure of educational value. Jedoch, the connection between prestige in research and an institution’s ability to add value to a student’s mastery of complex subjects is very weak. With the exception of small seminars focused on a research faculty’s narrow line of work, there may not be much of a link between an institution’s research prowess and its ability to deliver educational value to the general population of its students. CAN WE DO BETTER? Higher education is under enormous pressure to provide a large percentage of the population with sophisticated skills at low cost. Größtenteils, these institu- tions have not taken a serious look at the structural changes needed to do this. This is essentially a problem of management. The demand is enormous, tools to meet the new demand are available, but the process of connecting solutions to problems is not functioning effectively. Given the high national stakes involved, the federal government has a unique responsibility to intervene in ways that facilitate the search for solutions. It should not do this by imposing a rigid set of prescriptions, but should instead encourage the emergence of an infrastructure that would facil- itate the development and vetting of new approaches, backed by a serious and sus- tained program of research that could both drive basic invention and provide national metrics of success. A strategy of four key components could be considered. Metrics The earlier discussion focused on measuring the quality of individual institutions. The capabilities of modern information technologies, Jedoch, open an entirely new set of possibilities: systems that permit superior approaches to delivering learning services, which will enable education institutions to acquire national and international markets quickly if they achieve results. Such systems would permit the operation of a worldwide market for effective approaches to learning, inde- pendent of the specific institution delivering instruction. These markets would have to operate on a national or international scale to be effective. The basis of competition would presumably include price, time to achieve mastery, and the quality of the student’s experience. 142 Innovationen / Sommer 2008 Von http heruntergeladen://direct.mit.edu/itgg/article-pdf/3/3/133/704197/itgg.2008.3.3.133.pdf by guest on 07 September 2023 Continuous Improvement in Undergraduate Education Such a system would depend on agreeing how to evaluate whether an individ- ual had achieved an acceptable level of expertise in a field—something that may well be impossible in some fields—and it would require some market-like mecha- nism with which students, faculty, and others could rate and rank different approaches so that market-like forces could drive continuous improvement. In principle, the market could evaluate both the approach to instruction (Did the stu- dents find it engaging and effective? Did subsequent instructors find the students well prepared?) and the methods used to evaluate expertise (Did the measurement translate into effective performance in either a job or an advanced course?). Some market-like mechanism is essential to achieve a process of continual invention and improvement in educational service delivery. Traditional methods of using large statistical samples to determine whether an educational intervention is working make no sense in an environment where many experiments must be underway simultaneously and where the underlying technology is changing con- tinuously.49 Many modern firms now make the bulk of their income from products and services that weren’t available a year earlier. They are forced to rely on agile methods for testing new concepts and on markets to tell them whether they are succeeding. It is difficult to maintain a solid research base that ensures that con- cepts requiring long-term basic inquiry get adequate support. The federal govern- ment has a unique role in filling this gap—an issue I will discuss later. The metric of a student’s expertise driving the markets could mean a student’s mastery of a complex body of knowledge as evidenced by an ability to perform dif- ficult tasks. Student expertise can also mean an ability to operate effectively in sit- uations closely mimicking work environments—whether this means perfecting social skills, a capacity for assembling information and making decisions, or an ability to display complex engineering or laboratory expertise in difficult situa- tionen. Any process of defining metrics of expertise should include both scholars in the field and the people who may hire the graduates. Employers should participate in defining both types of expertise they need in new hires and should be willing to accept performance-based testing as a criteria for making hiring decisions. Institutions of higher learning must also be willing to accept such measurements of student performance as the basis for moving to higher levels of instruction and for receiving degrees and other certification. It is much more likely that two-year institutions and for-profit colleges and universities would be willing to participate in such an experiment with metrics of student expertise if employers were also willing to link measured performance to hiring. Employees taking courses that lead to specific certificates or certifications would be obvious early candidates. The danger of starting with specific skills and certifications is that it may make it too easy for prestigious universities and colleges to argue that the system may work for working-class skills but never for the high-level intellectual achievements expected of their graduates. Natürlich, simulation-based exploration and learning cannot replace all conventional learning. There should be a variety of tools that innovations / Sommer 2008 143 Von http heruntergeladen://direct.mit.edu/itgg/article-pdf/3/3/133/704197/itgg.2008.3.3.133.pdf by guest on 07 September 2023 Henry Kelly could support to varying degrees a wide range of learning modalities—tutors, sim- ulations, collaborative platforms, virtual-delivery mechanisms, etc.—that can be applied to a wide range of learning objectives, approaches, and course materials. A New National Learning Infrastructure The national and international markets for innovative learning systems just described require a new kind of infrastructure to operate effectively. It is clearly possible to build such a system by adapting available commercial products for use in learning. Service enterprises worldwide have developed radically improved business practices that depend in large part on an infrastructure that supports con- stant collaboration and customer access, but with little regard for geographic loca- tion or work site. Efficient markets operate in ways that reward successful strate- gies with return customers and provide strong incentives for continuous innova- tion. No such tools are available to education organizations, in part because the very idea of a large team working to build and improve a course of instruction is not part of the current approach to education. Given the right infrastructure and review process, there is good reason to believe that many individuals working in higher education would willingly participate if the tools were available. Many peo- ple have contributed to the development of Wikipedia and the virtual-world Second Life, and a growing number of businesses have deployed platforms that encourage the public to help the company solve problems, identify innovations, and improve products. It would be possible, Zum Beispiel, to have large numbers of people worldwide collaborate in building sophisticated simulations that could be used for many dif- ferent types of instruction. Some of these could involve building historic cities complete with architecture, Kunst, und Musik, and simulations of the economic and cultural activities underway in these cities. The construction would require contri- butions from historians in many fields. Each object and activity would be peer reviewed. A viewer walking through the virtual city would be able to open any object (sagen, by clicking on it) and open a path to a complete set of documentation, pointers to online reference material, and review comments made while the object was being created. This could be continuously updated. A similar approach could be used to build a sophisticated simulation of anatomy or other biological models. These systems could be collaboratively built through a process managed in a way that combines a journal publication and software production. Instead of being built and abandoned, the system would be continuously improved by new research and by responding to comments from users. One enormous advantage of the new virtual-world platforms such as Wonderland, Second Life, Croquet, and Active Worlds50 is that they ensure that codes will not be rendered obsolete by changes in computer operating systems, graphic chips, and other changes that have often left sophisticated simulations stranded by obsolete media. Built collectively by many contributors, much like traditional publications, 144 Innovationen / Sommer 2008 Von http heruntergeladen://direct.mit.edu/itgg/article-pdf/3/3/133/704197/itgg.2008.3.3.133.pdf by guest on 07 September 2023 Continuous Improvement in Undergraduate Education these virtual worlds would combine knowledge from individual experts in a unique way. All objects in the virtual world would need to connect directly in the most concrete ways: virtual representations of historic buildings, Zum Beispiel, would be located in the (virtual) sites where the actual buildings were located. Since all objects in the virtual world will be linked seamlessly to the 2D web and citations to traditional literature, the new publication form would be an extension of rather than a substitution for traditional methods. Once such systems were built, other teams of people could use them as the basis for a variety of approaches to learning. The learning could be as simple as leading students on a guided tour through the space and asking questions, or it could be as sophisticated as designing elaborate problem sets and challenges where success requires mastery of the subject matter. Instructors or assistants could par- ticipate by appearing as avatars in the space or manipulating the simulation in response to student actions. Avatars with artificial intelligence could also provide answers and pointers to the literature. Learning modules could be rated by instruc- tors and students—important parts of the system. Assembling a course could include exploring the most successful learning modules developed worldwide. The way these markets could evolve is unclear; Jedoch, individual universities could establish their own branding by offering experiences in publicly available virtual worlds that are supported by unique contributions of their own faculty, who could meet students in person and in the virtual spaces. Like the World Wide Web itself, the technical core of the system would be an agreed-to set of standards that would allow combining different software services. Most of the software needed to deliver component services (z.B., instructional management tools, virtual 3D worlds, identity management tools, project manage- ment tools, tools for locating people with appropriate skill sets such as Facebook, and standardized approaches to graphics and scripting) is already available.51 The computing and communication needed to support a national and international system are already in place. There are precedents for a publicly supported education infrastructure. A gen- eration ago, the Corporation for Public Broadcasting used federal funds to pur- chase satellite communication services to deliver public broadcasting information, including a significant amount of educational video material.52 The National Science Foundation has invested heavily in supercomputer centers and high-speed networking that connects university and college campuses.53 It is also possible that major foundations would be willing to provide startup support for such a system if there were evidence that it would be used effectively. While a significant amount of the material created should be in the public domain, there must also be room for proprietary and protected extensions. Individual universities would continue to compete on the basis of the quality of their campuses faculty and facilities and access to faculty mentors who would facil- itate online learning. The missing element in all of this is a management system capable of building the few pieces of software needed to permit the system to operate, and encourag- Innovationen / Sommer 2008 145 Von http heruntergeladen://direct.mit.edu/itgg/article-pdf/3/3/133/704197/itgg.2008.3.3.133.pdf by guest on 07 September 2023 Henry Kelly ing agreement on standards. This could facilitate the formation of teams that would build key instructional simulations (z.B., the cities and other simulations). Success will depend on whether higher education is willing to provide the incen- tives needed to let scholars participate in the construction of these simulations and whether they are willing to accept the skills demonstrated in them as valid meas- ures of student expertise. Experiments with New Organizational Models The proposed infrastructure would permit ambitious institutional experiments in a virtual world that could short-circuit the elephantine decision-making processes that would have to be undertaken to conduct such experiments within existing institutions. Bedauerlicherweise, the volunteer users will remain marginal without a willingness from higher education institutions to experiment with the structural and institutional reforms needed to make these new tools more than a curiosity. The key issue is how to encourage experiments in the use of new technologies in higher education. It is possible that incentives provided by both employers and a federal agency could encourage experiments on a national level. The incentives would have to be both financial—inducements through hiring based on an agreed- to metric—and prestigious—based on proving that a new approach allowed for quicker mastery and used a method that learners found inherently more engaging. By far the majority of courses offered in higher education deal with basic sub- jects in engineering, biology, Psychologie, and other areas where large numbers of students take virtually identical classes. Teaching in these subject areas could ben- efit enormously from a technical infrastructure designed to help students master ideas at rates tailored to each individual, and it could also help close the gap between abstract knowledge and practical applications. It’s likely that even some advanced seminars could benefit if the faculty led a small group of students, who could be physically located in many parts of the world, through a simulated human cell showing how recent discoveries have updated standard models of cell biology. One intriguing possibility is that if learning productivity could in fact be increased by 30 percent or more,54 it might be possible for students to master the basic skills of a two-year college graduate by the 12th grade, thus making these skills a universal requirement of a high school education. Rapidly accelerating basic learning may seem out of reach, given the current K–12 system’s struggle to meet even elementary quality goals. Jedoch, a new approach could well increase the quality and interest in education in ways that could reengage the large number of students now bored or burdened with second- rate instruction. If this becomes possible, higher education could focus on both building key 21st-century skills and the specialized credentials needed in fields such as law, engineering, and medicine. 146 Innovationen / Sommer 2008 Von http heruntergeladen://direct.mit.edu/itgg/article-pdf/3/3/133/704197/itgg.2008.3.3.133.pdf by guest on 07 September 2023 Continuous Improvement in Undergraduate Education Research and Development Research in educational theory, pedagogy, and the neuroscience of education has greatly expanded our understanding of how people learn. Little work has been done, Jedoch, on the challenge of converting this knowledge into practical, actionable programs that can achieve learning productivity gains for large num- bers of people—even though it would be difficult to find an area where research investment could have a more important impact on the nation’s future. There is currently no national infrastructure for funding or conducting the ambitious pro- gram of research, Entwicklung, demonstration, and evaluation needed in educa- tion. We urgently need a structure capable of emulating the kinds of government, university, and business collaborations supported by institutions like the Semiconductor Manufacturing Technology Consortium (SEMATECH) which plays such a critical role in maintaining U.S. competitiveness in semiconductors. The “National Center for Research in Advanced Information and Digital Technologies,” recently passed as a part of the Higher Education Opportunity Act, has the potential to be the nucleus of such an operation.55 Detailed roadmaps of research needs have been designed by groups of experts spanning many disciplines.56 Research needs include developing a better under- standing of how people learn and master expertise most effectively in complex fields using technology-enabled learning tools, tailoring the pace and nature of instruction to individuals and teams, and finding ways to both define and measure the real expertise of these individuals and teams. CONCLUSION There’s no question that U.S. universities have been key enablers of national scien- tific and technical progress for a generation and have set a gold standard for uni- versities worldwide. But these institutions have resisted the new technology, man- agement, and organizational approaches that have enabled other service-providing enterprises to improve quality and cut costs. This failure is making it increasingly difficult to ensure that all Americans have access to the quality of educational serv- ices that the modern economy demands. Außerdem, rapidly rising tuition has put the issue into the national spotlight. Instead of reacting with creative new approaches, Jedoch, it is possible that the reforms introduced to cut costs will actually reduce the quality of the education delivered or force educational institu- tions to seek ever greater levels of funding. The economist William Baumol has argued that there is no great harm in the growing cost of education, since the increasing returns to education justify paying more.57 This is both a false choice and an example of market inefficiency. Just because the returns to education are rising is no excuse for failing to seek greater quality and efficiency of education and reducing its cost. Improving the quality and reducing the cost of education is especially important, given that a broader portion of the population, many with moderate economic means, need access to innovations / Sommer 2008 147 Von http heruntergeladen://direct.mit.edu/itgg/article-pdf/3/3/133/704197/itgg.2008.3.3.133.pdf by guest on 07 September 2023 Henry Kelly some postsecondary education and training simply to maintain a middle-class sta- tus. Other information-intensive service industries have found creative ways to use new technology to improve what they offer, including personalizing services. There is every reason to believe that the same result can be achieved in education. Given the resistance higher education institutions have to the kinds of changes required, Jedoch, external force may need to be applied to stimulate the development of the basic processes and skill sets needed to attain essential change. 1. Examples of thoughtful efforts in improving undergraduate education can be found at
“Effectiveness of Reading and Mathematics Software Products: Findings from the First Student
Cohort,” but its approach was widely criticized. See H. Kelly,
“Technology Can Transform
Schools,” Education Week, Aprill 3, 2007.
50. See http://vworld.fas.org/bin/view.
51.
Für
sehen
http://faslearn.org/universalclient/index.php?title=Universal_Opensource_Virtual_Worlds_Clie
nt_Whitepaper
verfügbar
survey
Werkzeuge
von
A
52. See Corporation for Public Broadcasting http://www.cpb.org/aboutcpb/.
53. See the National Coordination Office for Networking and Information Technology
www.nitrd.gov.
54. See Fletcher, Advanced Technologies for Learning.
55. HR 4137, Abschnitt 802. http://www.rules.house.gov/110/text/110_hr4137.pdf
150
Innovationen / Sommer 2008
Von http heruntergeladen://direct.mit.edu/itgg/article-pdf/3/3/133/704197/itgg.2008.3.3.133.pdf by guest on 07 September 2023
Continuous Improvement in Undergraduate Education
56. Learning Federation. For broad research goals and medical research, see http://www.fas.org/pro-
sehen
for using game concepts
grams/ltp/publications/roadmaps.html;
http://www.fas.org/programs/ltp/publications/summit/index.html.
learning,
In
57. Quoted in James Duderstadt, A University for the 21st Century, University of Michigan Press, Ann
Laube, MI, 2000, P. 140.
Innovationen / Sommer 2008
151
Von http heruntergeladen://direct.mit.edu/itgg/article-pdf/3/3/133/704197/itgg.2008.3.3.133.pdf by guest on 07 September 2023
PDF Herunterladen