Testimony of Dr. Carol A. Twigg
Executive Director
Center for Academic Transformation
Rensselaer Polytechnic Institute

Committee on Education and the Workforce
Subcommittee on 21^st Century Competitiveness

July 10, 2003

Mr. Chairman and Members of the Subcommittee:

Thank you for inviting me to testify. I am Executive Director of the Center for Academic Transformation at Rensselaer Polytechnic Institute. The Center manages a national program in higher education course redesign. Its goal is to demonstrate how effective use of information technology can improve student learning and reduce instructional costs.

American colleges and universities continue to be challenged by the need to increase access to higher education, to improve the quality of student learning, and to control or reduce the rising cost of instruction. These issues are, of course, interrelated. As tuition costs continue to rise, access is curtailed. If the quality of the curriculum inhibits students from successfully completing courses and programs, promises of increased access become hollow.

Solutions to these challenges appear to be interrelated as well. Historically, either improving quality or increasing access has meant increasing costs. Reducing costs, in turn, has meant cutting quality, access, or both. In order to sustain higher education’s vitality while serving a growing and increasingly diverse student body, it must find a way to resolve this familiar —and seemingly intractable--trade-off between cost and quality.

Many colleges and universities are also discovering exciting new ways of using technology to enhance teaching and learning and to extend access to new populations of students. For most institutions, however, new technologies represent a black hole of additional expense. This is because the majority have simply bolted new technologies onto an existing set of physical facilities, a faculty already in place, and an unaltered concept of classroom instruction.

Under these circumstances, technology becomes part of the problem of rising costs rather than part of the solution. In addition, comparative research studies show that, instead of improving quality, most technology-based courses produce learning outcomes that are only "as good as" their traditional counterparts—what has come to be known as the "no significant difference" phenomenon. By and large, colleges and universities have not yet begun to realize the promise of technology to improve the quality of student learning, increase retention and reduce the costs of instruction.

Supported by an $8.8 million grant from the Pew Charitable Trusts, the Program in Course Redesign (http://www.center.rpi.edu/PewGrant.html) was created in April 1999 to address the issues discussed above. Managed by the Center for Academic Transformation (http://www.center.rpi.edu/) at Rensselaer Polytechnic Institute, the program is supporting colleges and universities in their efforts to redesign instruction using technology to achieve quality enhancements as well as cost savings.

Selected from hundreds of applicants in a national, 30 institutions each received a grant of $200,000, awarded in three rounds of 10 per year. Participating institutions include research universities, comprehensive universities, independent colleges, and community colleges in all regions of the United States. Detailed descriptions of each redesign project can be found on the Center Web site.

All 30 redesign projects focus on large-enrollment introductory courses that have the potential to affect significant student numbers and generate substantial cost savings. Why focus on such courses? Because undergraduate enrollments in the United States are concentrated heavily in only a few academic areas. In fact, just 25 courses generate about half of all student enrollments in community colleges and about a third of enrollments in four-year institutions.

The topics of these courses are no surprise and include introductory studies in disciplines such as English, mathematics, psychology, sociology, economics, accounting, biology, and chemistry. Successful completion of these courses is critical for student progress toward a degree. But typical failure rates in many of these courses—15 percent at research universities, 30 to 40 percent at comprehensive universities, and 50 to 60 percent at community colleges—contribute heavily to overall institutional drop-out rates between the first and second year.

The insight that these figures point to is simple and compelling: In order to have a significant impact on large numbers of students, an institution should concentrate on redesigning the 25 courses in which most students are enrolled instead of putting a lot of energy into improving quality or cutting costs in disparate small-enrollment courses. By making improvements in a restricted number of large-enrollment prerequisite or introductory courses, a college or university can literally affect every student who attends.

The Program in Course Redesign has produced many different models of how to restructure such courses to improve learning as well as to effect cost savings. In contrast to the contention that only certain kinds of institutions can accomplish these goals, and in only one way, the program is demonstrating that many approaches can achieve positive results. And to counter the belief that only courses in a restricted subset of disciplines—science or math, for instance—can be effectively redesigned, the program contains successful examples in many disciplines including the humanities, math and statistics, the social sciences, and the natural sciences. In each case, the whole course rather than a single class or section is the target of the redesign.

Here is a breakdown of the thirty participating institutions by curricular area:

QUANTITATIVE (13)

SOCIAL SCIENCE (6)

HUMANITIES (6)

SCIENCE (5)

[For specific descriptions of each project, please see the case studies at http://www.center.rpi.edu/PewGrant/ProjDesc.html.]

Each of the 30 participating institutions is conducting a rigorous evaluation focused on student learning, comparing the outcomes of redesigned courses with those of courses with the same content delivered in a traditional (pre-redesign) format. Preliminary results show improved student learning in 20 of the 30 projects, with the remaining 10 showing no significant difference. Each institution has also developed a detailed cost analysis of both the traditional and redesigned course formats, using a spreadsheet course-planning tool (http://www.center.rpi.edu/PewGrant/Tool.html) developed by the Center.

Preliminary results show that all 30 reduced costs by 40 percent on average, with cost saving ranging from 20 percent to 84 percent. Other positive outcomes associated with redesigned courses include increased course-completion rates, improved retention, better student attitudes toward the subject matter, and increased student satisfaction with the new mode of instruction. Collectively, the 30 redesigned courses impact more than 50,000 students and produce a cost savings of $3.6 million each year--while improving student-learning outcomes and increasing retention at the same time.

The redesign projects have effected significant changes in the teaching and learning process, making it more active and learner-centered. The primary goal is to move students from a passive, "note-taking" role to an active-learning orientation. As one math professor involved in the project put it, "Students learn math by doing math, not by listening to someone talk about doing math." Lectures are replaced with a wide variety of learning resources, all of which involve more active forms of student learning or more individualized assistance. In moving from an entirely lecture-based to a student-engagement approach, learning is less dependent on words uttered by instructors and more dependent on reading, exploring, and problem-solving undertaken actively by students.

Many of the projects show statistically significant improvements in overall student understanding of course content as measured by pre- and post- assessments that examine key course concepts. For example, at the University of Central Florida, students enrolled in a traditionally configured political science course posted a 1.6-point improvement on a content examination, while the average gain of 2.9 points for students in the redesigned course was almost double that amount. The University of Tennessee, Knoxville found a significant and favorable 5-point difference between student scores on a redesigned course exam in Spanish and the scores of students enrolled in traditional sections.

Other projects demonstrate statistically significant improvements in student understanding of course content by comparing the performance of students enrolled in traditional and redesigned courses on commonly administered examinations. Redesign-course students in statistics at Penn State, for example, outperformed traditional students on a content-knowledge test, with 60 percent correct answers in the traditional format and 68 percent correct in the redesigned classes. At Carnegie Mellon University, the performance of redesign-course students in statistics increased by 22.8 percent on tests of skills and concepts, and redesign-course students also demonstrated an enhanced ability to identify the appropriate statistical analysis to employ in a given real-world problem situation. At Florida Gulf Coast University (FGCU), the average score achieved on a commonly administered standardized test by students enrolled in the traditional fine arts course was 70 percent; in the redesigned course it was a significantly higher 85 percent.

Many of the projects also reported significant improvements in their drop-failure-withdrawal (DFW) rates. At the University of Southern Maine (USM), a smaller percentage of introductory psychology students dropped the redesigned course or received failing grades, moving the DFW rate from 28 percent in traditional sections to 19 percent in the redesigned course. At Virginia Tech, the percentage of students completing a redesigned linear algebra course and achieving grades of D- or better improved from an average of 80 percent to an average of 87 percent. At the University of Idaho, the percentage of students earning a D or failing was cut by more than half. Drexel University reduced its DFW rate in computer programming from 49 to 38 percent, FGCU from 45 percent to 11 percent in fine arts, Indiana University-Purdue University Indianapolis (IUPUI) from 39 to 25 percent in introductory sociology, and the University of New Mexico from 42 percent to 25 percent in psychology.

What techniques have the projects found to be the most effective in improving student learning? The most prominent are the following:

Continuous Assessment and Feedback: Shifting the traditional assessment approach in large introductory courses, which typically employ only midterm and final examinations, toward continuous assessment is an essential pedagogical strategy in these redesigns. Many of the projects include numerous computer-based assessments that give students almost instantaneous feedback on their performance. Automating assessment and feedback enables repeated practice as well as providing prompt and frequent feedback--pedagogical techniques that research consistently has proven to enhance learning.

Students are regularly tested on assigned readings and homework using short quizzes that probe their preparedness and conceptual understanding. These low-stakes quizzes motivate students to keep on top of the course material, structure how they study and encourage them to spend more time on task. Online quizzing encourages a "do it till you get it right" approach: Students are allowed to take quizzes as many times as they want to until they master the material.

Quizzes also provide powerful formative feedback to both students and faculty members. Faculty can quickly detect areas where students are not grasping key concepts, enabling timely corrective intervention. Students receive detailed diagnostic feedback that points out why an incorrect response is inappropriate and directs them to material that needs review. Since students are required to complete quizzes before class, they are better prepared for higher-level activities once they get there. Consequently, the role of the instructor shifts from one of introducing basic material to reviewing and expanding what students have already been doing.

Increased Interaction among Students: Many redesign projects take advantage of the Internet’s ability to support useful and convenient opportunities for discussion among students. Students in large lecture classes tend to be passive recipients of information, and student-to-student interaction is inhibited by class size. Through smaller discussion forums established online, students can participate actively. Central Florida and IUPUI create small online discussion groups in which students can easily contact one another in their redesigned American government and introductory sociology courses. Students benefit from participating in the informal learning communities that are created in this manner. Software allows instructors to monitor the frequency and quality of student contributions to these discussions more readily and carefully than would be the case in a crowded classroom.

Continuous Support: A support system, available around the clock, enables students to receive help from a variety of sources. Helping students feel that they are a part of a learning community is critical to persistence, learning, and satisfaction. Active mentorship of this kind can come from a variety of sources, allowing students to interact with the person who can provide the best help for the specific problem they have encountered.

Many of the redesign projects replace lecture time with individual and small-group activities that take place in computer labs staffed by faculty, graduate teaching assistants (GTAs) and/or peer tutors. In several instances, increasing lab hours has enabled students to get access to more one-on-one assistance. Students welcome the reduction in lecture time and the opportunity to work in groups to apply what they have learned. Collaboration also triggers peer pressure within groups, which can be a powerful incentive for students to keep up with their work.

Online Tutorials: In redesign courses, Web-based resources—either commercially produced or developed by colleges and universities—that support greater student engagement with the material replace standard presentation formats. Such resources may include interactive tutorials and exercises that give students needed practice; computerized or digitally recorded presentations and demonstrations; reading materials developed by instructors or in assigned textbooks; examples and exercises in the student’s field of interest; links to other relevant online materials; and individual and group laboratory assignments.

Ideally, materials like these are modularized and tailored to incorporate examples drawn from a variety of disciplines to match the learning circumstances of students with different professional and personal goals. Using modularized materials also allows changes in content or format if students are having difficulty understanding a particular part of the course.

The University of Wisconsin at Madison and Virginia Tech are among the most sophisticated users of online tutorials. Building on substantial experience in using and developing interactive materials, Wisconsin had developed 37 Web-based instructional modules in chemistry by July 2001. Each module leads a student through a particular topic in six to 10 interactive pages. When the student has completed the tutorial, a debriefing section presents a series of questions that test whether the student has mastered the module’s content. Students especially like the ability to link from a problem they have difficulty with directly to a tutorial that helps them learn the concepts needed to solve the problem.

Virginia Tech uses a variety of Web-based course-delivery techniques like tutorials, streaming video lectures, and lecture notes as tools for presenting materials in a linear algebra course. Consisting of concrete exercises with solutions that are explained through built-in video clips, such tutorials can be accessed at home or at a campus lab. In redesigned courses, tutorials have taken over the main instructional task with respect to transmitting content: 84 percent of the students enrolled in Virginia Tech’s linear algebra course reported that the computer presentations explain the concepts effectively.

Undergraduate Learning Assistants (ULAs): Both the University of Colorado-Boulder (UC) and SUNY at Buffalo (UB) are employing ULAs in lieu of Graduate Teaching Assistants (GTAs). Both universities have found that ULAs turned out to be better at assisting their peers than GTAs because of their understanding of the course content, their superior communication skills, and their awareness—based on their own recent experience--of the many misconceptions that undergraduate students often hold.

In Colorado’s redesigned introductory astronomy course, the instructor meets weekly with the ULAs and discusses in detail what is working and where students are having difficulty. Feedback from these weekly meetings gives the instructor a much better sense of the class as a whole, and of the individual students in it, than would otherwise be possible with a class of more than 200 students.

People who are knowledgeable about proven pedagogies that improve student learning will find nothing surprising in the above list. Among the well-accepted Seven Principles for Good Practice in Undergraduate Education developed by Arthur W. Chickering and Zelda F. Gamson in 1987 are such items as "encourage active learning," "give prompt feedback," "encourage cooperation among students," and "emphasize time on task." Good pedagogy in itself has nothing to do with technology, and we’ve known about good pedagogy for years. What is significant about the faculty involved in these redesigns is that they were able to incorporate good pedagogical practice into courses with very large numbers of students—a task that would have been impossible without technology.

In the traditional general chemistry course at the University of Iowa, for example, 21 GTAs used to be responsible for grading more than 16,000 homework assignments each term. Because of the large number of assignments, GTAs could only spot-grade and return a composite score to students. By automating the homework process through redesign, every problem is graded and students receive specific feedback on their performance. This, in turn, leads to more time on task and higher levels of learning. Applying technology is not beneficial without good pedagogy. But technology is essential to move good pedagogical practice to scale, where it can affect large numbers of students.

There are a variety of ways to reduce costs. As a result, there are also a variety of strategies for pursuing instructional redesign, depending upon institutional circumstances. For instance, an institution may want to maintain constant enrollments while reducing the total amount of resources devoted to the course. There are two primary ways an institution can decrease costs per student even though the number of students enrolled in the course remains unchanged. First, it can use technology for those aspects of the course where it would be more effective, engaging faculty only in tasks that require faculty expertise. Second, it can transfer other tasks that are less academically challenging to those with a lower level of education. This approach makes sense when student demand for the course is relatively stable.

But if an institution is in a growth mode or has more demand than it can meet through existing course delivery, it may seek to increase enrollments while maintaining the same level of investment. Many institutions have escalating demand for particular subjects like Spanish or information technology that they cannot meet because they cannot hire enough faculty members. By using redesign techniques, they can increase the number of students they enroll in such courses and relieve these academic bottlenecks without changing associated costs. The University of Tennessee, for example, has been able to increase by one-third the number of students served by the same instructional staff in introductory Spanish.

Another way to reduce costs is to decrease the number of course repetitions due to failure or withdrawal, so that the overall number of students enrolled each term is lowered and the required number of sections (and the faculty members to teach them) are reduced. At many community colleges, for example, it takes students about two-and-a-half tries to pass introductory math courses. If an institution can move students through in a more expeditious fashion by enabling them to pass key courses in fewer attempts, this will generate considerable savings--both in terms of institutional resources and in terms of student time and tuition.

As noted earlier, 13 of the 30 projects have thus far reported a noticeable decrease in DFW rates, ranging from 10 to 20 percent. As an example of the levels of resources that can be saved, Central Florida has calculated the savings resulting from a 7 percent increase in course retention in its American government course. Applying this rate to 25 redesigned sections results in a one-course-section reduction, amounting to a $28,064 cost savings each time the course is offered.

Not surprisingly, many of the redesign projects are trying several of these approaches to saving resources simultaneously. All intend to reduce course repetitions. In each case, a translation of the savings to cost-per-student can be used for comparative purposes.

What are the most effective cost-reduction techniques used by the redesign projects? Since the major cost item in instruction is personnel, reducing the time that faculty members and other instructional personnel invest in the course, and transferring some of these tasks to technology-assisted activities are key strategies. Some of the more predominant cost-reduction techniques used by the projects include:

Course management systems can automatically generate many different kinds of tailored messages that provide needed information to students. They can also communicate automatically with students to suggest additional activities based on homework and quiz performance, or to encourage greater participation in online discussions. Using course-management systems radically reduces the amount of time that faculty members typically spend in nonacademic tasks like calculating and recording grades, photocopying course materials, posting changes in schedules and course syllabi, sending out special announcements to students—as well as documenting course materials like syllabi, assignments, and examinations so that they can be used in multiple terms.

Online quizzing sharply reduces the amount of time faculty members or GTAs need to spend on the laborious process of preparing quizzes, grading them, and recording and posting the results. Automated testing systems that contain large numbers of questions in a database format enable individualized tests to be easily generated, then quickly graded and returned.

Virginia Tech’s use of similar online course delivery techniques in its linear algebra course has enabled radical reductions in teaching staff. Individual faculty members are no longer required to present the same content through duplicative efforts. Nor do they need to replicate exercises and quizzes for each section. Interactive tutorials can replace part—and, in some cases, all--of the "teaching" portions of the course.

For example, Penn State has constructed an easy-to-navigate Web site for its introductory statistics course that contains not only material on managing the course but also a large number of student aids and resources, including solutions to problems, study guides, supplemental reading materials for topics not otherwise treated in the text, and student self-assessment activities. Putting assignments, quizzes, exams and other course materials on a community Web site for the course can save a considerable amount of instructional time.

Although the employment of ULAs was in these cases originally driven by the need to reduce costs, ULAs have also proven more effective than most GTAs, as noted earlier. Another solution, implemented by Rio Salado College, is to employ a "course assistant" to address the many nonacademic questions that arise as any course is delivered—questions that can characterize up to 90 percent of staff interactions with students. This frees the instructor to handle more students and to concentrate on academic interactions rather than logistics.

With regard to cost savings, the redesign methodology is an unqualified success. Redesigned courses are reducing costs by an average of 40 percent, with specific savings ranging from 20 percent to 84 percent. Collectively, the 30 courses are expecting a savings of about $3.6 million annually. Some are saving more than they planned to; others less. Round I projects planned to reduce costs by about 37 percent, with a range of 20 percent to 71 percent. They actually reduced costs by 33 percent on average, with a range of 16 percent to 77 percent. Round II projects planned to reduce costs by about 44 percent, with a range of 20 percent to 84 percent. They actually reduced costs by 43 percent on average, with a range of 25 percent to 74 percent. Round III projects are on track in their plans to reduce costs but final results are not yet available.

Why is there such a large range in cost savings across the projects? Differences are directly attributable to the different design decisions made by the project teams, especially with respect to how to allocate expensive faculty members. Redesigns with lower savings tended to redirect, not reallocate, saved faculty time: They keep the total amount of faculty time devoted to the course constant, but they change the way faculty members actually spend their time (for example, lecturing versus interacting with students.)

Others substantially reduce the amount of time devoted to the course by non-faculty personnel like GTAs, but keep the amount of regular faculty time constant. Decisions like these reduce total cost savings. By radically reallocating faculty time to other courses and activities, in contrast, Virginia Tech shows cost savings of 77 percent in its redesigned linear algebra course. But most of the other projects could have saved more with no diminution in quality, if they had made different design decisions.

By using technology-based approaches and learner-centered principles to redesign their courses, these 30 institutions are showing us a way out of higher education's historical trade-off between cost and quality. Some of them rely on asynchronous, self-paced learning modes, while others use traditional, synchronous classroom settings but with reduced student/faculty contact hours. Both approaches start with a careful look at how best to deploy all available instructional resources to achieve the desired learning objectives. Questioning the current credit-for-contact paradigm of instruction, and thinking systematically about how to produce more effective and efficient learning, are fundamental conditions for success.

Now that it is clear that redesign can produce substantial savings, a natural next question is who should benefit from these savings? Legislators would probably prefer to see some, if not all, of the savings passed on to the public or consumers—by reducing tuition, for example. If the institution retains some or all of the savings, though, how should those funds be reallocated? Should the resulting extra resources, for example, be reinvested in the ongoing course development? Perhaps the academic unit should capture the savings to reinvest in further course redesign. Or should the savings be returned to the institution to be reallocated for other uses?

If the savings are captured by the department or institution, there are relatively few incentives for faculty members to improve productivity by increasing enrollment or improving retention. Some project participants therefore believe that the faculty members involved in the redesign should benefit directly as reward for increased productivity. If the individual instructor captures the savings in the form of faculty time, it may mean she or he has more time to do research or more time to pursue personal interests. How an institution rewards faculty and staff for increased productivity is thus an important consideration in building the case for academic restructuring.

The 30 institutions participating in the Program in Course Redesign intend to use the savings generated in many different ways. In descending order of popularity (counts of projects provided in parentheses), cost savings in the 30 participating institutions will

• stay in the department to support continuous improvement of the course and/or for the redesign of other courses (9);
• be used to underwrite a greater range of course offerings at the upper division or graduate levels(9);
• allow the institution to accommodate greater numbers of students with the same resources (4);
• stay in the department to reduce teaching loads and to provide more time for research (3);
• allow the institution to redesign similar courses outside of the original department (2);
• enable the institution to offer distance learning courses that were previously impossible due to resource constraints (1);
• allow the institution to reduce rental expenditures as a result of the reduction in face-to-face class time (1); and
• be used to improve the training of part-time faculty (1).

Once institutions start creating pools of surplus instructional resources in this manner instead of just consuming everything available, we will be forced to rethink many of our assumptions about planning and budgeting. A host of institutional policy issues about who gets what and for what will be involved, as well as numerous practical matters like ensuring continuous investment to support the innovations that will be needed to keep on generating such cost savings.

Higher education has traditionally assumed that high quality means low student-faculty ratios, and that large lecture-presentation techniques supported by cheap labor constitute the only viable low-cost alternatives. But it is now clear that course redesign using technology-based, learner-centered principles can offer higher education a way out of this historical trade-off between cost and quality. New models demonstrate that it is indeed possible to improve learning and reduce costs at the same time. For the first time, we can have our cake and eat it too.