www.sciencemag.org SCIENCE VOL 344 11 APRIL 2014 155
Engaging students in inquiry practices is known to motivate them to persist in science, technology, engineering,
and mathematics (STEM) fields and to create
lifelong learners (1, 2). In inquiry, students
initiate investigations, gather data, critique
evidence, and make sophisticated drawings
or write coherent essays to explain complex
phenomena. Yet, most instruction relies on
lectures that transmit information and multiple-choice tests that determine which details
students recall. Massive Open Online Courses
(MOOCs) mostly offer more of the same. But
new cyber-learning tools may change all this,
by taking advantage of new algorithms to
automatically score student essays and drawings and offer personalized guidance.
Empowering Learners, Aiding Instructors
Inquiring students make predictions, gather
new ideas (from investigations, visualizations, or observations); use evidence to distinguish among their predictions and ideas;
and create a coherent explanation (1, 3, 4).
When they write explanations, students learn
more than when they select among multiple-choice answers or read explanations (5).
Experimenting with visualizations and making drawings to illustrate ideas can develop
students’ spatial-reasoning skills (6). And,
when students analyze resources to develop
an explanation, they appreciate the beauty
and complexity of new fields (7).
Analyzing students’ essays or drawings and using the results to guide them
can improve outcomes (8), but this requires
more human capital than precollege and college instructors have. Precollege instructors
often have five or six classes of 30 to 40 students, and college instructors may have hundreds or thousands of students in required
courses or MOOCs.
However, advances in computer tech-
nologies may help offset limits in instructor
time and effort. Immediate, personalized,
computer-generated guidance can motivate
students to deepen their understanding of
complex materials. Instructors can review
the automated scores to identify students
who continue to flounder. Because these
systems can assign guidance to every stu-
dent, even students who are reluctant to ask
for help can progress.
To make sure that inquiry activities lead to
new insights and not to erroneous or superfi-
cial conjectures, experienced teachers moni-
tor student progress and regularly add hints to
keep students on the right track (3, 4, 9) (see
the first figure). To figure out what hints will
help students explore a complex problem,
researchers and teachers (often collabora-
tively in professional development programs)
analyze large numbers of student essays or
drawings and try out alternative approaches
(3). Online learning environments like the
Web-based Inquiry Science Environment
(WISE) streamline this process by record-
ing student ideas and supporting experiments
for which researchers can randomly assign
diverse forms of guidance.
These experiments show that guidance
encouraging students to distinguish among
their predictions and new evidence helps
students to better integrate their ideas about
the topic (9). Once the links between student
responses and effective guidance are established, environments like WISE can score
student essays or drawings and automatically
assign guidance designed to help students
develop coherent explanations.
Guiding Writers, Drawers, and Teachers
Advances in natural language processing now
enable computer-based learning environments to use scored answers to create systems
for scoring future responses. For instance, the
Educational Testing Service’s (ETS) “c-rater”
tool used human-rated responses to develop
to Improve Science Learning
EDUCATION TECHNOLOG Y
Marcia C. Linn1*, Libby Gerard1, Kihyun Ryoo2, Kevin McElhaney3, Ou Lydia Liu4, Anna N. Rafferty5
Automated guidance on essays and drawings
can improve learning in precollege and college
1Graduate School of Education, University of California
Berkeley, Berkeley, CA 94720, USA. 2School of Education,
University of North Carolina, Chapel Hill, Chapel Hill, NC
27599, USA. 3SRI International, Menlo Park, CA 94025,
USA. 4Educational Testing Service, Princeton, NJ, 08541,
USA. 5Computer Science Division, University of California
Berkeley, Berkeley, CA 94720, USA. *Corresponding author.
Students were asked to use stamps to represent the chemical reaction between two methane
molecules (CH4, ) and oxygen (O2, ) to yield carbon dioxide (CO2, ) and water (H2O, ).
The drawing interface allowed students to create multiple “frames” in their drawing, with the first
frame representing the reactants and the second frame representing the products.
Good start. You have correctly created 2 frames that represent the reactants and products
of the methane combustion reaction.
Can atoms in the reaction be spontaneously CREATED OR DESTROYED?
Reread the direction and revisit steps 3.6–3.8. Then improve your drawings.
Computerized hints to improve understanding. Student response to the chemical reactions item that WISE
can automatically score. Automated guidance was as effective as guidance provided by an expert teacher.
[Adapted figure, based on WISE]
Automated guidance was as effective as guidance provided
by an expert teacher.