From: Subject: Testing Higher Order Skills in Mathematics Date: Sat, 18 Jul 2009 10:42:41 +0100 MIME-Version: 1.0 Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Content-Location: file://C:\Documents and Settings\Jeff Ross\Local Settings\Temporary Internet Files\OLK1CC\2009-7-EAA-Swansea.html X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.5579 Testing Higher Order Skills in Mathematics

Testing Higher Order Skills in=20 Mathematics

Christopher Sangwin

CAA in mathematics

This talk concerns CAA of mathematics.=20

Introduction

STACK: a computer aided assessment system for = mathematics.=20 The focus is on=20

In = CAA

 if simplify(sa-ta) =3D 0 then =
 mark :=3D 1 else mark :=3D 0&= nbsp;=20 We are assessing a student provided answer. This is an = objective=20 test. This is not MCQ (or similar).=20

System demonstration

STACK http://localhost/stack-1-0/=20

Mathematical activity

There are two important strands: (1) = The use of=20 routine techniques.=20 (2) Problem solving.=20 Principle 1: mathematicians try = to solve=20 problems. Principle 2:
standard algorithms are both useful and = are=20 cultural artifacts. Principle 3: mathematicians justify their=20 solutions.
The outcome is a correct chain of reasoning. Principle = 4:=20 accuracy is important. Principle 5: it is important to = acknowledge the=20 place of conventions.=20

Tensions

Linking CAA to class work

  1. Solve the cubic ....=20
  2. Find the equation of the line tangent to two adjacent roots.=20
  3. Find the coordinates of the intersection of the tangent line with = the=20 cubic.

Assessing mathematics

In mathematics the majority of tasks may = be=20 classified:=20
1. Factual recall
2. Carry out a routine calculation or algorithm
3. Classify some mathematical object
4. Interpret situation or answer
5. Prove, show, justify - (general argument)
6. Extend a concept
7. Create an example/instance
8. Criticize a fallacy
Why = does this=20 matter?=20
Assessment strategies dictate
learning = styles.=20
hence=20
We may influence learning styles through = choice of=20 assessment strategies.
In contraposition ... if we = do not=20 make conscious choices of assessments we can hardly complain about how = students=20 develop!=20

Checking for properties

CAS can also check for = properties.=20 Example question 1  
Give an = example of a=20 prime number. The CAS checks whether a student's answer is = prime.=20 To mark Example question 2  
Give = an odd=20 function. 1. calculate f(x)+f(-x),
2. = simplify,
3. check equality to zero.=20

Creating examples/instances

Some questions ask for examples of = objects.=20 They require higher level thinking. Such questions are rare. (11.5 = questions=20 from 486 =BB 2.4 %)
Pointon=20 and Sangwin, 2003 Perhaps because they are time consuming to = mark.=20 CAA may mark some questions of this style. Exemplar=20 questions=20

Students' answers

Students show great variety in their answer, = and=20 method. For example, 190 students were asked for two functions that = satisfy=20 f=A2(1)=3D0. Their answers were marked by = CAA. The students=20 (N=3D190) gave 93 `different' answers.
1st Answer 2nd Answer
x2-2 x 45 x3-3 x 29
[(x2)/2]-x 31 x2-2 x 10
[(x3)/3]-x 11 [(x3)/3]-x 9
x2-2 x+1 7 (x-1)2 8
x2-2 x+3 7 [(x4)/4]-x 8
(x-1)2 5 x4-4 x 5
2 x2-4 x 5 ex-1-x 1
[(x3)/3]-[(x2)/2]=20 5 ex-1+e-x+1 1
0 4 ln(x)-x 1 =
Two=20 strategies emerged:=20
JL: Ok, just take the parabola and shift it one. =BC B: I said, x-1=3D0, then=20 integrated it.
These problems can be used to generate = (short)=20 discussions.=20

Automatic feedback

Sophisticated automatic feedback may be = provided=20 by
computer algebra systems. This=20

Feedback

One third of feedback interventions decreased=20 performance. Kluger, A. N. and DeNisi, A., = Psychological=20 Bulletin (1996). The nature of feedback determines its = effectiveness.=20

Common misconceptions

Computer algebra can also test for a = type=20 of incorrect answer. Misconceptions may be identified by=20

Odd functions

On examining the odd functions given by students, = the=20 majority of coefficients ( =B9 1) are = odd,
eg=20
Students'=20 concept image of an odd function requires odd coefficients.=20 Furthermore, f(x)=3D0 is odd, but was absent.=20

Functions that are odd and even.

When asked for a = function that=20 was both odd and even 35% gave the correct answer = (eventually),
35%=20 failed to answer the question. Incorrect answers revealed that 24% of = the=20 students added an odd and even function. Examples include
3x5,   =20 5x7,    7x5-3x.
<= /TBODY>
x+x2,   =20 x2+x3,    = x5-x6.
The=20 computer algebra system can test for these misconceptions.=20

Conclusion

The important features of STACK=20