Assessing Creativity?

Introduction

Right then well I'm going to talk about assessing creativity. My name is Mark Taylor-Batty, I work in the Workshop Theatre which is part of the School of English in the University of Leeds here. And the reason I have been asked to talk about this is that we have a lot of students who quite clearly who are interested in becoming actors or directors, even though we are not a drama school. But we teach the theory of drama and the history of drama and the application of drama and we do that through practice as much as we do through theory, so there will be a number of practical classes attached to the courses we provide and in these classes of course we expect them to engage with dramatic material either to create themselves or to engage with a script that exists whether it be Shakespeare or something contemporary, you know, or whatever there is.

And in doing that of course we are asking them to harness their own creativity. The problem therefore comes, when that creativity is attached to something that is being assessed, how do we assess that material, are we assessing their creativity, because of course creativity tends to produce better product than not being creative. Or is that incidental to the material that we are essentially assessing. So that's the kind of philosophical problem if you like in terms of assessing that kind of material.

So what we did is we created over the years we've created and changed and improved a set of criteria for the assessment of practical work and that is what I have got on the first 3 slides.

We broke this down into three categories translation of source material to the stage which can mean a number of things, I'll talk about that. The individual engagement with the group creative process because of course we are assessing individuals but out of necessity they have to work in groups and so that causes certain dynamics which have to be considered.

And finally the kinds of skills that they have to bring to the assessed work and that is also problematical of course because we are not a drama school so we are not teaching them how to act or how to breath properly so that they can project. How to develop those kinds of skills, so it is unfair to assess their ability to act for example, whilst asking them to act. So that has to be addressed as well.

Criteria used for assessment 1

So on the first slide we have got a set of criteria we have developed over the years for the translation of source material to the stage, and when we say that, source material can be anything of course it doesn't necessarily mean translation of a script to the stage, but it could be whatever matter that you are working with and how that generates performative material.

And what I'll do is I'll just go through the list of things that are on that slide.

So for the purposes of assessing practical work it is considered important that:

1. an understanding of stage convention and dramatic form is apparent in the translation;
2. there is evidence of appropriate and imaginative use of dramatic techniques;
3. there is evidence of the ability to select and synthesise dramatic material arising out of rehearsals;
4. the appropriate means of communicating dramatic material to specified audiences;
5. the final presentation demonstrates the ability to deal with complex dramatic structures;
6. the approach chosen by the group in the process of rehearsal or devising within the rehearsal is articulated to the spectators whoever they may be.

Now if you look through those criteria they are things that are quite easy to identify. An understanding of stage conventions, evidence of appropriate choices having been made, appropriate means of communicating. So for example if they want to demonstrate some material that they want to have some kind of political force perhaps. If they want to change the opinions or address the opinions they can guess in advance a body of people might already have. What dramatic might best facilitate that provocation if it is.

Or if they simply wanted to entertain them what dramatic structures might best do that. And it is very clear to make decisions as to whether or not those things have been done regardless of whether they have been done well you could say.

So to put a play on which nobody laughs at, nobody enjoys, are bored by but we can see that they made the right decisions but they perhaps don't have the talent to see it through fully and successfully. Now its rare that they fail to that degree but never the less what we are doing in that situation is not assessing how good the piece is, we are assessing how well they have approached the construction of that piece.

So I mean obviously there are things to do with how they are communicating at the time. How successfully they are portraying things but that comes in a later set of criteria.

So what I am saying is quite clearly these are things that can be identified that are not essentially to do with how creative they are being.

Criteria used for assessment 2

If we go to the second slide we have got the individual engagement with the group creative process. Now to be honest this set of criteria is here to make sure they are aware we are not going to let them sit back and let other people do the work and that has to be the case with group work, but they are also things that can be identified for example.

Individual initiative and input within the group process.

When you are working with a group of people you can see who are generating the ideas and who are leading and who are following. The problem with that is of course so people may be good at leading but don't get the opportunity so that is the teachers responsibility to make sure that those things are facilitated.

Individual commitment to a harmonious collaborative process, in other words don't be a pain, you know. Get on with it, do what's needed even if you don't necessarily agree with something, find a way of negotiating something different rather than not turning up or being belligerent.

An awareness of the need for constructive criticism. In other words, if you don't agree find a way of dealing with it. And individual discipline in the careful organisation of time. Well that just saying turn up on time, you know, be punctual.

But nevertheless those are the things we can use as criteria and they are very easy to identify, they are not to do with creativity. Perhaps they are not the most important of the criteria but its to with getting the group to work together so that can facilitate the creation of things that can be assessed by the other criteria.

Criteria used for assessment 3

And then we come to the more difficult ones and that's performance and practical skills and I'll read through these to start with.

1. an engagement with and understanding of the dramatic material being offered in performance;
2. an understanding of the effective use of stage space;
3. a supportive relationship with other performers having been developed;
4. an awareness of creative interaction with the audience;
5. the ability to communicate through performance the ideas developed in the rehearsal process;
6. an awareness of and, where appropriate, a creative use of the technical elements of performance.

Now these have all been written in such a way that we can say we can identify these things. So its an engagement with, an understanding of, an awareness of, the ability to.

So in other words we are saying these are things that can be achievable and they are identified and therefore measured. They do involve creativity of course so an engagement with, an understanding of the dramatic material being offered in performance can require a creative engagement with that material but the creativity of it, the originality of it, the genius if you like of what they might produce, is something that we can identify but not measure.

Creativity like apathy is some thing that is recognisable but difficult to measure but if you can frame it, if you frame what might be creative within these ways we find it easier to therefore consider it. Likewise as I was saying earlier if you had somebody who is a bad director or a naive director or somebody who is a developing actor but not quite at the peak of their powers yet, that's kind of irrelevant if they demonstrate an awareness of creative interaction with the audience so for example they know that certain techniques might be employed to get the audience on your side. They know that certain use of silences in certain rhythms, or certain alignment of bodies towards the audience creates a different kind of investment from that audience in a material. They might have the awareness of that, they might not just be particularly good at actually doing it situ.

Hopefully we have got the experience enough to be able to recognise the former without necessarily worrying too much about the latter.

Now if we were a drama school of course the latter would be the thing that we would be considering important but we are not, we are not there to train actors. We are there to train people to consider dramatic material, just as in English you wouldn't expect people to be teaching people how to be novelists, but you want them to be able to consider and reflect about processes that novelist's employ.

The 'learning journal'

So that kind of covers that area. I want to move on to talking about how we use a learning journal as part of a package of methods that we employ in order to help us assess creative material but also to help the students to consider the manner in which that material is being generated for assessment but also generated for learning.

You know learning and assessment are very different things. Students don't always appreciate that especially more and more these days we tend to find students are more and more obsessed, perhaps understandably with the numbers that come at the end of a module and certainly in recent years I have found students are more and more engaging with the process by which we generate those numbers and what rights they have in relationship with those. Perhaps rightly so, its not a problem but its something that seems to me deflects from, takes away from a consideration of the process of learning which as far as I'm concerned is why they are there, not to get the collection of numbers which then goes towards the diploma, although of course that is the ultimate goal.

So the learning journal hopefully helps the students to concentrate on that learning process, or the learning journey we like to refer to it as, so they can think of it as something that doesn't have just an end product but is a series of stations if you like that lead towards an end product which is then assessable. We have got a set of criteria for the assessment of the learning journal and they are these:

1. Regular upkeep and satisfactory completion of the journal.
2. The degree to which it is evident that students have explored topics raised in the sessions through personal study.
3. The degree to which it is evident that students have examined and engaged with concepts and problems that have arisen in discussions or as a result of practical work.
4. The success and impact of the student's communication of any insight gained.

Now we are not asking them, you might say that these are the kinds of criteria that might sometimes be applied to writing of essays, but what we are not asking them to do is write essays. We are asking them to capture their thoughts, to reflect on material they might have done in a rehearsal.

And of course when it comes to rehearsing we as the teachers, as the facilitators can't always be there. If they have got 20 hours worth of rehearsal on top of class time its simply not feasible for us always to be there. The learning journal is a way of getting us, allowing us access to the material that they are generating, that it wouldn't be profitable for us to sit back and watch. But in doing that it is also forcing them to focus on whatever processes they are going through. A consequence of that is that they can very often recognise when something is useful and when something isn't so useful. They can see what mistakes they've made and they have an opportunity to consider those mistakes in advance of going ahead and correcting them and all that is then captured in ways that allow us to see the degree to which they are interrogating the processes that we are asking them to consider and that is very, very useful.

And that can be used, that kind of material, that body of material can be used to moderate a mark or it can be used as something that can be marked in itself, just as one might mark an essay, but it doesn't have the disciplines of an essay. It doesn't have a specific word length and doesn't even have to be spelt correctly or be grammatical. It can just be snatches in the corner of a piece of paper, then turn over and write something a bit more formal.

We do try and ask them not to write too much because we can sometime end up with volumes, but on the whole we allow them to be fairly flexible with that.

And so that is the learning journal and as I say we find it very useful for moderating final work because of course in performance very often it is the final piece most people are focused on. Either those who are coming to see it or those engaged with it can just see this horizon that they are aiming towards but learning is a process and how they get to that product is just as important and sometimes more important than what that product is in itself.

Not only because they are learning and we want them to engage with that learning but also because that body of material that they are generating is in itself evidence of engaging with knowledge and so therefore is assessable.

The 'learning contract'

We also use in some modules something called the learning contract. Perhaps contract is the wrong term for it, it sounds a little bit too formal and it does get them a little bit worried. Perhaps in a useful way sometimes, but perhaps also in ways that hinder its function and we are considering changing the name. Nevertheless at the moment it is still called the learning contract and what we ask them to do is in certain modules write down in this contract what it is they are aiming to do, how they are aiming to do it. What methodologies they might employ.

And what the objectives are, what they hope to prove or what they hope to discover. Its very loose and flexible and we don't hold them to it. We do say this may change, but it is very useful in order to get them thinking in advance.

Specifically in the case of performance to stop them thinking about that horizon as being the be all and end all. When do we get the makeup, when do we get the clothes. Don't worry about it, what are the processes that get you to those decisions rather than worrying about how you are going to materially furnish that final production.

So on this slide I have a series of descriptions really of the contract. Firstly it's a pedagogical tool not a tool of assessment. We do use it to frame how we might later assess.

So we might have a discussion with them and say look you've stated these are your methodologies we will therefore employ these criteria to assess those and we can negotiate that with them if they are not happy about how we might go about assessing it.

We ask then for more information about how they feel it would be best assessed. But nevertheless in the first instance it's a tool that allows them to consider what processes they are going to use and so it is therefore pedagogical.

Its pedagogical because its about getting students to think about how their work might relate to the criteria that will be used to assess it. That's useful because it allays those fears. If they get all obsessed about whether its going to be 60 or a 70 something that can actually get in the way of the processes that will get them to that 60 or 70 something.

But if they can say look these are the criteria, how do you think what you are doing is going to be assessed by those? And if you are really obsessive, how are you going meet these criteria rather than just end up with those and although that might seem a little pandering to that kind of obsessiveness it does actually help them to relax and get back in to enjoyment of learning and embracing the process of learning.

It is also useful because it gets students to focus on aims objectives and possible outcomes of research.

In other words to engage with what actually research means and at an undergraduate level we find that very, very fruitful. The students that don't normally consider or have to consider research techniques and methodologies because they are doing practice they can't avoid that and by emphasising that with a contract they realise that they are doing something a little bit more than just putting a play on or doing a rehearsal process. This is actually a body of research and it can be articulated as such and once you articulate it as such you then start improving it. You start realising there are all these other things that you can bring in to develop it. So it is pedagogical and useful from that point of view.

And the final bullet point I suppose I have already covered. It gets students to consider methodologies, it makes them think about what processes they are going to go through in order to move towards an assessed body of material.

Assessing Creativity?

So I've come to my final slide after I've gone through all these criteria and the methods that we employ to the question, are we assessing creativity at all really when we are asking these students to be creative in their engagement with the material we are looking at.

Basically I say that there are these 3 things we are doing when we are assessing their creative work.

One we are assessing a student's understanding of a body of knowledge, whether that is about theatre, the application methods involved in practice or the theory of theatre.

Two we are judging their ability to think critically about that knowledge and the use to which that knowledge can be put.

And three we are considering their ability to communicate their perceptions about these things, and I would argue that there is no difference there with the assessment of any traditional material or any kind of assessment is essentially that. The student's engagement with a body of knowledge and their ability to communicate understandings and perceptions from engaging with body of knowledge.

So my argument I suppose, is that we are not assessing creativity but that we are aware that creativity comes into everything that anyone does when learning to different and varied degrees. Its something that one recognises but because the parameters of creativity are so flexible it can't be defined it can't be measured but we recognise that it is there and we can find ways of considering how its contributed to a process which can be defined, can be identified and can be measured.