Impact!
This is a Paper on what impact is, particularly with regard to the application of learning technologies, and it starts with Newton's laws of motion.
There are two problems with attempting to measure Impact. First is how does one define the criteria that allows measurement. Secondly, how do we know that what we are measuring is a unique consequence of the effect of pure e-learning?
This explanation attempts to provide answers to both questions by deconstructing the processes and makes suggestions for how impact can be better understood as it relates to e-learning.
Impact is the outcome of two actions.; a striking force on an object and its equal and opposite reaction as a result of that strike.
In education, we might say impact is the degree of effort put into the enterprise of change as a means of enjoying the the (hoped for) benefits of the resultant equal and opposite reaction.
It follows that the degree of benefit should outweigh the challenge and turbulence caused by effecting the change.
Graphically it would appear like this:
Impact assesses the striking force between physical objects which should cause an equal and opposite reaction, yet in e-learning terms, ‘technology in action’ in this sense is metaphysical and therefore beyond physical laws of measurement. Capturing a sense of how a teacher or student feels, relative to how they felt prior to the change (excited, engaged, supported, driven, challenged, enthralled, bored, disappointed) becomes the language of impact. Impact is assessed by the emotive responses recorded by those affected. That language is based on the experience summed up in the rate and profundity of change against the benefits felt and the degree of turbulence caused in getting through and settling the change. Attitude itself is an important yet overlooked critical factor in the degree of take up and success engendered.
Distinguishing the unique e-learning contribution - The double-negative list
Using a double-negative list is a method of distilling the features of change that are unique e-learning activity. This is achieved by writing down everything that would not have happened had the change not occurred. Discipline is required to leave out things that happened for other reasons, or would have happened regardless of the e-learning intervention. Sometimes it may be necessary to adjust the language to claim an aspect of change if not the whole element. The double negative route provides a list of discrete ‘positives’. This list then needs to be tested with the objects of the reasons for change. A narrative is produced where those are asked to explain the difference in the quality of experience of the change. The experience will reflect on the degree of turbulence was not felt and how the learner and learning experience has changed for the better.
Benchmarking
External comparison is of very limited value. It gives a sense of where you are generally relative to others. But both these things are of no value in improving anything and knowing what is needed is already known or available in a quick and easy meeting with an expert mentor (RSC etc?). Benchmarking suggests that there is ultimately a common structure and behaviour that all can strive for, led by the few. It suggests it is possible to standardise the personal and yet personalising is one of the great enduring strengths of e-learning. Valuable effort is used debating relative merits. Of much greater value is to know that progress is being made internally, according to learners and that there is discernible progress over time in the stories captured and outcomes achieved. Critically, it is being able to contribute that knowledge with evidence into the wider reporting procedure so e-learning can make informed contributions to progress in teaching, learning and assessment.
Finding evidence
As technology fragments more and more, reducing to single applications, supporting personalised, perhaps idiosyncratic learning, the ability to make mass assessment of technology centrally is reducing and telling less and less of the real story. Ways need to be found of capturing examples of impact in a central place as indicative of what is being reported by learners and what is occurring in their learning record. A solution might be the ability to capture evidence in a separate community using e-portfolios is an area worthy of exploration.
The inverse rule of Impact
It is possible that we have been deceived, in the past, in our search for the impact of technology, expecting it to be physically manifested and overt in everything teachers do. It could be that successful impact is evidenced by the opposite; technology use being one element in many artfully constructed student-centred learning experiences that are part of everyday practice such that the technology is not really seen nor acknowledged, simply being used when deemed appropriate for learning as planned. This appears to be the case with the use of Moodle VLEs to pick one example.
Perhaps only new or troublesome or unsatisfactory technologies with little immediate application to practice remain on the surface and open to view, and thus to questioning through the turbulence they cause to the smooth running of the curriculum? The greatest impact of technology actually occurs where its use is not seen, nor recognised and only emerges through prompts that promote reflection on practice. The value of technology is not in the way it alters the nature of education but alters its delivery and the structure of our working day.
What we are attempting to do then in testing the idea of impact is to consider how teachers and support staff might work without the support of technology assess how long the College would remain open if technology were removed.
This explanation attempts to provide answers to both questions by deconstructing the processes and makes suggestions for how impact can be better understood as it relates to e-learning.
Impact is the outcome of two actions.; a striking force on an object and its equal and opposite reaction as a result of that strike.
In education, we might say impact is the degree of effort put into the enterprise of change as a means of enjoying the the (hoped for) benefits of the resultant equal and opposite reaction.
It follows that the degree of benefit should outweigh the challenge and turbulence caused by effecting the change.
Graphically it would appear like this:
Impact assesses the striking force between physical objects which should cause an equal and opposite reaction, yet in e-learning terms, ‘technology in action’ in this sense is metaphysical and therefore beyond physical laws of measurement. Capturing a sense of how a teacher or student feels, relative to how they felt prior to the change (excited, engaged, supported, driven, challenged, enthralled, bored, disappointed) becomes the language of impact. Impact is assessed by the emotive responses recorded by those affected. That language is based on the experience summed up in the rate and profundity of change against the benefits felt and the degree of turbulence caused in getting through and settling the change. Attitude itself is an important yet overlooked critical factor in the degree of take up and success engendered.
Distinguishing the unique e-learning contribution - The double-negative list
Using a double-negative list is a method of distilling the features of change that are unique e-learning activity. This is achieved by writing down everything that would not have happened had the change not occurred. Discipline is required to leave out things that happened for other reasons, or would have happened regardless of the e-learning intervention. Sometimes it may be necessary to adjust the language to claim an aspect of change if not the whole element. The double negative route provides a list of discrete ‘positives’. This list then needs to be tested with the objects of the reasons for change. A narrative is produced where those are asked to explain the difference in the quality of experience of the change. The experience will reflect on the degree of turbulence was not felt and how the learner and learning experience has changed for the better.
Benchmarking
External comparison is of very limited value. It gives a sense of where you are generally relative to others. But both these things are of no value in improving anything and knowing what is needed is already known or available in a quick and easy meeting with an expert mentor (RSC etc?). Benchmarking suggests that there is ultimately a common structure and behaviour that all can strive for, led by the few. It suggests it is possible to standardise the personal and yet personalising is one of the great enduring strengths of e-learning. Valuable effort is used debating relative merits. Of much greater value is to know that progress is being made internally, according to learners and that there is discernible progress over time in the stories captured and outcomes achieved. Critically, it is being able to contribute that knowledge with evidence into the wider reporting procedure so e-learning can make informed contributions to progress in teaching, learning and assessment.
Finding evidence
As technology fragments more and more, reducing to single applications, supporting personalised, perhaps idiosyncratic learning, the ability to make mass assessment of technology centrally is reducing and telling less and less of the real story. Ways need to be found of capturing examples of impact in a central place as indicative of what is being reported by learners and what is occurring in their learning record. A solution might be the ability to capture evidence in a separate community using e-portfolios is an area worthy of exploration.
The inverse rule of Impact
It is possible that we have been deceived, in the past, in our search for the impact of technology, expecting it to be physically manifested and overt in everything teachers do. It could be that successful impact is evidenced by the opposite; technology use being one element in many artfully constructed student-centred learning experiences that are part of everyday practice such that the technology is not really seen nor acknowledged, simply being used when deemed appropriate for learning as planned. This appears to be the case with the use of Moodle VLEs to pick one example.
Perhaps only new or troublesome or unsatisfactory technologies with little immediate application to practice remain on the surface and open to view, and thus to questioning through the turbulence they cause to the smooth running of the curriculum? The greatest impact of technology actually occurs where its use is not seen, nor recognised and only emerges through prompts that promote reflection on practice. The value of technology is not in the way it alters the nature of education but alters its delivery and the structure of our working day.
What we are attempting to do then in testing the idea of impact is to consider how teachers and support staff might work without the support of technology assess how long the College would remain open if technology were removed.