School of Education, University of Wales, Bangor
© EURODL 2000
A Needs Analysis and Instructional Systems Design
BA and the creation of knowledge.
Accountability, Evaluation and Change
This paper arises from the need to find formalisms for designing learning environments mediated by WWW and internet technologies based on collaborative, creative and reflexive activity. The limitations of earlier systematic approaches of educational systems design are discussed as an introduction to other approaches. The paper describes some of the ideas that inform the types of systems to be developed based on social cultural approaches to human activity. Guidelines and heuristics drawn from these ideas are investigated in relation to formalisms for describing the design of general computer systems proposed by a universal modelling language. There is a brief illustration of how the approach is applied to the design of a specific course in development.
It is increasingly recognised that the real capital of an organisation is in its knowledge. The knowledge of an organisation is distributed through its human resources, its data sources its processes and methods and in the tools it uses or design. Models of training and learning are often based on deficit models of learners/trainees, whereas significant knowledge is already vested in those learners/trainees. Historic patterns of the application of learning technology are often based on the efficient transmission of "new knowledge" to learners who lack that knowledge. However learning to be better at a job is not the same thing as getting a new skill. This paper is based on current experiences in designing on-line learning for initial and continuing professional development of professionals in school teaching and vocational education and training.
Much of the description is based on the REM project, which was a Telematics Application Programme Project of the European Commission DGXIII from 1996 to 1999. A virtual learning environment for developing collaborative, constructive learning. Learning was designed based on undertaking real tasks with other learners across the Internet. The following characteristics featured in the system:
Experience in this research programme lead us to consideration of fundamental course design issues. Course design needs to be situated in both the social and cultural context of the learner and the host institution and put the discourse between the learners, tutors and other actors at the centre of the design process.
There are sufficient ideas and theories available for the conception of learning environments (virtual or otherwise) that can assist in the formulation of designs for these learning environments. However there is a clear need for an overall planning framework which can help us design new courses which map on to the aspirations of reflexive, situated and collaborative professional development and training. This paper will outline some of the key emergent education and training ideas that inform thinking about the design of such training systems, and a search for a tool that can help the analysis and synthesis of learning and training experiences.
The process of development of multimedia resources has a long history. Key factors in design have been an analysis of needs and what is to be taught followed by a systematic choice of media appropriate to the specific instruction. Content is at the centre of the design process. We will describe why the prescribed methods were not used in our design processes.
There is a process that has been called rational curriculum design or managed learning (see for instance Department of Employment (1993)). This has often been subject to some criticism. A particularly robust criticism came form Stenhouse (1975). The rational design process is often characterised by writing a list of things that a trainee can do at the end of a period of training (often specified in behavioural terms) and then reducing that list to ever finer and finer concepts or actions until it becomes "teachable". The strongest and best articulated descriptions of the practice of instructional systems design can be found in the works of Gagné. The process of needs analysis in such systems is the process of reducing the final behaviour into teachable units and finding appropriate instructional strategies. It has been a key tool in educational technology and has been a classic methodology in producing multimedia packages of earlier generations as described by Townsend (1976). The system is particularly favoured for training as compared with education.
This model is found wanting in several respects. Merril and others (1990) suggest that the prevailing model of instructional design have the following flaws. There is a:
In the design of our materials we felt that many of the above
criticisms of the instructional design process valid. We needed to break away from
prescriptive and reductionist approaches.
When one begins to examine the work of professionals it is clear that real professional life is about integration of their knowledge into contexts which are changing continuously. This does not deny the need for knowledge of specific "factual" knowledge however, in many professions career does not derive from professional knowledge but professional practice. The knowledge base is not valued unless it is contextualised.
There is a considerable difference in emphasis in the notion and techniques required for professional development as compared with a technical approach to training. This has been articulated by Bevis et al (1990). Professional education requires judgement rather than answers. Bevis compares a technical model against a professional model for clinical nurse education as a table (Table 1). This does indeed provide a dichotomy that curriculum design has to span or avoid. However the dichotomy between knowledge transmission and professional competence in modern United Kingdom orthodoxy in vocational activity, however, which would encourage us to search for an alternative approach.
The Technical and Professional Models (from Bevis , Em, O. and Watson , J. 1990)
|Technical Model||Professional Model|
|The only learning worth evaluating can be seen as behavioural changes.||Worthwhile learning is often personal, obscure and private. Only some learning appear as behavioural changes.|
|Everything that exists, exists in some quantity, and therefore can be counted and measured.||Many things that exist are note externally verifiable.|
|The teacher selected
goals are the important ones, therefore the evaluated ones.
Both teacher and student selected Important, therefore the evaluates ones
|Both teacher and student selected goals are important, as is learning attained without goals.|
|Comparing behaviours to some objectively held criteria or comparing to the progress of other students determines how well something is learned.||Educative learning can not be rated on a scale. Most learning can not be compared either to some "objectively" conceived criteria or to the progress of other students.|
|The teacher-student relationship is hierarchical and the teachers assigns students by how well they have met specific objectives.||The teacher-student relationship is egalitarian. Learning requires a process of trusting grades to exploration among expert and novice learners and thrives on constructive criticism.|
|The quality of rigour of a course can be determined by how well it helps its students meet the discipline requirements as reflected by test scores, attainment of behavioural objectives, and accreditation requirements, since these reflect the agreed upon discipline content.||The quality of rigour of
a course can be determined by how well it helps students
collect paradigm experiences, develop insights, see patterns, find meanings in ideas and experiences, explore creative modes of enquiry, examine assumptions, form values and ethics in keeping with the moral ideal of the caring scholar-clinician, respond to social needs, live fully and advance the profession.
"In a process of enlightenment, there can only be participants." Jurgen Habermas
There is a clear need for the development of alternative
conceptions in the use of learning technology that can provide practical guidelines in the
implementation of new systems that support learning with is based on practitioners
reflection on their work, which gains strength from collaborations with their colleagues.
This is particularly true when developing computer mediated courses to be made available
in open and distant modes, as the "system" is the reality of course for its
The process of professional development within a formal framework ie a course needs to take into account the socio-cultural context in which such a course is going to exist. This is particularly true when the course is innovative (however most changes are necessarily innovative). In a course of professional development the process of development of a new professional involves engagement in a community; and knowledge and understanding of the professional activity is knowledge distributed amongst a community and artefacts (documents, classrooms in our case and so on).
Following the work of Vygotsky (1980), Engestrom and Cole (1993) propose a model for understanding systems of distributed knowledge as cultural-historic activity systems.
Figure 1 describes CHAS
In the initial formulation of activity in the world, Vygotsky
illustrates the idea through a simple triangular relation between the subject, the object
and the mediation system. The mediation system being the linguistic and cultural
(including tools) means in which the communication between subject and object take place.
Engestrom and Cole's formulation extends this model to incorporate wider issues including
the rules (such as legislative framework), community (eg the shared value system of
significant colleagues), and division of labour (eg who else is involved in the enterprise
and what are co-responsibilities).
For the course designer this framework of analysis facilitates the identification of those issues that need to be considered in the development of a new activity system, which in turn may need to fit into and existing activity system.
In terms of a teacher development system, which the author is developing, it may be necessary to take into account the following factors:
Rules may include:
The Community may include:
The division of labour may include:
A trainee may find that they are in a school that is avoiding
innovative activity because of government inspection and thus do not experience the
opportunity of "being innovative". In another case the school community may
reward administration (with promotion) over and above other aspects of a teachers work. In
other cases the rules may inhibit the trainees access to a photocopier thus limiting their
ability to produce their own unique resources.
A corollary is that the trainee does not have to "invent" systems from scratch. The trainee is entering into a system where there are already rich bodies of tools, discourse and organisation that the "system" has already devised to "act" in its environment. The trainee does not have to invent these for themselves. On the other hand the trainee should not necessarily take these as givens. Further the trainees themselves are not outside these activity systems. The trainee has a major body of experience of teaching, and what it is to be a teacher. It is unlikely that a trainee would not put themselves forward for training as a teacher if they did not have fairly firm ideas about what constitutes teaching and will have well developed concepts of teachers' role, the community of teaching and societies expectation of teachers. The key issues in the design of learning environments however are that these are issues which will greatly influence outcomes.
Kaptelinin (1997) also adopts the use of Activity Theory as a methodology for human-computer systems design that also has relevance in this study. Activity theory encourages a focus on how a system transforms the work life of a computer user. He also stresses are that Activity Theory is a set of basic principles that constitute a general conceptual system, rather than a highly predictive theory. He views the basic principles of Activity Theory as including the hierarchical structure of activity, object-orientedness, internalization/externalization, tool mediation, and development.
These issues have provided Kaptelinin with a set of guidance that will be incorporated into a proposed model for design discussed below.
The kind of learning we propose to support
is highly focussed on interaction between professionals. However this is not a haphazard
interaction. There is a need for structure and the tools must provide support for that
structure. There is a belief that process of professional development comes from
interactions with fellow and more expert professionals.
Ijikuro Nonaka(1998) discusses the knowledge creation as part of the learning of professionals. Nonaka explains a Japanese concept of Ba. Ba is a "place" for the foundation of knowledge and shared meanings. It is a shared space for emerging relationships. This space can be physical, virtual or mental. Nonaka proposes 4 kinds of Ba in which knowledge is converted. The notion of conversion is based on the premise that we work on tacit and explicit knowledge, and the process of gaining new knowledge is the conversion from one to the other. He proposes a spiral process "SECI". Initially we start with tacit understanding, which is in our internal meaning ba (originating ba). Through interaction with our immediate peers this knowledge becomes externalised and hence explicit (interacting Ba). This shared; local social knowledge is then tested against the "community". Nonaka recognises the use of ICT in this process and describes this as "cyber ba". Finally this new group knowledge becomes the internalised tacit knowledge of the community and the modus vivendi, the exercising ba.
In each of these parts of the spiral ( tacit -> explicit -> explicit ->tacit), Nonaka suggest differing conversational modes which in turn relate to systems we may design. These are reflected in the following table (table 2):
|Conversation Pattern||Type of Ba|
|Tacit -> Tacit
Through joint activities and physical proximity rather than written or verbal instructions
Pure experience Zen learning
Self-transcendence -> Sympathy, Empathy
Examples: Apprenticeship, walking around
Where individuals share feelings, emotions, experiences, and mental models
Entrainment (synchronizing behavior) and improvisation
Knowledge, vision and culture
|Tacit -> Explicit
2 key supporting factors: 1) Articulation, 2) Readily understandable form
Individual transcends the inner- and outer-boundaries of the self
Examples: Group integration activities
More consciously constructed
Dialogue (metaphor, sensitivity of meaning)
|Explicit -> More
complex set of explicit
Capturing and integration new explicit knowledge,
Editing and processing
Justification -> Agreement -> Concrete Steps
Throughout the organisation
|Explicit -> Tacit
Identify one's self and the knowledge relevant for one's self
2 dimensions: 1) Action and practice, 2) Simulation experiments
Training with mentors and colleagues
Learning by continuous self-refinement though on-the-job or peripheral and active participation
In the construction of curricula for professional development the
model provides pointers for the integration of models of activity. Cycles of private
thinking, learning from peers, learning from practical experience on the job and the
continuous relationship between acting and reflecting, and builds to the goal of
reflective practitioners, and integrates that practice into new shared tacit knowledge.
The concept of reflective practitioner is closely associated with the work of Schon (1990). Schon presents a model of becoming "professional" reflective action and induction into a profession. The model is very appealing however it needs some further elaboration in socio-cultural terms. A profession (teaching or whatever) is in itself not aproblematic. Following the arguments of cultural historic activity theory or the ideas of Nonaka a profession is a dynamic system which is subject to renewal and change and therefore models of professional development need to base their processes on that dynamic redefining.
Lave 1991 argues that learning as it
normally occurs is a function of the activity, context and culture in which it occurs
(i.e. it is situated). This contrasts with most classroom learning activities which
involve knowledge which is abstract and out of context.
Social interaction is a critical component of situated learning -- learners become involved in a "community of practice" which embodies certain beliefs and behaviours to be acquired. As the beginner or newcomer moves from the periphery of this community to its centre, they become more active and engaged within the culture and hence assume the role of expert. Furthermore, situated learning is usually unintentional rather than deliberate. These ideas are what Lave & Wenger (1991) call the process of "legitimate peripheral participation."
Collins, Brown, and Newman (1991) offer the concept of cognitive apprenticeship. The idea being that traditional learning through an apprenticeship involved induction into a community of expert practice in which the "teacher" continuously engages in and is a master at the practice being learned. His or her performance constitutes the standard for the apprentice. Collins, Brown, and Newman (1991) propose incorporating key elements of traditional apprenticeship. The model ignores the usual distinctions between academic and vocational education, its objective being to initiate the novice into a community of expert practice.
These models of learning firmly base professional development and learning as process of conversation and action: a reflexive dialogue. The challenge for the learning technologist is to conceive of systems that provide support for this reflexive activity to proceed.
A teaching and learning system is more than
a system for knowledge building and creating. It is contextualised within institutions
(physical or virtual) and society, and this places practical demands on the ways a course
is structured and delivered. In particular issues of accountability are important for the
benefit of the students and the provider, in recording and validating the work on the
There is a need for the validation system that reflects the experiential learning. The notion assessment distinct from the work on the course does not fit. The process of assessment that needs supporting is one that allows the recording of the evidence of achievement, knowing and doing. It is a system for developing evidence of work done on course rather than terminal assessment or even "course work". The idea of a "course memory" (Conklin 1992, Guerrero 1997) provides the framework for such a tool.
A course memory would capture the significant facts of a course, knowledge of students, the teachers and the products of a learners learning. The course memory provides views of the memory for different activities for different actors in the system. Thus the learner has their own record of evidence of achievement whilst the course administration is able to have an overview of the system community will have an ever increasing knowledge base on which they can draw.
Further in a wish list of things that should be included in any system which would help us design a learning environment concern the issues of change. A failure of past learning technologies has been the development of monolithic highly integrated and contained systems. There was a tendency for the reductive analysis to define to fine details because often these systems would then be irrevocably committed to the printing press of the computer encoding. These are expensive processes and therefore there is a clear need to be all-inclusive. Additional print runs were often out of the question. The systems we describe call for a flexibility dynamism and change. A new system needs to have ease of evolution, so that there can be formative evaluation and change in use. In computer software terms, the system should have ease of maintenance. This quality is often a function of modularity in design.
There are from these elements key requirements that arise from the above considerations and make more specific the original user needs defined by our REM project:
We have a list of requirements therefore based on the need for activity and dialogue and recording with a variety of actors taking different roles in a system. Kaptelinin and Nardi suggest a checklist in their application of Activity Theory to human computer systems design. The checklist is a conceptual tool for identifying the most important factors influencing the use of computer technologies in a particular setting. As this has been designed from a need identified in the development of software there process is described in a technology-oriented language.
The process of starting to design from their perspective follows a sequence: Starting from observational data to indicate potential problems, formulate requests for further analysis, and provide some suggestions on how the "problem" can be solved. In the second phase an Activity Checklist is introduced. The general structure of the Checklist corresponds to the four main perspectives on the use of the technology to be evaluated:
There are reliable systems for software engineering which allow the design of complex software that can start with prose description of the complexity and proceed to a modular abstraction of the complexity. Rambaugh (1991) has described such systems as an object modelling techniques and Booch (1996) as a Universal Modelling Language (UML). Such systems describe complex software systems in terms of their actions in relation with human and other agents with whom they interact. I do not wish to suggest that such systems can effectively model the complexity and the range of nuance which human learning systems demand, however the systems do begin to provide some ideas for formalisms which can serve to describe the design and development of rich and complex learning environments. The Booch method distinguishes between the logical and physical structure of a system and describes for both the static and dynamic semantics.
Macro Development Process
The macro process is used for controlling the micro process. It discusses five activities:
Micro Development Process
The micro process basically represents the daily activities of the developer(s) and consists of four (non-sequential) major steps:
These are of course computer science terms. If we substitute language that is more descriptive of a learning environment we arrive at:
There is some mapping of the system proposed from Activity Theory
and the system based on UML. This allows us to believe that we can use aspects of the UML
modelling methodology to design a learning environment/course that takes into account the
necessary totality of a system.
UML models an object-oriented software system using several types of diagrams to represent different perspective views of the system. It is a system that identifies the processes communications and collaborations that need to take place for transformations to take place or "things to happen". These analyses in UML appear as a series of diagrams.
In designing a learning environment the fits stage is therefore to:
Our current practical example is the design of an on-line version
of a post graduate initial teacher training qualification through on-line open and distant
learning. The actors in this situation are the trainees, the teaching faculty (tutors),
the training support in the school (mentors, senior mentors), course administration
(course manager) and college administration (admin). There may be finer distinctions in
practice, but the system can cope with that by having sub classes of say mentor>>
science mentor, The objects can be resource library, course memory, registry etc.
The next stage is to create "cases" or activities that are represented from the point of view of one of the actors in the system and the interactions with other actors and other entities in the system.
A student view of a lesson might look like (fig 2):
A view of other systems (say from administration) may look like (fig 3).
Each one of theses cases will have instances that will need
further details. A methodology used within our system for describing specific lesson
encounters is based on the work of the Sandholz (1997) and the Apple Classroom of Tomorrow
experience with Units of Practice (UoP). A UoP describes elements that fit into a
"chunk of teaching and learning".
The items used to specify a unit of practice are given as:
Using the UoP methodology it is possible to specify the
"content" "methods" "sequence" of the cognitive
apprenticeship model and specify tasks within the overall framework of reflexive learning
that have been specified above. Further the inputs and outputs of a unit of work that
interact with the resource manager and course memory are well specified by the process.
These systems need to interact, and the activity of a lesson needs to be recorded by the Course memory.
In assembling the course each of these interactions need to fit onto a time line for the actors and objects, this provides a sequential view of the course. This corresponds to the sequence diagram in UML. This also provides a management tool. Such issues are also covered in UML with diagrams like Fig 4.
The model also provides for bottom up development. Tutors can plan their course using UoPs that in turn describe the actors, the inputs and outputs, the timing and the sequence, the demands on other resources and the interaction with the accountability within the management of the course. The implication of the bottom up or top down implementation is that this very modularity allows for course change (maintenance) and evolution.
This methodology is under development by
viewing how it performs in practice in the design of real teaching experiences. The author
is currently applying it to the design of a pre-service teacher education course for
trainee secondary school teachers and shortly to be applying the methodology to the
development of a Masters level course in the professional development of people in
the field of vocational education and training. This domain is an area where the community
of reflexive practice model is particularly appropriate and the need to share and remember
knowledge (Brown, A 1997).
An output of the REM project has been the development of a standards based resource management tool that serves as a resource library and in coming months we can foresee its potential application as a course memory system as well.
The next logical phase in the testing of this model of course design is a wider implementation for courses where there are a significant number of actors involved in the design process.
Bevis, E.O., Watson, J. "Towards A caring Curriculum: A new pedagogy for nursing", National League for Nursing , New York, 1990
Booch, G. (1994), Object-Oriented Analysis and Design With Applications (Addison-Wesley Object Technology Series), Cambridge Mass., Addison-Wesley
Brown, A. (1997) (Ed) Promoting Vocational Education and Training: European Perspectives, Tampereen yliopisto
Brown , J.S. and Duguid, P. (1996) 'The social life of documents', First Monday 1(1) [http://www.firstmonday.dk/index.html] (May 1996)
Collins, A. Brown, J.S. and Newman, S. (1991) 'Cognitive Apprenticeship: making thinking visible', American Educator (winter) ,6-11,38-46
Conklin J. (1992) "Capturing Organizational Memory". Groupware 92, pp. 133-137. San Mateo, CA, USA, 1992.
Department of Employment / Admiral Training Ltd (1990) "Multimedia Design A Newcomers guide", Sheffield, Department of Employment
Engeström , Y and Cole, M. (1993) "A cultural-historical approach to distributed cognition", in Soloman, G (ed) Distributed Cognition, Cambridge, Cambridge University Press.
Gagné, R.M. (1967) The Conditions of Learning and Theory of Instruction, New York, Holt Rheinhardt Wilson
Guerrero, L. A. and Fuller, D.A. (1997) , 'CLASS: A Computer Platform for the Development of Educations Collaborative Applications' , Proceedings of CRIWG'97, 3rd International Workshop on Groupware, Madrid, Spain, 1-3 Oct 1997.
Kaptelinin V,. Nardi , B.A., "Activity Theory: Basic Concepts and Applications", Workshop CHI 97, http://www.acm.org/turing/sigs/sigchi/chi97/proceedings/tutorial/bn.htm
Kolb, D.A., Rubin, I.M. and McIntyre, J.M. Organisational Psychology: an experiential approach. London, Prentice Hall
Lave, Jean and Wenger E., (1991), Situated Learning: Legitimate peripheral participation. New York: Cambridge University Press.
Merril, M.D.; Li, Z.; Jones, M. K. "Second Generation Instructional Design" Education Technology 30(2) pp 7-14
Nonaka, Ikujiro, and Noboru Konno (1998) , "The Concept of "Ba": Building a Foundation for Knowledge Creation," California Management Review, 40/3 (Spring 1998): 40-54.
Rumbaugh J. et al. (1991) Object-Oriented Modeling and Design, New York Prentice Hall, 1991.
Sandolz, J.H et al (1997) Teaching With Technology: Creating Student-Centered Classrooms, New York, teachers College Press
Schon, D. The Reflective Practitioner, New York, Basic Books ,1983
Stenhouse, L., (1975), Introduction to Curriculum Research and Development, London, Heinnemann
Townsend, I., (1976), "Producing a Tape-slide package: Initial Analysis", Visual Education, Feb 1976, pp17-19.
Vygotsky, L.S, Vygotskty, V., & John-Steiner (Editor) (1980) Mind in Society: The Development of Higher Psychological Processes Harvard University Press.
24/11/00 (article submitted July 1999)