back

Peer Observation for Online Distance Learning Tutors: Creating the Conditions for Effective Peer Exchange

Richard Walker [richard.walker@york.ac.uk], The University of York, Heslington,
York, N. Yorks, YO105DD. United Kingdom, [http://www.york.ac.uk]

Abstract

This article explores how online distance learning tutors working within a higher education context may be supported in their professional development through participation on an institutional peer observation programme. Drawing on the reflections of participants from two cohorts (2011 and 2012) at the University of York, the article reviews the learning outcomes arising from the peer observation process and investigates the necessary conditions for fostering critical reflection on practice between tutors. Feedback from participants highlighted the importance of investing time in relationship-building at the outset of the process - a necessary first step before free-ranging and critical exchanges can flourish between partners. Pairings which invested time at this stage appeared to develop greater levels of trust, enabling them to probe their tutoring practices in a more critical way and explore areas of perceived weakness as well as tackling emergent themes in their dialogue. These observations have informed the development of a guidance framework for peer exchange at the University of York, which is summarised in the concluding part of this paper.

Keywords: peer observation; online tutoring; peer review

Introduction

Fully online course delivery is growing in importance across the UK higher education sector: over 2,800 HE courses are now offered online and the numbers are rising (Baxter, 2012). The emergence of MOOCs has captured the attention of government (QAA, 2013) and university senior managers (Gaebel et al., 2014; Nutbeam, 2013) alike, as a way of engaging new types of learners and directing them to accredited programmes, and yet questions remain around the impact of the pedagogies employed for online learning and the skill-set of tutors to support online learners effectively (Online Learning Task Force, 2011). Gallardo, Heiser and Nicholson (2011) note that the online environment offers a fresh and demanding context in which to manage interactions with learners based on more flexible and diverse engagement methods, and this requires different skills and values (Baxter, 2012; Kirkwood & Price, 2012).

The challenges of designing and supporting effective online learning have promoted calls for the ‘unlearning’ of established classroom pedagogies (McWilliam, 2005), with tutors encouraged to develop new strategies for engaging with learners and as well as their own online teaching persona (Harper & Nicolson, 2013). Bennett and Marsh (2002) see this as an imperative, given that that “the majority of tutors new to online tutoring do not have that background of online learning experience upon which to draw” (p.15). The transition to online tutoring requires staff to develop pedagogical and managerial skills, as much as technical skills in support of their online learners (Bowskill et al., 2000; Harper & Nicholson, 2013). However, conventional practice has tended to neglect these measures, with tutors commonly faced with having to develop their e-teaching skills without sufficient institutional support (Hauck & Stickler, 2006).

How then should institutions tackle this deficit in skills and values, supporting new and existing tutors in their professional development? In recent years there has been growing interest in applying peer observation techniques to online teaching and tutoring activities as a way of addressing this challenge (e.g. Bennett & Barp, 2008; Bennett & Santy, 2009; Goldsmith & Rogers-Ward, 2003; Harper & Nicolson, 2013; Swinglehurst, Russell & Greenhalgh, 2008; Tonkin & Baker, 2003). Indeed a variety of methods have been developed across the UK higher education sector to support peer review of this kind, with a primary focus on support for distance learning delivery. The ‘Peer-to-peer Reflection on Pedagogical Practice’ (PROPP) model developed at University College London is one such example, providing a framework for individual tutors to come together to discuss a problem, issue or a project from a course and use this material as a basis for reflection on course design and delivery methods (Swinglehurst et al., 2008). The Universities of Hull and Staffordshire have developed their own collaborative model of peer observation of online practice for online tutors (COOLAID), engaging tutors across both institutions (Bennett, Lee, Lynch & Howard, 2010).

Both the PROPP and COOLAID programmes mark a departure from traditional peer observation approaches for classroom teaching which have focused on quality improvements and the evaluation of teaching performance; they are designed instead to offer learning opportunities for both observer and observee in sharing practice and pedagogic approaches to online tutoring, drawing inspiration from Gosling’s ‘Peer Review’ model (Gosling, 2002) which emphasises the added value of dialogue and the sharing of perspectives for teacher development. Relating peer review methods to online tutoring, Bennett and Santy (2009) note the potential for a supported dialogue between tutors to offer a ‘window’ on each other’s e-learning practice, providing a means by which they can deepen their understanding of the online learning and teaching process and develop an awareness of what works in the online environment. It is precisely within this context of peer dialogue between colleagues that this research is situated – exploring the conditions for effective exchange to take place for distance learning tutors through an institutional peer observation programme. Through a review of participants’ feedback, the paper discusses both the challenges and opportunities for the development of tutors’ online skills and values.

York’s approach to peer observation

The University of York in the United Kingdom currently offers 17 distance learning programmes, almost exclusively for postgraduate students, which are managed locally by departmental teams. Programme teams are subject to the quality assurance and management oversight of their affiliated departments, but do send representatives to the University’s Distance Learning Forum – a body set up in 2007 to foster peer support and disseminate good practice between those involved in running these programmes. The Forum facilitates the exchange of information and discussion of issues of common interest in the running of distance learning programmes, whilst also serving as a source of advice and expertise to the University’s Teaching Committee on distance learning matters, touching on quality enhancement and policy matters.

Recognising the untapped potential to share innovative teaching and programme delivery approaches across the distance learning community whilst also supporting individual tutors in their professional development, the University of York’s Distance Learning Forum took the decision in 2011 to launch an institutional peer observation programme as a complement to the University’s Peer Support for Teaching provision. The programme was influenced in its design by insights from the COOLAID project and related approaches informed by Gosling’s Peer Review model (2002). The Forum also drew on the practical experience of three of the distance learning programme teams at York, whose own internal observation schemes pre-dated this initiative. Peer observation was presented as a development opportunity for the individual tutors involved, rather than as a quality assurance mechanism or way of monitoring performance, focusing on the ‘mutual exchange of practice’ (McMahon, Barrett & O’Neill, 2007). Tutors from all of the University’s online distance learning programmes were invited to opt in to the programme and were free to determine the terms under which they would participate – i.e. the development objectives which they would wish to pursue.

Table 1 below captures the similarities and differences between the University of York’s model and COOLAID. Whilst retaining the central purpose of supporting non-judgemental and constructive dialogue on tutoring practice, the York model differs from established peer observation frameworks (e.g. Bennett et al., 2010; Carroll & O’Loughlin, 2013) that favour the self-selection of peers. Participants who opted in to the York programme were assigned to random pairings in the role of observer and observee, bringing colleagues together from different disciplinary backgrounds with contrasting levels of tutoring experiences. This was intentional in exposing participants to different pedagogic perspectives on tutoring skills, as well as unfamiliar programme contexts and technologies (learning platforms), taking them out of their ‘comfort zone’. It was also intended to encourage them to focus on tutoring rather than disciplinary issues and in this way extend the opportunities for mutual learning within the pairings. Furthermore, there was no requirement for reciprocity in the observation process: participants were free to determine how and when they would conduct the peer observation – whether this would be a reciprocal arrangement (i.e. participants experiencing both roles as observer and observee) or one-way arrangement. The only requirement was for participants to assist with the evaluation of the programme, by engaging in a reflective interview on their experiences with the coordinator upon completion of the process.

Table 1:   York’s peer observation model

Table 1

Peer observation programme format

Figure 1 below captures the key steps in the delivery of the peer observation programme. For the inaugural programme, an online consultation phase was introduced over a one-week period prior to the launch (14th – 21st Feb 2011), which aimed to help tutors understand the aims of the programme and foster a sense of ownership over the peer observation process. A shared blog space was set up for participants to comment on the observation process and the guidance documentation that they would be using. Tutors were then blind matched by the programme coordinator and invited to make contact with their partner and proceed with the identification of objectives and definition of roles as part of their observation ‘contract’.

For both cohorts, general guidance notes were made available to participants at the beginning of the programme on the principles of peer observation, approaches to defining the focus for an observation and tips on the delivery of feedback to observees. Practical advice was also provided, such as a reminder that observees should notify their students if a ‘live’ teaching site was being observed by an external tutor, as opposed to an archived site where a notification to students would not be necessary. Participants also received guidance through the provision of a pro forma document, which was intended to help participants reflect on their goals for the activity, recording their observation arrangements (objectives and observation aims; arrangements for observation/information to provide to observer and tips on delivery of feedback). Individuals were requested to return the first section of the form to the programme coordinator during the spring term and before commencing the observation process, outlining the agreed aims of the observation. Partners were left free thereafter to manage the observation process as they saw fit, defining their own engagement methods and timelines, with a deadline of the end of the summer term set for the completion of the observation process and evaluation of learning outcomes. The lack of direction was intentional in giving participants maximum flexibility to manage the relationship and balance the observation tasks with their day-to-day tutoring responsibilities.

Images 1

Figure 1. Key steps in the delivery of the peer observation programmes (2011 & 2012)

Research methodology

The research study aimed to explore the necessary conditions for the effective conduct of observations for online distance learning tutors– specifically how tutors could be encouraged to engage in critical reflection on their tutoring practice through peer exchange and dialogue. As part of the investigation into the conditions, the study explored the impact of the blind matching process to see how the combinations of pairings worked and why, as well as the range of contextual information that should be shared prior to an observation in order to support an effective exchange between participants.

The research approach was designed around participants’ perceptions of their experiences, establishing ‘meaning’ from the standpoint of the actors (Cohen & Mannion, 1994), rather than through objective measurements of performance or levels of engagement in the peer observation programme. Evaluation of learning outcomes from the programme was consequently based on interpretive methods through semi-structured interviews with observers and observees after the completion of each observation. This research approach was selected to provide an insight into tutors’ perceptions of their shared experience, as expressed through their own language and reflections (Gillham, 2000). All participants were approached for feedback on their experiences at the end of the observation cycle, with tutors invited to discuss their experience in response to six stimulus topics: (i) expectations towards the observation process; (ii) reflections on the blind matching process; (iii) explanation of how they negotiated the peer observation contract; (iv) experience of the observation and feedback processes; (v) lessons learned from the observation process and impact on professional development and (vi) key conditions for the observation process to work effectively. The interview was conducted, where possible, as a mediated dialogue between partners, face-to-face or by telephone. Transcripts of the interviews were shared with participants for validation purposes. Participant reflections on the learning outcomes were then cross-referenced with the declared objectives for each observation, which had been documented in the pro forma completed by participants at the outset of the process, as a sense-checking measure as well as a way of reviewing the evolution of the learning exchange for each pairing.

Interview transcripts for each cohort were then combined together and a qualitative content analysis was performed of recorded comments for each of the six discussion topics as part of an inductive evaluation approach (Hsieh & Shannon, 2005). This aimed to identify key themes in the participant experience and support grounded theory development in relation to the research questions. As a final step, summary findings from the content analysis were shared in a short report with each cohort of tutors to check for validity – specifically checking that the correct inferences had been drawn from the data.

Profile of the peer observation participants

Expressions of interest were received from 12 tutors from three distance learning study programmes in 2011, of which 10 tutors went on to participate, and from 16 tutors from five programmes in 2013, of which 14 participated. Five tutors participated across both programmes, working with a different partner each time, as illustrated in Table 2 below.

Table 2:   Participating tutors in the peer observation programmes (2011 & 2012)

Table 2

Volunteers were drawn from different disciplines and study programmes with contrasting online tutoring approaches, including asynchronous group discussion and one-to-one supervision, involving a mix of tools and learning platforms (e.g. Moodle and Blackboard) to support their students. Table 3 summarises the contrasting online teaching approaches and delivery models for the participating programmes from which the tutors were drawn.

Table 3:   Contrasting teaching and delivery models of the participating distance learning programmes

Table 3

Differences were also evident in the levels of tutoring experience which participants brought to the peer observation programmes, ranging from tutors with eight – nine years’ experience to one year or less. One of the tutors had only taught two terms at York for the online programme, but had face-to-face teaching experience to draw on. A common characteristic shared by all though was their self-confidence and willingness to share practice with peers outside their programme. As one participant noted in discussing her rationale for involvement on the programme:

I am confident about my teaching style but not clear if this is working online –I wanted to find out.” (Lifelong Learning tutor – 2011 programme)

Another of the less experienced tutors commented on the desire to “improve my abilities”:

“I was not concerned about exposing myself – maybe yes if this had been in my first term’s teaching – but I was confident enough to do it”. (Public Policy & Management tutor – 2012 programme)

The fact that the programme was non-compulsory reassured some participants and helped them to focus on it as a staff development opportunity, rather than as a competency measurement:

“I felt in control of the process and quite relaxed.” (Public Policy & Management tutor – 2011 programme)

Indeed features of the peer observation process were already established as part of working practice across some of the distance learning programme teams. Public Policy and Management and MA in Teaching English to Young Learners tutors were familiar with peer observation as a staff development tool, having engaged with colleagues on their own programme in the observation of their tutoring. The opportunity to work with someone outside the programme offered fresh perspectives as well as learning opportunities on how another distance learning programme works.

“I was excited by working with an external. We have done our own peer observation internally with all following the same approach – this is an opportunity for another pair of eyes although it was slightly nerve wracking.” (Public Policy & Management tutor– 2011 programme)

Research findings

All 24 participants across both programmes completed and returned the pro forma document, capturing their objectives for the observation process, and exit interviews were also held with all participants – conducted mainly face-to-face or by phone as individual or mediated discussion sessions between the observation partners and the programme coordinator (the author of this paper). Two interviews were conducted by email correspondence for tutors posted overseas in different time zones, who were unable to join a mediated discussion. Feedback from both cohorts was then cross-referenced against pro forma objectives and then the transcripts were combined and analysed in relation to the key research themes, as discussed below.

Blind matching process

Feedback from the exit interviews revealed that the blind matching approach was positively received by participants on both programmes. Contrary to findings from previous studies (e.g. Carroll & O’Loughlin, 2013) which have highlighted the importance of participant control in peer matching as a way of building confidence, York participants appeared happy to relinquish control, expressing no strong preferences over the identity of their partner. An external perspective promised a more objective review of tutoring practices, with no preconceptions about the programme or individual’s tutoring style:

“It brought objectivity to the process. My partner didn’t know me or my tutoring style.” (Public Policy & Management tutor – 2011 programme)

“It allows a certain level of objectivity. You can stand back outside of your course without any link with the set-up and comment.” (MA in Teaching English to Young Learners tutor – 2011 programme)

Pedagogic values and levels of tutoring experience were not viewed as critical factors in determining an effective ‘match’.

“Sharing the same pedagogic approach can shut down opportunities to learn. With blind matching you can get a nugget from watching anyone.” (Lifelong Learning tutor – 2011 programme)

“It’s not too much of an issue whether your partner has a little or a lot of tutoring experience.” (Public Policy & Management tutor – 2012 programme)

Participants noted that the natural inclination for many would be to choose someone from a similar discipline, but this might restrict the scope of the learning on offer between partners - specifically the contrasting perspectives on pedagogy, terminology and teaching practice. The blind matching process acted as an encourager for participants to go beyond their comfort zone and work with tutors from different disciplines and pedagogic outlooks– in turn this ensured that the focus of the observation was on tutoring practice rather than teaching content.

“The major objection to blind matching is that you can’t comment on content and specifics -you can’t comment on the real content of courses. But the purpose is on how teaching and tutoring are done. It’s a good idea to get a mix of academic backgrounds as you can then hope to avoid discussing details (of a course). The internal peer observation discussion starts on tutoring but soon shifts to the details – the content and not tutoring style.” (Public Policy & Management tutor – 2011 programme)

“You can get bogged down in your discipline – you should be focusing on teaching practice rather than teaching content...by stripping out the content I could focus on the observation issues” (MA in Teaching English to Young Learners tutor – 2011 programme)

Participants acknowledged though that to make the blind pairing work, they needed to seek out common ground between them to establish an effective working relationship and rapport– a prerequisite for constructive dialogue (Gosling, 2002; Shortland, 2007, 2010). This led one participant to argue that it is more important to match individuals according to personality – to ensure a compatible working relationship – than to match around discipline or pedagogic outlook.

The effectiveness of the observation also depended upon clarifying the course context up front – teasing out the information that the observer requires to make sense of the online activities under review. This encompasses the ‘rules of engagement’ for the course: i.e. whether participation in online activities is a requirement for the course; what’s visible online in terms of learning and tutoring support provision and what’s happening offline – e.g. telephone contact with students. The gaps in knowledge and assumptions about tutoring styles needed to be addressed up front or pursued through clarifying questions as the observation progresses:

“..that aspect of not knowing the course – you have to ask a lot of questions about the background of the course. For one course there was a lot of email / telephone traffic between the student and tutor which was not apparent through simply observing the online discussion. There were therefore gaps in our knowledge when reviewing the online discussion. But if we repeated the process we would have that knowledge. Could do with that knowledge (context of course) upfront.” (Public Policy & Management tutor – 2011 programme)

Emergent working models for peer observation

Bennett et al. (2010) in their COOLAID approach describe peer observation as a structured process, requiring participants to tackle three phases of engagement, namely:

  1. Preparation: scene setting; establishing roles, discussing objectives for the peer observation and expectations on what will be observe and by when – recorded in a ‘contract’)
  2. Observation: review of tutoring practice and recording of notes in line with ‘contract’
  3. Discussion: delivery of feedback to observee; reflection and discussion; points for future in professional development.

Feedback from the York participants revealed a great deal of variation in how pairs tackled these phases of engagement, with three working models emerging, namely:

  • Model 1:face-to-face contract negotiation and discussion of feedback
  • Model 2: contract negotiation at a distance with interactive (oral & written) feedback and discussion
  • Model 3: contract negotiation at a distance with written (one-off) feedback

Figure 2 below captures the key features of these peer engagement models, summarising how participants prepared for and conducted the peer observation and exchange of feedback with their partner.

Model Preparation Observation & Discussion

Figure 2

Figure 2. Emergent working models for the conduct of the peer observation process

Interestingly, all pairings bar one interpreted the peer observation process as an opportunity for mutual sharing of their practice (observing and being observed), with one participant postponing her observation to a later date when she had begun tutoring. The choice of engagement model varied a great though per pairing and appeared to be determined by a range of variables including the geographical location of tutors (location and time zones) and time constraints (i.e. the personal time investment that could be devoted to the various stages of the peer observation process). For example, two of the pairings involved in the 2012 programme were working in different time zones and countries, which complicated opportunities for synchronous communication, leading to an asynchronous working relationship.

Two pairings from the 2011 programme were able to meet in person (Model 1) and used an initial meeting to define the scope of the observation and to surface a range of shared concerns, which they then worked up into an agreed set of objectives for the observation - of which there was a close overlap of personal interests between partners. A follow-up face-to-face meeting provided an opportunity to discuss observation findings and explore their significance for tutoring practice, before a summary report was exchanged. This reflected a collaborative and conversational approach to the discussion of aims and findings. In contrast, participants working at a distance (Models 2 and 3) tended to focus on their own context when identifying objectives for the observation, rather than looking for common areas for exploration with their partners first, as a precursor to completing the forms. Where meetings were arranged by Skype between partners (Model 2), they tended to focus more on introducing each other and addressing expectations on how the observation process would work, rather than exploring common ground in tutoring values. The key difference between these relationships ‘at a distance’ was in the delivery of feedback, with Model 2 pairings discussing outcomes from the observation after the exchange of reports, whereas Model 3 pairings simply exchanged feedback reports.

Learning outcomes from the peer observation process

The exit interviews revealed different perceptions from participants on the criteria for an effective peer observation experience, with perspectives shaped by the working model that individuals had agreed on with their partner. For Model 1 pairings the contract objectives and agreed focus for the observation of tutoring practice represented a starting point for a more wide-ranging discussion of pedagogic practices, addressing differences in tutoring techniques and the language and semantics used to describe their respective approaches. The rapport developed by Model 1 pairings at the outset of this process enabled them to identify common ground or ‘closer pedagogic proximity’, providing a platform from which to develop a rich learning exchange.

“There will be more learning points if there is closer pedagogic proximity between observer and observee.” (Public Policy & Management tutor – Model 1)

Model 2 and 3 pairings tended to be more transactional in their outlook, valuing a clear return on their time investment in the peer observation process. In the eyes of these participants a successful exchange was largely determined by the degree to which partners honoured the time commitments bound up in the initial contract in conducting the observation and delivering timely and constructive feedback in relation to the objectives that had been set. A successful exchange could be measured by the number of ideas that tutors could apply to their own practice; one participant assessed the value of the relationship in terms of a simple costs/benefits analysis of the ‘take-aways’ matched against the time investment in engaging with the peer observation process.

Valuable insights on tutoring practices were derived from different sources. The ‘learning by discovery’ process in reviewing another course site exposed participants to different tutoring models. Typically participants noted that they had observed a specific technique or activity in their partner’s course which they would like to develop in their own course.

“I especially liked the way that you introduced the links with some hint of what the viewer / reader would find there. I am thinking about ways to use this approach with introducing some of the optional readings for each unit in Moodle…” (Public Policy & Management tutor – Model 1)

The observation process for many was also helpful in validating their tutoring approach, offering encouragement to continue along similar lines. The observation process for one Health Economics tutor reinforced the value of the discussion rules that had been established in their own programme in establishing etiquette for how students and tutors used their discussion space. For a Public Policy and Management tutor (Model 3), the affirmative feedback that was received “reinforced and gave confidence to do what we would be going to do anyway”.

Feedback from participants also indicated that learning was not restricted to the predetermined focus and agreed aims, but was also experienced incidentally through general observation of differences in programmes delivery and ways of doing things. The process of conducting the observation with the opportunity to comment on another tutor’s course was as valuable as the feedback received from an external observer, stimulating reflection on one’s own practice:

“The asynchronous exchange forced me to think about why we do things the way we do” (Public Policy & Management tutor – Model 2)

Whilst these outcomes were commonly reported across all observation pairings, distinctive benefits were cited by participants with a well-developed personal rapport – specifically Model 1 participants and Model 2 tutors who had been blind-matched with a colleague with whom they had some prior familiarity. Engagement between these colleagues tended to be more free-flowing – a continuing dialogue rather than a self-contained and clinical observation of practice and delivery of feedback to an unfamiliar partner. Tutors could seek out clarifications on specific aspects of the courses that they were reviewing as the observation process unfolded, exploring how the context of the course influenced the tutoring approach that had been adopted. This provided a frame of reference from which more challenging observations could be made on tutoring practice:

“X noticed a lack of peer work in the module. I am quite busy in student-led discussion. I will back off more and change my behaviour. We are looking at peer review assessment across the programme team and I am going to bump this up some more.” (Public Policy & Management tutor – Model 2)

For pairings following a more transactional mode of engagement, discussion tended to be less spontaneous, with perceived barriers to what could be conveyed to a partner. Delivering critical feedback was viewed by some Model 2 and 3 participants as challenging, with observers admitting a bias towards reporting positive things in their feedback. As one participant admitted – “if I had seen bad practice, I would have probably struggled to tell them”. Other participants noted that they stuck closely to the objectives that they had been given by their partner and did not go further, or reverted to “neutral feedback”, sticking closely to the pre-set objectives for the observation, without negotiating challenging or new areas:

“It would have been easier to deliver feedback if a relationship had been established. Instead I gave neutral feedback, as you would to students, as you don’t know if you will be misunderstood…I would have felt reluctant to give negative feedback – say negative things in a direct manner. I would have hidden comments in a more positive statement.” (Model 3 Public Policy & Management tutor)

In contrast, a pair of tutors who met face-to-face during the contract negotiation and delivery of feedback phases (Model 1) reported that they were able to probe areas of weakness, drawing on the relationship and rapport that they had built up. The observer homed in on a suspected area of weakness that the observee had wanted to be reviewed as part of the observation process:

“We were able to negotiate criticism – it boils down to personality and trust. You need to be undefended in receiving criticism. It was easier to negotiate face-to-face. If we had done this online, we could have been constructively critical, but it would have been a different experience.” (Model 1 Lifelong Learning tutor)

Delivery of critical feedback was conveyed through questions or suggestions for alternative modes (alternative ways of doing things – “have you thought about?”; “on our course we did this…”), but participants noted that care was needed in arriving at critical remarks, given the risk of false assumptions based on incomplete information about the course that they were observing. Observers needed to be aware of what they couldn’t see – i.e. the levels of student-tutor interaction which were taking place outside the observable learning space by telephone or email.

Some participants used the technique of asking focused questions to explore why their partner’s course adopted a particular approach, and through this approach they were able to suggest alternative tutoring techniques:

“I noticed that the private study area was very underused and I was wondering about ways in which their attention could be focused there, to support struggling students. Would it be useful to place the definitions of terms in this area…” (Model 3 Lifelong Learning tutor)

Discussion

The evidence from the exit interviews for both peer observation cohorts highlights the importance of relationship building at the outset of the process. One participant described the building of trust as crucial “because you are letting someone in” to observe your practice. This appears to be a necessary first step to help create the conditions for a free-ranging and critical relationship to develop and one that has the potential to extend beyond the boundaries of the formal observation period. The findings show that Model 1 pairings which engaged in negotiation and discussion appeared to develop the greatest levels of trust, which enabled them to probe tutoring practice in a more critical way, exploring areas of perceived weakness as well as tackling emergent themes, moving beyond the agreed objectives. The delivery of critical feedback and attention to sensitive areas were reported as being easier to negotiate ‘in person’, where a rapport and shared understanding has been established, prompting reflection on practice and declared changes in future tutoring behaviour. This also applied to Model 2 pairings, who serendipitously had found themselves matched with colleagues with whom they had some prior familiarity, which enabled a more personal relationship to develop online. Pairings which worked at a distance without that level of familiarity tended to develop their own individual objectives and then adhere more closely to the contract, focusing on predetermined themes with objective comments, with limited evidence of critical feedback in their exchanges. Exchanges between partners working at a distance typically focused on clarification on the context of courses under observation and the rules of student engagement, rather than a broader discussion of tutoring principles. The influence of physical distance on working relationships and online behaviour is summed up neatly in the following comment:

“Being physically separate leads to individuals forming their own objectives; physical proximity may lead to shared values and objectives.” (Public Policy and Management tutor – Model 1)

The findings from this study therefore concur with previous research on peer observation (e.g. Donnelly, 2007; Gosling, 2002; Shortland, 2007) in highlighting the importance of trust in creating a conducive environment for open discussion, exploration of issues and risk-taking. This raises an important question of how, in the absence of face-to-face contact or a prior relationship to draw upon, a bond may be generated between partners to support an effective online exchange of practice. Is there a way of supporting online tutors who are working in different time zones and physical locations, so that they may approach the observation process in an undefended way, ready to open up their practice to the scrutiny of an unfamiliar partner? The limited literature that exists on peer observation for online participants is less helpful in providing answers to this question.

The feedback from Model 2 and 3 pairings in this study perhaps offers useful insights to address this challenge. The feedback suggests that the development of critical online relationships for tutors working at a distance is unlikely to occur without some form of structured relationship development being embedded within the observation process.

“Encourage partners to meet (face-to-face or Skype with video) before the process begins to get to know each other, and meet afterwards to discuss outcomes.” (Health Economics tutor – Model 2)

We may infer from this feedback that the engagement of tutors in a preliminary relationship-forming activity at the outset of the process may help partners to develop a closer bond, identifying common interests and shared objectives. Combining this preliminary step with a closure activity based on an interactive feedback process, rather than purely textual delivery of observations to a partner, may also encourage tutors to discuss emergent themes in tutoring practice and engage in more critical exchanges – sharing aspects of their practice which have not proven to be effective and using their partner as a sounding board to reflect on why that has been the case. Taking this feedback into account, a set of guidelines has been developed to give tutors who are new to peer observation a steer on the expected actions and phases of engagement with their partners. These guidelines are summarised in Table 4 below:

Table 4:   Managing peer observation online – guidelines for participant engagement

Table 4

Conclusion

In drawing conclusions from this study, we should acknowledge the limitations of the research evidence that has been generated, which is derived from a small population of self-selecting and largely experienced distance learning tutors; some tutors were indeed already sympathetic to the aims of peer observation through participation in schemes within their own programme teams. Unsurprisingly the evidence confirms the findings reported in other studies (Bennett et al., 2010; Goldsmith & Rogers-Ward, 2003; Harper & Nicolson, 2013) that online peer observation may contribute important learning benefits for online tutors and serve as a vehicle for professional development. Learning benefits were recorded by all participants across the two peer observation programmes - irrespective of the working model employed by participants. In contrast though to previous studies (e.g. Carroll & O’Loughlin, 2013), the evidence suggests that blind matching may serve as a useful stimulus to manage peer-learning, encouraging participants to seek out common ground in their pedagogic approaches and tutoring techniques, rather than concentrate on the familiar territory of teaching content and disciplinary issues.

This study also underlines the importance of relationship building (contract negotiation and the search for common ground) between participants at the outset, as well as some form of closure activity at the end of the observation process (clarifying ‘meaning’ and discussing emergent themes of tutoring practice) – whichever matching technique is used or working process subsequently adopted. The evidence suggests that relationship building is a necessary first step to help create the conditions for a free-ranging and critical engagement to develop, which may endure beyond the formal timelines of the official observation.

“Peer observation is best as part of an ongoing process – it creates the basis for future discussions. The value is not completely contained in the interaction at the time – it goes on beyond that.” (Public Policy and Management tutor – Model 1)

The guidance framework which has emerged from this study suggests a possible way of managing the peer exchange process, by embedding relationship building components into a structured step-by-step programme to support remote online participants. Future research may seek to test this framework with a wider set of online tutors to judge its suitability in creating the conditions for effective peer exchange. Any future study should also explore the value of the framework for both self-selecting and blind-matched tutors to judge whether participant control on the selection of pairings has any bearing on the relationship building process.

References

  1. Baxter, J-A. (2012). The impact of professional learning on the teaching identities of higher education lecturers. In European Journal of Open, Distance and E-Learning, 2012(2). Available online at http://www.eurodl.org/index.php?p=archives&year=2012&halfyear=2&article=527
  2. Bennett, S.; Barp, D. (2008). Peer observation – a case for doing it online. In Teaching in Higher Education, 13(5), (pp. 559-570). Available online at http://dx.doi.org/10.1080/13562510802334871
  3. Bennett, S.; Marsh, D. (2002). Are we expecting tutors to run before they can walk? In Innovations in Education and Teaching International, 39(1), (pp. 14–20).
  4. Bennett, S.; Lee, S.; Lynch, P.; Howard, L. (2010). COOLAID (Collaborative Observation in the On-Line Environment for enhancement across Institutional Divides. HEA Evidence Net. Available online at http://www-new1.heacademy.ac.uk/assets/documents/learningandtech/ University_of_Hull_Briefing_Report_26_May_2010.pdf
  5. Bennett, S.; Santy, J. (2009). A window on our teaching practice: Enhancing individual online teaching quality though online peer observation and support. A UK case study. In Nurse Education in Practice, 9(6), (pp.403-406).
  6. Bowskill, N.; Foster, J.; Lally, V.; McConnell, D. (2000). Networked professional development: Issues and strategies in current practice. In The International Journal for Academic Development, 5(2), (pp.93-106).
  7. Carroll, C.; O’Loughlin, D. (2013). Peer observation of teaching: enhancing academic engagement for new participants. In Innovations in Education and Teaching International, 51(4), (pp. 446-456). Available online at http://dx.doi.org/10.1080/14703297.2013.778067
  8. Cohen, L.; Mannion, L. (1994). Research Methods in Education (4th ed.). London: Routledge.
  9. Donnelly, R. (2007). Perceived Impact of Peer Observation of Teaching in Higher Education. In International Journal of Teaching and Learning in Higher Education, 19(2), (pp.117-129). Available online at http://arrow.dit.ie/cgi/viewcontent.cgi?article=1023&context=ltcart
  10. Gaebel, M.; Kupriyanova, V.; Morais, R; Colucci, E. (2014). E-learning in European higher education institutions: results of a mapping survey conducted in October-December 2013. European University Association Publications: Brussels. Available online at http://eua.be/Libraries/Publication/e-learning_survey.sflb.ashx
  11. Gallardo, M.; Heiser, S.; Nicolson, M. (2011). Teacher development in blended contexts. In M. Nicolson, L. Murphy & M. Southgate (eds.), Language teaching in blended contexts, (pp. 219–231). Edinburgh: Dunedin Academic Press.
  12. Gillham, B. (2000). The Research Interview. London: Continuum.
  13. Goldsmith, D.J.; Rogers-Ward, C. (2003). An Interactive Peer Examination of Online Teaching: Experienced and Novice Instructors Explore Best Practices. In Connecticut Distance Learning Consortium.
  14. Gosling, D. (2002). Models of peer observation of teaching. Learning and Teaching Support Network (LTSN) Generic Centre. Available online at http://learningandteaching.vu.edu.au/teaching_practice/improve_my_teaching/ evaluation_support_for_my_teaching/Resources/ id200_Models_of_Peer_Observation_of_Teaching.pdf
  15. Harper, F.; Nicolson, M. (2013). Online peer observation: its value in teacher professional development, support and well-being. In International Journal for Academic Development, 18(3), (pp. 264-275). Available online at http://dx.doi.org/10.1080/1360144X.2012.682159
  16. Hauck, M.; Stickler, U. (2006). What does it take to teach online? In Calico Journal, 23(3), (pp.463-475). Available online at http://www.equinoxpub.com/journals/index.php/CALICO/article/view/23153
  17. Hsieh, H-F.; Shannon, S. (2005). Three approaches to qualitative content analysis. In Qualitative Health Research, 15(9), (pp.1277-1288).
  18. Kirkwood, A.; Price, L. (2012). Missing: Evidence of a scholarly approach to teaching and learning with technology in higher education. Institute of Educational Technology, The Open University, Milton Keynes, UK. Available online at http://www.lth.se/fileadmin/lth/genombrottet/Missing_-_a_scholarly_approach-HO.pdf
  19. McMahon, T.; Barrett, T.; O’Neill, G. (2007). Using observation of teaching to improve quality: Finding your way through the muddle of competing conceptions, confusion of practice and mutually exclusive intentions. In Teaching in Higher Education, 12(4), (pp.499–511).
  20. McWilliam, E. (2005). Unlearning pedagogy. In Journal of Learning Design 1(1), (pp.1-11).
  21. Nichols, M. (2007). Institutional perspectives: The challenges of e-learning diffusion. In British Journal of educational technology, 39(4), (pp.598-609).
  22. Nutbeam, D. (2013). Keynote presentation: ‘Massively Open Online Courses: Internationalising UK Higher Education Through Technology’. Inside Government: Enhancing Further and Higher Education Through Mobile Technology, Thursday 11 July 2013, London.
  23. Online Learning Task Force (2011). Collaborate to compete. Seizing the opportunity of online learning for UK higher education. Report to HEFCE by the Online Learning Task Force. Available online at http://www.hefce.ac.uk/media/hefce1/pubs/hefce/2011/1101/11_01.pdf
  24. QAA (2013). David Willetts’ address at ‘We need to talk about Quality: MOOCs’, 8 July 2013. In QAA website. Available online at http://www.qaa.ac.uk/publications/podcasts/podcast?PubID=145
  25. Shortland, S. (2007). Participation, justice and trust within developmental peer observation of teaching: a model and research agenda. In The International Journal of Management Education, 6(1), (pp.27-37).
  26. Shortland, S. (2010). Feedback within peer observation: Continuing professional development and unexpected consequences. In Innovations in Education and Teaching International, 47, (pp. 295-304).
  27. Swinglehurst, D.; Russell, J.; Greenhalgh, T. (2008). Peer observation of teaching in the online environment: An action research approach. In Journal of Computer Assisted Learning, 24(5), (pp.383–393).
  28. Tonkin, S.; Baker, J.D. (2003). Peer Coaching for Online Instruction: An Emerging Model for Faculty Development. Paper Presented at the 19th Annual Conference on Distance Teaching and Learning, Wisconsin.

Acknowledgement

Special thanks to Ellen Roberts and Jane Lund for their constructive criticisms of this paper and for their invaluable support in launching the peer observation programme at York.

 

Tags

e-learning, distance learning, distance education, online learning, higher education, DE, blended learning, ICT, information and communication technology, internet, collaborative learning, learning management system, MOOC, interaction, LMS,

Current issue on De Gruyter Online

– electronic content hosting and distribution platform

EURODL is indexed by ERIC

– the Education Resources Information Center, the world's largest digital library of education literature

EURODL is indexed by DOAJ

– the Directory of Open Access Journals

EURODL is indexed by Cabells

– the Cabell's Directories

EURODL is indexed by EBSCO

– the EBSCO Publishing – EBSCOhost Online Research Databases

For new referees

If you would like to referee articles for EURODL, please write to the Chief Editor Ulrich Bernath, including a brief CV and your area of interest.