MARKETING MATERIAL
7202 Teleteaching93 - Large Scale Conferencing
large scale conferencing for inexpert users
(david mercer - open university)
ABSTRACT
The paper describes a successful development of computer conferencing in the context of large numbers of inexpert students, working in small groups. The main operating factors emerged from the students' demand for positive benefits to be directly derived from the content, with technical ease of use being only a facilitator.
INITIAL RESULTS
The most important result was that the computer conferences, for inexpert participants were successful - based on a number of measures:
I) OVERALL PENETRATION - nearly three quarters (73%) of the students made some significant use of the computer conferencing offered.
II) SATISFACTION - nearly half (46%) of those using it found it fairly or very useful.
III) NUMBERS OF CONFERENCES - the overall number of active conferences was 99 (121, if major topics within conferences were counted separately).
IV) MESSAGE RATES - three quarters (74%) of the main conferences recorded thirty or more messages; and two fifths (38%) logged more than a hundred.
V) CONFERENCE MEMBERSHIP - the majority of the conferences were based upon tutor groups and hence most (54%) of their membership levels fell in the range 10-19 members (and 73% in the range 10-29 members).
No less important an outcome was that smaller groups, even those without any previous (shared) group identity, worked productively with each other. The great majority (31) of the core conferences were based upon small (tutor) groups, and yet they generally worked together without obvious any problems. More surprising was the fact that many of the sub-groups, usually with no more than six core members, also worked well - despite the fact that these members had no shared interests other than those of the conference
SUMMARY
The paper describes a successful development of computer conferencing in the context of large numbers of inexpert students, working in small groups. The main operating factors emerged from the students' demand for positive benefits to be directly derived from the content, with technical ease of use being only a facilitator.
BACKGROUND
To date computer conferencing has generally been seen as the province of expert users. Typically it has either been used under controlled conditions within organisations, especially those in the IT industries, or in a more limited way as bulletin boards for a wider audience, but still a technically expert one. Occasionally it has been used as a teaching tool, usually in higher education - but again for relatively expert audiences.
The Open University (OU) is one of the few institutions which has an extended track record of running large scale computer conferences in a teaching environment. Its largest such course, run by the Technology faculty, has already been taken by several thousand students, and is currently being studied by 1,200 undergraduates each year. In common with similar courses in other universities, however, these students are studying IT; so computer conferencing is taught as a technical subject and the medium really is the message.
On the other hand, students from the Open Business School (OBS) within the OU are normally practising managers with little or no special IT expertise; or any great wish to acquire this. In this rather different context computer conferencing simply becomes another means of communication. Its use needs, therefore, to be as simple and transparent as possible - any problems detract from its value, where they may actually enhance the learning experiences of IT students.
It is a major plank in OBS computing strategy, however, to introduce computer conferencing as a communication medium for its geographically dispersed students. The most important reasons for this strategy are:
1) To extend direct, 'face-to-face', contact with tutors - where OBS mainly uses distance teaching techniques in its work.
2) To encourage contact between students - in the first instance based upon self-help groups.
3) To develop the medium as a vehicle for contact with dispersed academic staff; including OU tutors and academics in other institutions.
This paper describes the results obtained from the first stage of development.
DESIGN CRITERIA
The conferencing system chosen was CoSy[1], of which the OU has had a number of years experience - across several thousand users. It was a technically stable system which offered most of the basic features needed by OBS students. It was accessible to students throughout the United Kingdom for the price of a local phone call, and its central telephone support centre already has considerable expertise in dealing with student problems. The technical problems which bedevil many new ventures were, thus, minimised - and are not explored in this paper. The remaining, 'non-technical', criteria were:
a) TECHNICAL EASE OF USE - even so, the main problem was originally seen to be the substantial technical hurdle to be overcome by students. They were only required to have a minimal level of experience in the use of personal computers, and none of using telecommunications via a modem. Despite their lack of expertise, these students had to learn the CoSy system in the shortest possible time (1-2 weeks) with little or no direct support from the OU.
b) SOCIAL EASE OF USE - students also required guidance on the practical use of the content, and on the complexities of using a new means of communication. Computer conferencing, if it was to be used as a practical channel for communication rather than as a limited technical exercise, demanded that students learned new rules of conduct - new etiquette if not new grammar.
c) MINIMAL COMPLEXITY IN USE - with hundreds of students accessing a relatively large number of computer conferences, the organisational structures were likely to be complex; requiring the development of new approaches to the presentation of the information held on the system.
d) INCENTIVES TO USE - it was recognised that, above all, there had to be very strong reasons for the students to make use of the system; clear benefits which would persuade them to participate in a form of communication which was perhaps five to ten years of its time.
PILOT WORK
Fortunately, another MBA course, 'Creative Decision Making', was being launched a year ahead of the one which was chosen as the vehicle for this development; and it too intended to offer computer conferencing to this wider audience, albeit on a more informal basis. This earlier course was, therefore, used to pilot the proposed systems in three main areas, with the following results:
a) EASE OF USE - the conferencing systems, and in particular their documentation, were successfully tested, and debugged, with the target audience. This resulted in a viable 'teach yourself CoSy' pack for students.
b) TUTOR USAGE - on the other hand, a major problem emerged in terms of the proportion of tutors providing even basic support for students - less than a third involved themselves in this aspect of the course. Where these personnel are the focus of student activities this posed a major limitation on the use of computer conferencing.
c) LIMITED RESPONSE - even more problematic was the low level of response from the students - less than a quarter ever signed onto the system and only a handful were active users.
d) FAILURE TO DEVELOP TUTORIAL GROUPS - further, it had not proved possible to develop any CoSy conferences at the tutorial group level - the level which was the key to the proposed developments - and the organisers of the pilot believed it would not be possible to run conferences in such small groups.
Thus, the preliminary indications were that, while the technical problems had been resolved, there remained a number of potentially insuperable problems - in terms of providing a useful system for all but a small minority of students.
IMPLEMENTATION
As a result of the observations emerging from the pilot, the course in this development, 'The Challenge of the External Environment', was redesigned so that its computer conferences incorporated the following key features;
a) TUTOR SUPPORT - the 34 tutors involved were initially provided with a weekend training school; which 80% attended. This represented a start-up investment of approaching £10,000. Subsequently their performance was closely monitored and supervised, and further help provided where necessary.
b) DIRECT STUDENT INCENTIVES - OU students are, along with most university students, most highly motivated to undertake work if it directly relates to the core of their studies; and especially if it relates to the assessment of that core. Use of CoSy on the course was, therefore, assessed - though a non-CoSy alternative was also available. The assessment was also, for the first time within the OU, linked to team-work; to encourage the formation of self-help groups, initially based on the use of CoSy within existing tutor groups.
c) INDIRECT INCENTIVES - national conferences had been relatively successful in the pilot, so these were again provided; but reinforced by the introduction of experts on subjects of direct relevance to course-work.
INITIAL RESULTS
The final results, including those emerging from a full survey of all the students (following the end of the course in November 1992), will be presented at the conference. The results below are, therefore provisional. They are based upon an analysis of the conferences involved, as well as of the results of a short questionnaire handed out at residential schools, which took place approximately half way through the course; to which 302 students responded (from the overall total of 460 on the course - a 66% response rate).
1) (RELATIVELY) SUCCESSFUL OUTCOME
By far the most important result, and one which had not been predicted by the pilot work, was that the computer conferences were successful - meeting and surpassing our most optimistic expectations (though it should be noted that these expectations had been lowered by the poor pilot results). This success was evident from a number of measures:
I) OVERALL PENETRATION - based upon the results from the initial survey it would seem that nearly three quarters of those on the course (73% of those answering) made some significant use of the computer conferencing offered. This was the same as the level achieved, after several years of growth, on the OU's IT course. It seems to show that wider acceptance of computer conferencing may pose less problems than was expected.
II) SATISFACTION - CoSy was not seen to be an unqualified success. Most users wanted changes to the system - including better technical usability and people usage (with consistency across courses - where the other OBS courses did not make any use of computer conferencing). Nevertheless nearly half (46%) of those using it found it fairly or very useful; which is comparable with the results on the OU IT course after several years of experience, again indicating that there may be relatively few insuperable problems even for a technically inexpert audience.
III) NUMBERS OF CONFERENCES - the number of major conferences, 34, was in line with expectations; representing conferences hosted by the 90% of tutors who did support their students, together with three national conferences. On the other hand, the overall number of active conferences was 99 (121, if major topics within conferences were counted separately). This indicated a level of (spontaneous) activity which was much greater than expected.
IV) MESSAGE RATES - three quarters (74%) of the main conferences recorded thirty or more messages; and two fifths (38%) logged more than a hundred. Only 12% showed less than ten messages. Again this indicates a high level of activity - though, as is normal in such patterns of conferencing activity, most of the messages originated from a minority of the students involved.
Messages % of conferences
0 - 9 12
10 - 29 15
30 - 99 35
100+ 38
V) CONFERENCE MEMBERSHIP - the majority of the conferences were based upon tutor groups (which typically contain 20 students), and hence most (54%) of their membership levels fell in the range 10-19 members (and 73% in the range 10-29 members).
Membership Overall Last Two Months
% conferences % conferences
0 - 4 3 32
5 - 9 16 29
10 - 14 32 22
15 - 19 22 9
20 - 29 19 3
30+ 8 5
As can be seen, in terms of sustained membership (that having been active within the past two months at the end of the course) the pattern shifted down, so that 51% then fell in the band 0-9 members. This may have been due in part to reduced activity by the core membership, especially as planned activities were much reduced as the course came to a close, but it might also reflect a reduction in the numbers of 'voyeurs' (those students dipping into conferences away from their own core/tutor group).
Reduction in Size to Starting Size of Conference
0 - 14 15+
--------------------------------------------------
0 - 49% | 39 | 25 |
|------------------------|----------------------|
50+% | 13 | 23 |
--------------------------------------------------
Looked at, as in the matrix above (which excludes the later conferences which were still highly active), in terms of the original size of membership it would seem that the larger conferences were subject to lower fall-off rates; possibly because there were a larger cores of activists to keep activities going.
b) SMALL (TUTORIAL) GROUP WORK
In practical terms, for teaching at least, a no less important an outcome was that smaller groups, even those without any previous (shared) group identity, can work productively with each other. The great majority (31) of the core conferences were based upon small (tutor) groups (typically with only 20 students - of whom normally less than 16 participated in the conferences), and yet they generally worked together without obvious any problems.
More surprising was the fact that many of the sub-groups, usually with no more than six core members, also worked well - despite the fact that these members had no shared interests other than those of the conference; though these very small groups were less stable in terms of maintaining membership over the longer term.
It would appear, therefore, that computer conferences can be used - again under the right circumstances - for joint work in small groups by previously unrelated memberships.
c) LIMITATIONS
The majority of students were, on the other hand, less than complimentary about some aspects of their experience of computer conferencing. Their complaints included:
I) TECHNICAL PROBLEMS - they felt they had been subjected to too many technical distractions (partly from the system itself, but particularly from PTT line problems) and that the hardware/software involved was clumsy and out of date. This is a problem which must await the development of better software.
II) ORGANISATIONAL PROBLEMS - they felt that the social interfaces had not been as well structured (and maintained) as they would like. This should be resolved as those involved (especially the tutors) gain experience.
III) LACK OF WIDER CONTEXT - despite these perceived problems one of the major criticisms was that computer conferencing was not available on other courses, so that the students could use their new skills in a wider context. This should be resolved within the OBS when more courses use computer conferencing as an integral part of their teaching.
IV) VALUE FOR MONEY - indeed the main problem would seem to have been that students calculated the investment they put into the course (partly in the money required to purchase a modem and partly in terms of their learning curve) and balanced this against the practical use they could make of the skills they had learned - in this case this was limited to the duration of the one course. This again needs to be addressed by making the use of such conferencing more widely applicable across other courses run by the OBS.
DISCUSSION
The main finding was that large-scale computer conferencing may be well received and well used by inexpert users - even if they have no other direct membership links and are working in small groups.
Based upon the experiences of this course, it would appear that, with further experience, the level of students demonstrating symptoms of computer conferencing alienation (or 'illiteracy') might be more generally held to below 20%. Certainly, up to three quarters of these students were able to make use of the system; and, despite the problems resulting from inexperience inevitable on the first running of such a course, almost half of these found conferencing useful.
It seems probable that the requirements for success most directly relate to the benefits to be obtained from the content, and thus are more akin to the commonly recognised requirements of the popular media rather than the technicalities of computing.
The main requirement by students would, unsurprisingly, appear to be that the conferences must directly offer them a very positive benefit, which cannot be more easily achieved by any other means. The same way that daily papers are taken because the reader finds the content directly useful, in the context of these inexpert audiences it is the content of a conference which is most important.
Technical factors are only required to facilitate the transactions, though in the current state of development they may still intrude significantly on the smooth running of the related systems.
The main 'producer' role, therefore, is to
invest in the quality of the content and to stimulate the emergence of the key
'product champions' who will maintain the desired level of activity.
hits