Notes: Niu, H. & van Aalst, J.: Participation in Knowledge-Building Discourse: An Analysis of Online Discussions in Mainstream and Honours Social Studies Courses

Niu, H. & van Aalst, J. (2009). Participation in Knowledge-Building Discourse: An Analysis of Online Discussions in Mainstream and Honours Social Studies. Canadian Journal of Learning and Technology. 35(1). pp. 1-23

Abstract: Questions about the suitability of cognitively-oriented instructional approaches for students of different academic levels are frequently raised by teachers and researchers. This study examined student participation in knowledge-building discourse in two implementations of a short inquiry unit focusing on environmental problems. Participants in each implementation consisted of students taking a mainstream or an honours version of a tenth grade social studies course. We retrieved data about students’ actions in Knowledge Forum® (e.g., the number of notes created and the percentage of notes with links), and conducted a content analysis of the discourse by each collaborative group. We suggest the findings provide cause for optimism about the use of knowledge-building discourse across academic levels: there was moderate to strong evidence of knowledge building in both classes by Implementation 2. We end with suggestions for focusing online work more directly on knowledge building.

Intro

Knowledge building shares certain features with these approaches, including emphasis on collaboration, metacognition, distributed expertise, and use of computer-supported inquiry. As elaborated below in the section entitled “knowledge building,” its distinctiveness follows from the commitment to make processes of expertise and innovation prominent in school. In a class operating as a knowledge- building community, students are agents of their own learning, work toward goals of collective knowledge advances, and treat ideas as real things that can be improved by means of discourse (Bereiter, Scardamalia, Cassells, & Hewitt, 1997)¹ . Advocates for knowledge building assert that it fosters a host of 21st century skills.

but, what about:

“the belief that instruction of higher order thinking is an appropriate goal mainly for high-achieving students and that low- achieving students, who have trouble with mastering even basic facts, are unable to deal with tasks that require thinking skills” (Zohar & Dori (2003) p. 146)².

study design:

The goal of this study was to examine participation in asynchronous online discourse as an aspect of knowledge building, with a view to understanding its scalability across courses differing in academic level. To this end, we analyzed server-log data and the content of students’ contributions to an online knowledge-building environment (Knowledge Forum®, see http:www.knowledgeforum.com) from two implementations of a short inquiry unit in which students investigated environmental problems.

…

The study examined two kinds of questions in the context of two successive implementations of knowledge building: How do participation levels in the mainstream classes compare with those in the honours classes, and to what extent can we conclude students engage in knowledge-building discourse?

knowledge building:

Though knowledge building involves many types of interactions, discourse in Knowledge Forum plays a fundamental role; it provides a reliable and permanent record of experiments, classroom activities, ideas and questions that can be used to review progress and to develop understanding at progressively more complex levels.

According to van Aalst, students need to think of their work in the Knowledge Forum database as building a communal learning resource that has lasting utility rather than as online conversations. By contrast, teachers most often use asynchronous environments to promote the sharing, discussion, or debate of ideas.

analyzing participation in knowledge-building discourse:

We contrast two perspectives on participation in asynchronous online discourse: one focusing on individual students’ actions in the online environment and one focusing on the identification of evidence for emergent and collective phenomena within the discourse of a community.

keeping in mind:

Analysis of individual actions gives an incomplete picture. As many authors have pointed out, the actions in a discourse are mutually dependent (Sawyer, 2006³ ; Stahl, 2002⁴; Wells, 1999⁵ ). For example, when students are asked to write notes to summarize what they have learned from their discourse, some comment that others have already stated their most salient learning and that they therefore do not state it again.

variables that influence participation:

• prior domain knowledge
• motivation
• goal orientation
• writing apprehension
• epistemological beliefs
• ability to analyze arguments

but infeasible to measure all of these… this study focused on writing apprehension and ability to self-assess contributions (reflect on discourse)

Writing apprehension reflects a student’s attitude and emotion towards writing tasks and written communication. … What they write is available for everyone to see and critique, which for some could create apprehension, causing them to write little and avoid spontaneity and sophisticated language (Faigley, Daly, & Witte, 1981)⁶.

Ability to reflect on discourse is also important to knowledge building, especially for evaluating the progress of a line of inquiry and for setting communal learning goals.

for the reflection on discourse part, students produced portfolios of their contributions, made of notes in the discussion board. Students summarized evidence of the 4 phases of knowledge building (working at the cutting edge, progressive problem solving, collaborative effort, and identifying high points) with hyperlinks to relevant notes as evidence.

the study:

examined participation in online discussions in the context of two successive implementations of a three-week inquiry unit

research questions:

1. To what extent do students in mainstream and honours social studies courses participate in online discussions?
2. To what extent can the online discussions in both academic levels be characterized as knowledge-building discourse?

methodology:

first iteration analyzed post-hoc, latent analysis, studying only the discussion board posts after the implementation was over. second iteration involved the teacher, and gathered additional data (a questionnaire, and a writing apprehension test).

Procedures:

Teacher set up new discussion boards for each study iteration. Use of the discussion board was demonstrated by teacher. Students had one full class (70 minutes) per week in the computer lab to participate, and could participate from home as well.

The teacher designed the collaborative work to proceed in several phases: (a) showing the area of concern on a world map; (b) identifying the problem with historical and current information; (c) identifying causes, consequences, and solutions to the problem; (d) and explaining difficulties one might face in implementing a proposed solution. He had used this design for several years, and now created a view in Knowledge Forum for each phase. After setting up the database this way, the teacher did not systematically analyze the discussions or comment on them in Knowledge Forum. However, he regularly read notes when students had them open during class and asked students if they were making progress or needed assistance.

Participation in Knowledge Forum was not included in the formal assessment scheme for the unit. Instead, students were required to individually create portfolios using several of their own notes as artifacts; these were assigned at the end of the unit. Students were asked to identify two to three of their own notes and explain why they considered these notes as exemplary knowledge-building contributions. Thus, we may assume that students’ productivity in Knowledge Forum was not influenced by the need to meet a quota for note creation and reading. When students began preparing their portfolios, the teacher related the topics of investigation to the prescribed learning outcomes provided by the Ministry of Education to provide synthesis across the work by different groups.

limitations:

Perhaps the most significant limitation was that the course commenced only a few weeks before the inquiry unit, thus there was little time for students to develop as a community and to acquire values and practices conducive to knowledge building; the decision to assign the students to groups (thought necessary by the researchers in a large class) also limited community development. In addition, three weeks seemed short for observing emergent knowledge-building phenomena such as progressive problem solving and the articulation of general principles from the solutions proposed by the various groups. Thus, this study examines knowledge building in collaborative groups and during relatively short periods of time.

Measures & Analysis:

server logs were crunched, looking at:

• notes created (productivity)
• percentage of notes read (productivity)
• percentage of notes with links (responses to other notes)
• note revision (ideas as improvable objects) (really? is that what this demonstrates?)
• scaffold use (metacognitive prompts)

Content Analysis of Knowledge-Building Discourse:

all writing was analyzed as with the collaborative group - group discourse - as the unit if analysis.

5 principles used in analysis (others were dropped because of low inter-rater reliability):

1. working at the cutting edge - community value to advance the state of knowledge. new stuff.

• (Collective responsibility, community knowledge; epistemic agency; real ideas, authentic problems)

2. progressive problem solving - reinvesting efforts in response to new ideas, building on previous work

• (Improvable ideas; rise above)

3. collaborative activity - students making an effort to help each other understand ideas. includes service to the community.

• *(Collective responsibility, community knowledge; idea diversity; democratizing knowledge)

4. identifying high points - personal insight, metacognitive insight, insight into knowledge advancement processes (self and community)

• (Epistemic agency; rise above)

5. Constructive uses of authoritative sources - keeping in touch with the present state and growing edge of knowledge in the field

• (identifying inconsistencies and gaps in knowledge sources and using resources effectively for extending communal understanding)

Content Analysis

Though the mean scores were not high, they do suggest moderate evidence for participation by the groups in the mainstream class for three of the principles: working at the cutting edge, progressive problem solving, and collaborative effort. It is worth noting that while the scores were higher for the honours class, the data did not suggest large differences between the two classes compared with the large between-class differences for the server-log data. (Due to the small number of groups no statistical tests are done for the content analysis. The findings must therefore be interpreted with caution.)

Conclusions

It is important to understand why there was not a strong relationship between the server-log indices and the results of the content analysis of knowledge-building discourse in this study.

DN Notes:

• study had small sample size - but probably difficult to get a larger n. so statistical tests are pretty much useless.
• content analysis has problems with inter-rater reliability.
• transaction analysis may be more useful than straight (latent) content analysis - what are they doing, rather than an aggregate of what are they saying?