Methodological questions

[João PazParticipant]

Hi everyone

I am a PhD student and I am using the CoI framework in my research. I am using content analysis and the CoI survey to study the CoI development and, in particular, distributed teaching presence.
I have a few questions about the protocols of the CoI content analysis. May I leave them here?
I have already exchanged a few messages with Z. Akyol, that was most helpful, but would like to use this public space (and the presence of Randy Garrison ) to share my concerns and debate some decisions.

Best Regards
JPaz

[Madelaine BefusKey Master]

Hello Joao

I’d be interested in participating in a CoI content analysis discussion here. Post away…

Madelaine

[D. Randy GarrisonKey Master]

Joao,
First my apologies for not participating recently.
Zehra is a great source for addressing methodological issues.
I would also be happy to address any issues you or others may have.
Cheers,
Randy G

[João PazParticipant]

Thank you for your kindness and willingness to help, Madeleine and Randy.

I would like to congratulate everyone involved in the new CoI site, that has what was missing from the old one (and is capital in the CoI framework): interaction.

I tried to make a thorough review (IMHO…) of the framework and have some theoretical and methodological issues.

Coi Coding template
As I studied the CoI framework I noticed that there was some evolution, in particular regarding Social Presence. I was looking for the latest CoI coding template but in this site there is still the first version. It would be nice to have in the CoI site the latest version. The coding template present is the 2 editions of E-learning in the 21st century: a framework for research and practice reflects the diferences from 2003 to 2011.

I finally opted to use the template presented in:
Shea, P., Hayes, S., & Vickers, J. (2010). Online instructional effort measured through the lens of teaching presence in the community of inquiry framework: A re-examination of measures and approach. The International Review of Research in Open and Distance Learning, 11(3), 127–154. Retrieved from http://www.irrodl.org/index.php/irrodl/article/viewArticle/915
and
Shea, P., Hayes, S., Vickers, J., Gozza-Cohen, M., Uzuner, S., Mehta, R., Rangan, P. (2010). A re-examination of the community of inquiry framework: Social network and content analysis. The Internet and Higher Education, 13(1-2), 10–21. doi:10.1016/j.iheduc.2009.11.002
because I found it to be the most exhaustive (although with some novelties like the Assessment category to Teaching Presence).

I will start with the methodological questions, most of them about the content analysis.

1 Coding multiple categories in a Presence
Using the message as analysis unit, I am following the protocol to CODE ONLY ONE INDICATOR BY CATEGORY, referred in Garrison, R., Cleveland-Innes, M., Koole, M., & Kappelman, J. (2006). Revisiting methodological issues in transcript analysis: Negotiated coding and reliability. The Internet and Higher Education, 9(1), 1–8. doi:10.1016/j.iheduc.2005.11.001. If there are more than one indicator from the same category (for example, from Open Communication) I will select only one, the most representative.

Nevertheless, I am CODING THE ALL CATEGORIES FROM EACH PRESENCE. In the this paper R. Garrison says something opposed to this procedure

“Attempts to code the messages using the more granular indicators became progressively more difficult. The difficulty was compounded by the fact that there was often more than one possible category for each presence and more than one possible indicator for each category. In response, the primary researchers requested that the coders choose only the most salient category for each presence.” (p. 5)

As the paper refers, the coding option depends on the research objectives. But I would also like that the research would be comparable with other studies. I am studying the development of the CoI in an online course, using the course activities as time periods.

I know that the Presences have to be addressed differently. In the case of Social Presence, a message may present indicators of the three categories. But in the case of Cognitive Presence (as is clearly stated in the template I used) if we code a message as Integration we should not also code it as Exploration (for example, integrating from various sources is also sharing information but is more than that). I followed that recommendation.

Do you think the procedure is acceptable?

2 How to code a message in which there is sharing of links of resources? If it is from a student it is Cognitive Presence (Exploration) but if it is from the teacher it is Teaching Presence (Direct Instruction)?

The same question applies to a message where there is complimenting and praising contributions. If it comes from a fellow student it is Social Presence (Open Communication) but if it comes from the teacher it is Teaching Presence (Discourse Facilitation)?

As I am addressing Distributed Teaching Presence, you may understand why these are important.

3 Methodology of CoI development study

I have the follwing doubt regarding the study Akyol, Z., & Garrison, R. (2008). The Development of a Community of Inquiry over Time in an Online Course: Understanding the Progression and Integration of Social, Cognitive and Teaching Presence. Journal of Asynchronous Learning Networks, 20. Retrieved from http://eric.ed.gov/ERICWebPortal/recordDetail?accno=EJ837483

The course is divided in periods of three weeks and the number and % of messages of each category and Presence coded using content analysis.
Then, in the case of Social Presence, “(…) a 3X3 ANOVA with repeated measures was conducted to explore whether there were any changes in social presence posting patterns over time. The factors for the analysis were time (first 3 weeks, second 3 weeks and last 3 weeks) and categories of social presence (affective expression, open communication and group cohesion)” (p. 8).

A 3×3 ANOVA compares frequencies. How were they extracted (from the content analysis data)?

4 I was sensible to the argument that the analysis should be extended to the whole course and not only to forums. But there is a problem.
If in the forums we use the message as unit of analysis, how to proceed in the content analysis of student assignments or course artifacts?

Best Regards
JPaz

PS:Madeleine, thank you for the reminder of the special issue of Elearning and Digital Media about CoI (had it signaled but did not know it had already been published) and the CIDER session (just saw that it was yesterday but will see the recording)

[D. Randy GarrisonKey Master]

Joao,
There is so much to respond to here I am going to elicit some help.
Let me begin with a couple of theoretical comments. First, with regard to the first article, I am in full agreement re the inclusion of assessment as a fourth category for TP. This is in fact what we have done in our recent book that we discussed during the recent CIDER session. Indicators should be developed for this. Hopefully somebody will take this on.
Secondly, there has been no systematic updating of the coding template that I am aware of. Researchers have used a variety of approaches depending on there research topic. I generally do not have a problem with this. However, if it violates the basic premise of the framework, obviously there is a problem. With regard to the second article, this is where I have some reservations. While I think it raises some interesting issues, the problem is that it implicitly creates new presences (Teacher SP and Student SP) that violates the basic premise and assumptions of a community of inquiry. That is, TP is distributed among all the participants. Each of the participants take responsibility for teaching, social and cognition to the best of their abilities. So the question I have is with regard to the implications for the framework as a whole. This may apply to your second coding issue.
With regard to the first coding issue, I am going to defer to others such as Zehra. I have not done any coding for some time and would need to get up to speed. However, I will say that the key to me is to be clear and consistent whichever way you approach the coding. This applies to your second coding issue. I would code “complimenting and praising contributions” as SP regardless of who stated it. Again, whichever way you decide to go, just be consistent. With regard to the 3rd coding issue I will let Zehra respond. Finally, I would need to understand the 4th issue better to be helpful. My first question would be why not use the same template?
My apologies for deflecting some of your questions. Hopefully others can help in clarifying these issues.
Cheers,
R

[João PazParticipant]

Thank very much for your answer, Randy. I should have posted one question at a time.

First of all, a lapse correction. In the 3rd issue, when I wrote that ANOVA compares frequencies I meant “means”, obviously.

Regarding the 4th issue
To make my question clearer: when I coded (with the same template, yes) student assignments or end of course self-assessment documents, for instance, I took each one as the unit of analysis, as I took each participant forum message as unit of analysis in the forums.
But the size and complexity of these may be very different from a forum message. So I was wondering if there was another way of doing it (or if it had been done differently). I think not but am open to be corrected.

I agree with you that it is crucial to make the coding protocol clear and consistent. I will present the difficulties, the coding options taken and why they were taken, in my final work.

Best Regards
JPaz

[Zehra AkyolKey Master]

Hi Joao,

Regarding your third question about the methodology, I’m not so clear about what you mean extracting SP from the content asnalysis data???

In that particular study, the unit of analysis was each posting and each coded for CP, SP, TP. So, there was no issue of separating one category from others. Is this what you are asking?

Zehra

[João PazParticipant]

Hi Zehra, nice to “talk” to you again.

Not quite.

My third question is specif to the procedure you used in the paper referred.

The question apllies to all Presences, I have just referred SP because the citation was about it.

With the content analysis you get the Presences’ (and categories) frequencies. You can turn those frequencies in percentages of the total to compare the evolution of the Presences in the three time periods.

But when you aplly an ANOVA you compare means to find if there were significant difference between the values of the three time periods. My question is: how did you get those means?

For example, in your other paper (Akyol, Z., Garrison, R., & Ozden, M. Y. (2009). Online and blended communities of inquiry: Exploring the developmental and perceptional differences. The International Review of Research in Open and Distance Learning, 10(6), 65–83. Retrieved from http://www.irrodl.org/index.php/irrodl/article/viewArticle/765) it is clear. You use the CoI survey (from which you can get the means) and a T- Test and a Mann-whitney U to compare the results of two courses.

Hope it is clear now.

Best Regards
JPaz

[Author]

Posts