Category Archives: Assessment

Knowledge, Mind, and Assessment

What is Knowledge?
Knowledge is socially negotiated, situated, and distributed.  It is what Sir Karl Popper would term a “World 3” artifact, it’s own entity, taking on a form of its own (what Scardamalia and Bereiter would call a “conceptual artifact”) within a knowledge community.

Knowledge is situated in the sense that concepts are situated within the context of the surrounding environment, and these concepts are progressively developed.  It is a product of the activity, context and culture in which it is learned and used.  Conceptual knowledge as a tool is only fully understood through its use.  Furthermore, its use changes the user’s world view, causing the user to adopt the belief system of the culture in which the conceptual knowledge tool is used (Brown, Collins, and Duguid; 1989).  Brown, et al. (1989) propose a “cognitive apprenticeship” model of education, which promotes learning through activity, tool, and culture.  Cognitive apprenticeship is supported by collaborative social interaction and social construction of knowledge.  It honours the situated nature of knowledge, embeds learning in activity, and makes deliberate use of the social and physical context.

Bereiter (1997) argues for the value of overcoming situatedness of cognitionby creating World 3 knowledge objects that are transferrable across different situations.  He points out that the problem with situatedness of cognition is that as one learns more about a phenomenon, one knows increasingly more about the phenomenon as framed within that situation, making this knowledge less likely to be generalizable to other situational contexts.  Furthermore, treating all knowledge as situated would make knowledge objects invisible.  Bereiter (1997) calls us to treat ideas as “objects of inquiry” (p. 18), requiring “Disciplined movement back and forth between (Popper’s) World 1 (the physical world) and World 3 (immaterial world of knowledge and abstract objects)” (p. 18), yielding the “hypothetico-deductive method” and endless possibility for theory development, problem solving, and design.  Incidentally, Sir Karl Popper’s philosophy of knowledge is based on his proposition of 3 worlds:

  1. World 1 – the physical world
  2. World 2 – the world of the mind (mental states, ideas, perceptions)
  3. World 3 – the immaterial world that is the product of the human mind (e.g. the concepts and ideas represented or expressed by books, papers, paintings, symphonies; but not physical books, papers, or paintings themselves)
What is Mind?  Where Does Knowledge Reside?
Sebastian Seung of MIT discusses his theory of mind:  “I Am My Connectome” at a TED Talk (July, 2010):

Nueroscientists since the 19th century have hypothesized that one’s memories, personality, and/or intellect also reside in the connections between one’s neurons.  This would imply that knowledge resides in our neural connections.  These connections form what Seung calls one’s “connectome“.  According to Seung, neural activity encodes our thoughts, feelings, and mental perceptions; but neural activity can also cause our own neural connections to change (hence changing our connectome).  Conversely, our experiences can change our connectome.  Thinking can change one’s connectome.  Neural activity constantly changes in our brain, but it is the brain’s neural connections (which make up the brain’s neural network) that determine the pathways along which this neural activity flows.  These neural pathways as a whole is the connectome.  The Seung Lab at MIT seeks to read memories from connectomes, and Seung is leading the new field of connectomics to test these hypotheses.

As I consider both Bereiter and Seung’s theories of mind, I see the one commonality is that they both see knowledge as residing in connections and composed of networks of connections.  In Seung’s case, these connections are of a physiological nature residing in the human brain.  In Bereiter’s case, these connections are conceptual in nature, residing in World 2 and World 3; with collaborative knowledge building as the vehicle by which individual learner’s conceptual connections combine and build upon those of other learners in the creation of a World 3 knowledge object.

The Nature of Knowledge:  Implications on Assessment
Bereiter (1997) points out that situativity theorists have not advanced any educational ideas due in part to their failure to define the purpose of education and to their confusion between process and product.  The collaborative knowledge building that I have witnessed in my experience as a graduate student, classroom teacher, and instructional leader aligns with Bereiter’s assertion that “knowledge implicit in the process” (p. 23) should be distinguished from “knowledge that is the product of the process” (p. 23); and yet, this distinction is not recognized in K-12 classroom practice, curriculum policy, or assessment and evaluation policy.  Our education system is still misguidedly preoccupied with learning goals and the embodiment of knowledge in the form of reports and presentations, failing to see knowledge taking a form that can be worked with or even packaged and sold (Bereiter, 1997).  This is decidedly antiquated thinking, considering our knowledge society’s wealth will come from knowledge work – no longer from manufacturing as in the industrial society of old.  “What distinguishes knowledge work is not using knowledge by creating or adding value to it” (Bereiter, 1997: p. 23).  This is not to say that we should do away with learning goals, merely that the focus should shift from these to learning for deeper understanding and the construction of knowledge through collaborative means supported by technology.

Hutchins and his colleagues developed the distributed cognition approach in the 1980s as a way of understanding collaborative work practices, or “cognitive systems” – complex interactions between multiple people, artifacts to perform an activity, and technological systems (Rogers, 2006).  There are two levels of analysis:  (1) analysis of ‘the propagation of representational state across media’ and (2) analysis of human interactions (i.e. the problems, the breakdowns, distributed problem-solving, role of non/verbal behaviour, coordinating mechanisms, communication during collaboration, knowledge sharing/accessibility).

While I can see the value of having a distributed cognition framework with which to study the dynamics of collaborative work, I find that this approach emphasizes the interactions enacted to complete the task, and does little to examine closely how ideas/memes/theories evolve within and among a community.  Furthermore, the distributed cognition approach does not seem recognize knowledge as its own entity, as a ‘product’.  Rather, it acknowledges knowledge as a process but focuses on the knowledge community’s development of that process, not on the conceptual evolutionary process of knowledge in its own right.

This is especially problematic when one considers Carl Bereiter’s (2002) contentions that that the mind is not a container, knowledge and conceptual artifacts do not reside in learners’ minds,  and that the mind does not reside in the head of a person, nor does it necessarily reside within one person.  For Bereiter, knowledge objects and conceptual artifacts belong to World 3.

If distributed cognition analysis of a knowledge community can be used in such a way as to reveal how the community’s mind/cognition is distributed among the community, and hence, trace how the conceptual development of knowledge as a conceptual artifact is gradually constructed within the community; then distributed cognition could be a powerful approach to prove Bereiter’s theory of mind, and aid in the classroom assessment of collaborative knowledge building in asynchronous online environments.

In this regard, Activity Theory (Hew & Cheung, 2003), developed by Alexei N. Leont’ev and Sergei Rubinshtien, seems more promising.  Based on the premise that tools mediate between subject and object processes, rules mediate between subject and community processes, and roles mediate between community and object processes; one can use this framework to analyze several combinations of any three processes as illustrated by any three edges of the following triangles:

If the basis of our knowledge society’s economy is going to be knowledge work, then it stands to reason that schools must teach learners to be proficient knowledge workers.  Assessment is vital to this learning process, for teachers and learners alike.  Applying Gunawardena et al’s (1997) 5 phases of active knowledge construction and Henri’s (1992) 5 types of critical thinking (Hew & Cheung, 2003) to assess individual and community knowledge construction would be very useful in this regard.  In addition to being good knowledge workers, today’s learners will need to become excellent collaborators as well.  With this in mind, it would be helpful to apply Rourke et al’s (1999) 3 constructs of social presence and affective responses (Hew & Cheung, 2003) to assess social presence, and to apply Kirkley et al’s (1998) 7 means of moderator assistance (Hew & Cheung, 2003) to assess facilitation skills – or what I like to call “cognitive leadership” skills.  Of course, I do not mean to suggest that we implement the above mentioned frameworks in their pure form, but rather to reform our current assessment and evaluation policies and practices based upon a combination of these frameworks.

References:
Bereiter, C. (2002). Education and Mind in the Knowledge Age. Mahwah, NJ: Lawrence Erlbaum Associates. Online: https://portal.utoronto.ca/bbcswebdav/xid-1113981_1

Bereiter, C. (1997). Situated cognition and how to overcome it. In D. Kirshner & J. A. Whitson (Eds.), Situated cognition: Social, semiotic, and psychological perspectives (pp. 281-300). Hillsdale, NJ: Erlbaum.  Available: http://www.ikit.org/fulltext/1997situated.pdf

Brown, A. L., & Campione, J. C. (1996). Communities of Learning and Thinking, or a Context By Any Other Name. Contemporary Issues in Teaching and Learning, 120-126.

Brown, J. S., Collins, A. & Duguid, P. Situated Cognition and the Culture of Learning (1989). Educational Researcher, Vol 18(1), 32-42  Available at:
https://portal.utoronto.ca/bbcswebdav/users/brettcla/Course%20readings/BrownDuguid.pdf

Cole, M. & Wertsch, J. V. (1996).  Beyond the Individual-Social Antimony in Discussions of Piaget and Vygotsky.  Online: http://www.massey.ac.nz/~alock/virtual/colevyg.htm

Hew, K. F. & Cheung, W. S. (2003). Models to evaluate online learning communities of asynchronous discussion forums. Australian Journal of Educational Technology 2003, 19(2), 241-259. Online: http://www.ascilite.org.au/ajet/ajet19/hew.html

Hung, D. & Chen, D. T. (2001). Situated Cognition, Vygotskian Thought, and Learning from the Communities of Practice Perspective: Implications for the design of Web-based E-Learning.  Educational Media Internaitonal, 38(1), 3-12. Online: https://portal.utoronto.ca/bbcswebdav/users/brettcla/Course%20readings/SitCogWebDesign.pdf

Lave, J., & Wenger, E. (1990). Chapter 1 in Situated Learning: Legitimate Peripheral Participation. Cambridge, UK: Cambridge University Press. Online: https://portal.utoronto.ca/bbcswebdav/users/brettcla/Course%20readings/LaveLPP.pdf

Vygotsky, L. (1934) 2. Piaget’s Theory Child Language and Thought.  Online:  http://www.marxists.org/archive/vygotsky/works/words/ch02.htm

Rogers, Y. (2006) Distributed Cognition and Communication. In The Encyclopedia of Language and Linguistics 2nd Edition. Edited by Keith Brown Elsevier: Oxford. 181-202. Online: http://research.oise.utoronto.ca/~jhewitt/okc/UploadedFiles/dbii97/4/RogersDistrCog.pdf

Russell, R. (?). Experience based learning theories.  The Informal Learning Review. Online: http://www.informallearning.com/archive/1999-0304-a.htm

Scardamalia, M. (2002). Collective Cognitive Responsibility for the Advancement of Knowledge. Liberal Education in a Knowledge Society, 67-98.

Scardamalia, M. (2000). Social and Technological Innovations for a Knowledge Society. In S. S-C. Young, J. Greer, H. Maurer, & Y.S. Chee (Eds.). Proceedings of the ICCE/ICCAI 2000: Volume 1. Learning Societies in the New Millennium: Creativity, Caring & Commitments. (pp. 22-27). National Tsing Hua University, Taiwan: Taipei. Online: https://portal.utoronto.ca/bbcswebdav/xid-1113975_1

Scardamalia, M., & Bereiter, C. (1996). Student Communities for the Advancement of Knowledge. Communications of the ACM, 39(4), 36-37.

Smith, M.K. (2002). ‘Jerome S. Bruner and the process of education’, the Encyclopedia of Informal Education. Online: http://www.infed.org/thinkers/bruner.htm

Preparing Students for the 21st Century Knowledge Age

If this is what life is going to be like in 2019, what skills do today’s students need to be successful in the Knowledge Society?  How can we help them develop these skills?

Readings
Lately, I’ve been reading about Knowledge Building and Online Communities:
One can look up all the various 21st Century Skills projects to see a list of skills, which commonly include creativity, digital literacy, critical thinking, and problem solving:
Teacher A, B, and C (Scardamalia & Bereiter)
Dr. Marlene Scardamalia and Dr. Carl Bereiter, founders of the Institute for Knowledge Innovation and Technology at OISE / University of Toronto, maintain that schools need to adopt a culture of collaborative knowledge building if they are to prepare our society for the Knowledge Age.  Whereas traditional schools adhered to what they call a “Teacher A model” – teaching reduced to tasks and activities; I find that much of our current schooling adheres to what they have termed the “Teacher B model” – a focus on learning objectives/outcomes/expectations with students’ responsibility directed at tasks and activities, and the teacher taking on the cognitive responsibility.  The education utopia Scardamalia and Bereiter espouse is the “Teacher C model“.  In this model, strategic cognitive activity is turned over to the students.  Whereas Teachers A and B roles resembled that of “Engineer”, Teacher C does what Teacher B does – but with the added “objective of turning more of the learning process over to the students” (Bereiter, 2002; Ch. 8, p. 39).
Collective Cognitive Responsibility (Scardamalia, 2002)
Scardamalia defines Collective Cognitive Responsibility as the condition when learners “take responsibility for knowing what needs to be known and for insuring that others know what needs to be known” (2002, p. 2).  She and Bereiter assert that schools actually withhold cognitive responsibility when they operate on a Teacher A and/or Teacher B model.  They espouse that engaging a class in collaborative knowledge building, particularly in an online discussion forum such as Knowledge Forum, effectively enculturates students into the collective cognitive responsibility of their community knowledge.  Scardamalia goes on to elaborate on her 12 Knowledge Building principles (2002, p. 9-12), the basic building blocks of Knowledge Building (KB) pedagogy:
  1. Real ideas, authentic problems
  2. Improvable ideas
  3. Idea diversity
  4. Rise above
  5. Epistemic agency
  6. Community knowledge, collective responsibility
  7. Democratizing knowledge
  8. Symmetric knowledge advancement
  9. Pervasive knowledge building
  10. Constructive uses of authoritative sources
  11. Knowledge building discourse
  12. Embedded and transformative assessment
The following have been my KB-related learning goals since I first learned about KB 10 years ago at OISE (Master’s work):
  • how to implement KB in a publicly-funded classroom
  • possible modifications to the pure form of KB for realistic classroom implementation
  • how to recognize true KB
  • how to foster true KB in the classroom, given the realities of the publicly-funded classroom
My classmate, Kyungmee, writes:  “Moreover, according to Scardamalia (2002)’s paper, knowledge creation processes are basically CHANCY. ‘… knowledge creation depends on chancy processes of discovery and invention.’  Then, how to define knowledge creation processes? Is there any particular steps or conditions? If not, how teacher can try to teach or provide students knowledge building opportunities?”
I have been wrestling with this very question since I first learned of constructivist knowledge building at OISE 10 years ago!  I find that the concept of KB is far too open and abstract to be implemented in its pure form – especially if KB is to be implemented in public schools which are mandated to follow provincial curriculum expectations.
I did a pilot project in a few schools at my school board where I tried to add some structure to KB – a compromise between KB and the realities of time and curricular constraints that public school teachers face.  Teachers chose a curricular area to address, in which their students would begin the unit using the inquiry process infused with face-to-face (f2f) KB cross talks and virtual KB Knowledge Forum discussion.  We used Wiggins & McTighe’s Understanding By Design framework to co-design and backward map a curricular unit.  To satisfy current provincial assessment and evaluation policy which states that summative evaluation can only be of a product, students were then required to create a final flashy ICT-infused project to demonstrate their learning.
I’m sure Bereiter and Scardamalia would cringe at the way I chose to go about implementing KB in these pilot classrooms, but I had to start with where the teachers were at, while considering the realities of policy and classroom constraints that these teachers face.
Systemic Changes Needed for Knowledge Building in Ontario
I see 2 systemic changes necessary for authentic KB implementation in the classroom and Teacher C-ism:
  1. Provincial Assessment and Evaluation (A & E) policies must recognize that Knowledge Building (KB) and collaboration skills can and should be measured, evaluated, and reported upon.  As per Bereiter, A & E policy should recognize that KB is productive work, IS learning, and is a necessary skill for success in the knowledge society.
  2. To avoid Teacher B-ism (a focus on learning goals), is not, in my opinion, a realistic endeavour given our provincially mandated curriculum.  To get rid of a mandated curriculum is unrealistic as well, because I do think that a society should have a certain baseline of common knowledge in order to function cohesively.  However, might there be some sort of a compromise, where provincially mandated learning expectations exist, but does not drive education?  How might a “Teacher B.C” model work?

While it is nice to dream up how we might make systemic and policy changes, I am more interested in how we can make KB work within our existing reality.  Anyone have any other suggestions for implementing KB in classrooms?

Assessment of Knowledge Building
Page 31 of the Ministry of Education’s 2010 Growing Success document on assessment and evaluation (A & E) has a nice summary table of assessment for/of/as learning.
I would agree that assessment as learning (a.k.a. formative assessment) is key in helping students develop their collaborative knowledge building skills, as well as developing their metacognition of this.  It is unfortunate that under current A & E policy, formative assessment is not something that can be quantified and reported upon formally on a student report card.  Since Scardamalia and Bereiter emphasize the importance of these skills for success in the knowledge society, I think it is problematic that this is not addressed in the report card.
Page 11 of the Growing Success document defines “learning skills work habits”, and are part of the student report card.  I think that the Ontario report card should also have a “Collaborative Knowledge Construction” section, much like the Learning Skills section, in which various component skills related to collaborative knowledge building are listed, and teachers may assign qualitative “grades” to each skill.
My classmate, Vincent A. asks:  “How can educators effectively and authentically assess knowledge building, especially if there is no final tangible product?”
I’m quite sure that Scardamalia and Bereiter would disagree that “there is no final tangible product” in knowledge building.  As a member of a knowledge building community, once you put forth an idea, that idea becomes an artifact – its own entity, and belongs to the community collective (not to the person who contributed the idea).  In other words, the community collective is made up of tangible ideas.  The final tangible product would be the community’s Rise Above.
Knowledge Building in Wikis?
I like the idea of a class/group wiki.  Though I’m not sure about 2 things:
  1. Is there community knowledge building occurring, and if so, how might we track it?  I ask this because a wiki is usually a piece of work that demonstrates a group/community’s understandings.  It doesn’t actually show the discussion around it (unless the students use the discussion forum within the wiki), and hence, we can’t see the conjectures, the theorizing, etc.  We just see the factual information that the group has come up with via various resources.
  2. How do we assess wiki work?  Provincial A & E policy states that we can only evaluate students on an individual basis, not on a group-by-group basis.  So how do we evaluate individual students’ wiki work?  Going back and analyzing various iterations of wiki pages for each student is tedious and unrealistic.  Is there a better way?  How do you evaluate this?
Another Way for Knowledge Building Communities to Use Wikis
Prof. James Slotta of OISE (my PhD supervisor) has created a framework called Knowledge Community and Inquiry (KCI), in which groups of students work within a science inquiry model and contribute to a community wiki, then are challenged to solve an authentic problem in which they use the wiki as a resource to do so.