I agree in principle that we who work in education should be able to describe what we intend to do, and that it is important that we find a way to demonstrate to what extent we have met those goals.

But that principle is a principle of almost unimaginable complexity.

Rather than proliferate crude measures of recall or reductive “normed” evaluations of various templated essays, we should think much more deeply and comprehensively about assessment. To do this, we’ll have to start with what it means not only to learn something in the sense of committing it to memory, vital as that is, but also to understand it, to be able to sense and articulate and share the structure of that knowledge as well as the conjectures and dilemmas that surround it and propel it into new areas of inquiry. We need to think about domain transfer, and ask what kind of learning fosters the analogical and metaphorical thinking that leads to conceptual breakthroughs. We need to think about the teacher’s theory of other minds, as well as the students’. We need to master strategies of indirection that empower each other to imagine and perform what Douglas Hofstadter calls “perceptual regrouping,” that trick of the mind that can perform figure-ground reversals, separate sequences into smaller groups to yield new possibilities, and adapt Polya-esque heuristics to apparently novel situations to reveal surprising connections with apparently far-flung domains.

I have colleagues working as hard as they can to answer the need for complexity. I just hope their work can stem the tide of unthinking “learning outcomes assessment” that Jonathan Kozol pillories in Letters to a Young Teacher.

I really, truly do not think that Likert scales or uniform tests or other simplistic measures are up to the task of helping us map or understand this most profound practice we call “education,” by which I take it we mean a deliberate approach to learning, part of which must include learning about one’s own learning. In other words, the deliberate practice of leading another’s cognition into a richer and more effective relationship with itself.

Of empowering and advancing the brain’s self-shaping capabilities.

I don’t have answers, but I do have a deep intuition that we can best think about this kind of complexity by thinking about similar networks of complexity that have emerged in human experience. (Here’s where I wish I’d majored in anthropology.) There are two such networks I think about a lot these days: language, particularly written language, and the Internet. In this podcast, which records a presentation I did over a year ago at an EDUCAUSE Learning Initiative annual meeting at the invitation of my hero, friend, and colleague Chuck Dziuban, I try to think about assessment by thinking about the emergent properties of the World Wide Web. It seems to me very interesting that a big part of Web 2.0 has to do with assessment, evaluation, reviews, and so forth. Is there a way these emergent phenomena could suggest more comprehensive, inclusive, and meaningful modes of assessing learning? I don’t know, but I do think it’s a question worth asking.

Longtime listeners will hear some familiar themes in this podcast, but cast in a different light. The Shakespeare bits develop some ideas I first began to work on in the “Proof That Matters” talk I did for a K-12 Online Conference a few months before I did this talk. All the ideas here need a great deal more development. I do hope, however, that they’re moving in a more answerable direction than most of the assessment talk I’ve encountered during the last few years.

EDIT: Janet Hawkins alerts me to some parallel thoughts:

http://doyle-scienceteach.blogspot.com/2010/03/rttt-antithetical-to-public-education.html
https://www.blogger.com/comment.g?blogID=4956989639073843954&postID=3538222791054286821

Podcast: Play in new window | Download (17.1MB)

Welcome to WordPress.com. This is your first post. Edit or delete it and start blogging!

For days now, I’ve been mulling over this session, and the Twitter response to it while it was happening. (I was there and in that Twitter stream.) Jim’s post and the extraordinary set of comments it elicited have catalyzed my own efforts at response here.

It turns out that I have very conflicting responses. I’m sure I’ll have more as I continue to think about the session and its aftermath. I post these responses in an effort to keep my thoughts going. I have no ironclad conclusions to offer and I look forward to more conversations as I try to sort things out in my own mind.

I thought at the time, and still think, that Bob Young was not just ignorant of his audience, but at least mildly contemptuous of it. One colleague afterwards said to me that Young had been “baiting” us, and I think that’s right. I’m not a fan of confrontational ha-ha’s, particularly at the end of an event that works so hard to encourage mutual support, inspiration, and optimism–and not just through feel-good boosterism, but through thoughtful, open, determined conversations that have the essentially hopeful mission of education at their core.

When it became clear that Young had not prepared any remarks for us, that he had nothing to show us beyond the front page of lulu.com, I was at first mystified, then insulted, then angry. I also thought he was just a little too calculating in his constant self-deprecation, most of which took the form of sniping at school and academics generally. That’s not to say that school and academics don’t deserve attacks–I’d be the last to say that–but I thought his remarks were shallow and dismissive and unhelpful. That he felt he’d wasted four years on a history degree, without a single teacher or classmate or reading making any apparent impression at all, suggests not just that he feels thrown away by the educational establishment (as many people are, to be sure), but that he had a chip on his shoulder the whole time, and one that he wanted us to admire.

Then he started in on the “damn idiot students,” and I felt my gorge rising. This was my fifth ELI/NLII meeting, and I’ve never heard such casual cruelty from the podium.

Yet the nagging question remains: did Bob Young’s inexcusable behavior justify my own snarkiness on Twitter? No. There are some forms of solace that don’t really soothe anything, and I wish I had not been so free with my own anger and dismissiveness on a public forum that would represent ELI to the world. As one colleague often says of such behavior, it just “feeds the beast.” I knew better.

That said, there was also an attempt on Twitter to engage with Young honestly and seriously. There were moments of meaning as well as reaction. But it’s quite true that in the moment, emotions were running so high that communally-fed reactions outpaced communal meaning-making. And in the Twitter environment, those reactions have a long tail that they wouldn’t have if we had simply met for coffee afterwards and vented. I’m certainly not proud of my own snarkiness and venting on Twitter during the event, no matter how helpless (and hopeless) I felt as the runaway train careened down the tracks. These thoughtful responses from another colleague who was not there, but who saw the Twitter stream in action, are a valuable lesson for me in the destructive potential of the backchannel.

But there’s one other thing to note here. A keynote speaker has an enormous responsibility. At these moments, the entire conference comes to a point of focus on one speaker, one set of ideas, one address. ELI 2008 was full of enormously talented speakers, and any of the featured speakers would have been a much better closing keynote than Bob Young was, though I’m sure no one on the program committee had any idea Young would do what he did. But back to the point. Time slots on a program are always precious, especially when so many wonderful ideas and speakers are in circulation. I think we all felt an enormous wave of disappointment (this comment eloquently describes the feeling) that an extraordinary opportunity had been discarded by a speaker who seemed to have no sense at all of the gift he had been given. The program committee, acting on our behalf, gave him a treasure, a great privilege, and to him it appeared to be no occasion at all–nothing to rise to, nothing to answer, nothing to value. Instead, we got jokes about his inadequate speaker’s fee and the relative IQs of his various audiences.

This should not have been just another day on the IT circuit for Bob Young. This was a chance to engage with one of the best chances at academic transformation on the planet. We came to learn. I think we would have responded well to challenges, even to thoughtful provocation. Perhaps Young’s educational experiences really have scarred him to the point that he cannot be open or serious in the way he presents his own ideas, at least to an audience like ELI. But on that day, in that room, I felt hollowed-out and disheartened.

I won’t try to justify my own backbiting on the backchannel. I can’t, and I’m sorry for it. But it’s important to realize that Bob Young is not the only one who’s been made fragile by his educational experience. By analogy, if any of us was invited to speak for 45 minutes to a provost or president, to say nothing of a room full of them, would we do what Bob Young did? We know how rare and precious these visionary opportunities are.

Only at the end did I feel Bob Young was making any real attempt to connect with us, or engage seriously with ideas. When he shared his thoughts about keeping the MIT Press thriving in the midst of the challenge Lulu.com posed to its business model, I believed him, and wanted to hear more. When he told the story of the librarian who implicitly chided him for checking out so many books, and told us that this was the only teacher who had ever made an impression on him, I felt real sorrow over the way he had been cast aside by his own education, and I wanted to hear much more about how he had kept his head high and his determination alive in spite of being told again and again how he didn’t measure up. In a conversation after the session, another colleague said that Young should have led with the librarian story. I thought that a brilliant idea. Think of how the entire talk would have been reframed as a critique of academic processes and dismissiveness, but with the positive direction of imagining a new educational community that finds the brilliance in each student, and encourages real curiosity and intellectual diversity. That would have been a talk worth hearing.

Bob Young clearly has that talk in him, and he clearly has vital stories to share. Why didn’t he choose to give that talk and share those stories with us? At the end of it all, that’s the question that haunts me most.