We often miss opportunities to generate and share knowledge about actual practice from student competitions and CLE programs. This post in the mini-course suggests some ways to get more benefit from these activities.
When I have judged student skills competitions at my school, typically one or both of the other judges in my “panel” were practicing lawyers. We exchanged our views (privately and in feedback sessions with the students), which sometimes overlapped and sometimes didn’t. These discussions usually were quite interesting and I thought it was too bad that we felt rushed and that our feedback was limited to the small group of students in each performance.
To harvest the benefit of these discussions, competition organizers could hold a follow-up event in which student competitors and judges could discuss general issues in the students’ performances without identifying any students. This would be a great opportunity for academic and practitioner judges to discuss their perspectives, identifying areas of agreement and disagreement. This should provide a terrific learning experience for everyone. It might be particularly valuable for students who feel upset because they used techniques taught in class, only to be sharply criticized by practitioner-judges.
It should be pretty easy to organize a general debriefing event following the competitions. Ideally, it would be scheduled well in advance so that invitations to potential judges would include the date, time, and location of the event. Presumably, judges could get CLE credit for attending the event – and actually learn a lot more than from the judging experience itself. Light refreshments could be provided as an extra incentive for students to attend. Although this event presumably would be of interest primarily to the student competitors and judges, it could be open to everyone in the school and perhaps the local practitioner community as well.
The event could be run like a “focus group class,” with selected speakers on a podium, or as a general discussion moderated by a faculty member. To identify issues for discussion, event organizers might ask for suggestions from students in the competition and review judges’ scoring sheets.
Getting More Out of CLE and Training Programs
I love giving talks at CLEs and trainings. I am always fascinated to hear practitioners’ perspectives and they generally are very hungry to learn.
Many speakers get some basic information from the audience as part of their talks. It is not unusual to ask for a show of hands about the audience and their experience. For example, speakers might ask members of the audience about their practices (e.g., as an advocate, mediator, arbitrator), amount of experience, types of cases handled, etc.
It would take just a little extra planning to get more substantive information at the outset and at various times during the programs and feeding it back to them as part of the programs.
Speakers could elicit responses by having people raise their hands, complete short surveys, meet in small groups to discuss specific questions, use “clickers,” or perhaps use other techniques.
Ideally, you would arrange for some way to record the audiences’ responses, perhaps by typing them into a document projected on a screen or collecting survey forms. Any data collected might also be written up and circulated to attendees after the program. In my experience, practitioners generally are fascinated to know how they compare with their colleagues. So they probably would greatly value your distributing information collected at the program.
As part of the conversation between the program organizers and speakers, you might discuss whether something like this would be appreciated and, if so, what questions would be good to ask. At the last ABA conference, we had a program in which we asked attendees what they would like to know about negotiation and produced a valuable list of questions.
In addition to producing information for the CLE, you could collect data that could contribute to your scholarship. For some examples, see the federal court clerk, cooperative lawyer, and quality of mediation studies described in this post. Of course, if you want to use the information you collect for your scholarship, you would need to get your project approved by your school’s IRB or comparable ethics board.
For student competitions, I think it may be a good idea as well to take the time to sit with the students one at a time with the panel, in private, to discuss what each panelist found best about a student’s performance, and what they might work most towards improving. If you kept it to one best and one worst issue per panelist, per student, it would give the students more personalized feedback, without compromising too much time.
Thanks to Jennifer, Tom, and Debra for your comments.
Skills competitions are so prevalent – and problematic for the reasons you describe, Debra – that it would be good if we could extract more benefit from them.
For various reasons, I think it’s inevitable that judges will be inconsistent. I was involved in drafting instructions for judges some time ago and I am convinced that some judges will not follow instructions, no matter how clear, well drafted, etc.
Having a general de-briefing may address the inconsistencies to some extent, albeit imperfectly. If some judges tactfully express reasonable views, it could give students some validation for their efforts when some judges give problematic feedback.
The logistics for such de-briefings would depend on the circumstances of each competition. In my school, judges are supposed to arrive at 6 for dinner and instructions. Then we do three rounds of judging, leaving us free to leave at 10 – assuming that we don’t run behind schedule, which we always do. I suspect that no one would want to stick around to start a general de-briefing at 10:30 pm. Competitions operating with different schedules might arrange for general de-briefing right after the last round.
Being rather active in the competition arena, I think that the suggestion for a large scale de-brief post tournament is a fantastic idea, where the potential for “learning” is enhanced considerably.
I agree that a large scale debrief following a competition would be incredibly beneficial for students if the competition organizers can coordinate it and IF judges will commit more time. Maybe it would be possible to hold something like this immediately after rounds so the judges do not have to come back another time.
I coordinate all of our competition teams throughout the year (eight annually) and have been to almost every tournament as a coach. All too often, I have noticed that judge feedback is largely inconsistent – some practicing lawyers tell students that they were not “real world” enough while other judges who know the tournaments commend the students (or not) for playing to the competition scoring rubric. For students, it is the luck of the draw regarding which judges they get. I’ve been to some tournaments where the judges were not even lawyers and told the students they conveyed incorrect legal information or that they should not have talked about the law at all.
I honestly think this comes down to a larger issue of how ADR competitions are structured and how judges are selected. But overall, in my experience, the students crave feedback and have been pleased when they receive it.
There are a lot of connections between these ideas for improving CLE and the new ABA rules for formative assessment in law schools. If one summative assessment at the end of a course doesn’t promote learning, then certainly *no* assessment in any form in a CLE doesn’t promote learning either. Bringing in the low-stakes formative-assessment techniques would make CLEs more engaging and educational. The Law Teaching website (Institute for Law Teaching and Learning) is one good resource along these lines.