Judging judges

[Originally published as an Editorial]

Before every major pipe band competition, most competitors eagerly await word of who’s judging. The powers-that-are with various pipe band associations appoint the adjudicators. To our knowledge, judges are put on contests rather arbitrarily. A judge’s assignment to a specific competition has little to do with reputation, and more to do with 1) their availability, 2) their accreditation, and 3) when they last judged the event.

For the biggest contests, like the Grade 1 World Pipe Band Championship, it would appear that nothing less than a simple rotation system dictates if a judge will be put on a circle. Those who judged the grade the previous year or two are almost assured of not being on it the following year.

When the lists of judging panels come out, rarely, if ever, are all bands completely happy. One rogue judge (rogue, that is, in the eyes of a single band, not necessarily all competitors), in the case of the World Pipe Band Championships can dictate whether a band will win or not. Because the process of assigning judges is based on the points above, the competitors themselves have no input on who will call the contest.

A trend in major sporting events is for leagues to poll players and managers every year for their opinions of referees and umpires. The officials who rate the best in the eyes of the teams or players they judge are then placed on the most important events. While the umpires and referees still might make controversial or even wrong calls, the players and teams are more positive about it, since they had a direct hand in selecting them. Accusations of an umpire or referee preferring one team over another are reduced, since the players themselves essentially selected the officials whom they respect the most.

Why not use a similar system for pipe band competitions? Instead of associations considering their entire panel as basically equal in ability and reputation, wouldn’t it make more sense to ask bands after each season to rank each judge? The reality is that some adjudicators are considered to be better than others, and are respected more by the bands they judge.

We suggest a simple system: each band receives a single sheet that lists all of the association’s judges who meet the criteria to judge the band’s grade. The band then ranks each judge on a scale of one to five, one being poor and five denoting excellent. All surveys are then tabulated, and the judges with the most aggregate points are then assigned to the contests that are the most important.

We suggest a rating for each judge across three categories, in no specific order: 1) Integrity, Ethics, and Honesty, and 2) Musical Knowledge and Experience. The top rating for a judge would be a 10.

The best judges — say, those with a rating of more than 8.5 — would be reserved for grades 1 and 2, and that’s appropriate. These are the grades that require judges with the most knowledge, skill and respect of those they judge. The ability to distinguish between an excellent performance from an awesome one is a talent needed at the top competition level. The separation between lower grade bands tends to be much wider, and thus the contests are clearer cut.

Judges who receive a rating average of, say, between 2 and 3 would have their credentials reviewed, their past results and scoresheets appraised. They would be apprised of their situation and allowed to improve their rating through judging less important events. Those with a score below 2 would be removed from the panel outright, but allowed to re-sit an examination and go through an apprenticeship program to return to the panel.

The Competing Pipers Association took the bold and courageous step several years ago by asking every one of its members to rate all of the judges on the existing adjudicators list of the Joint Committee for Piping. Everyone who had been known to have judged any solo piping contest in the UK was included. The CPA assembled all of the results, and then produced its own ranking system. Judges who received particularly low marks were excised from the list, while those with the highest marks now sit on the most important events. While not foolproof or accepted by all competitions, the CPA’s system has been applauded by most of its members.

Because pipe band competitions are governed directly by associations, whatever the competitors want, the competitors should be able to get. Just as organized religions are answerable to “the body of the kirk,” pipe band associations must directly respond to their members’ wishes.

Competitors rating judges makes sense. It happens in professional sports leagues like football and baseball, and with subjective competitions like figure skating and Drum Corps International. A few egos might be bruised, but by placing judges approved by the competitors on the most important competitions, results would be more accurate and accepted more readily by the competitors themselves. And that’s what it’s all about.