About the Adjudication of Band Contests
By George Yenetchi (The Ross S. Sterling Band Association)

It happens every year in October and November. Marching contest judge bashing gets into full swing. Band Newsgroups like rec.arts.marching.band.high-school and alt.marching-band.texas are deluged with posts questioning the decisions of the judges of BOA, ISSMA, EMBA, USSBA, UIL, TOB, COB, DCI and every other marching contest sponsor. Many of the authors of these posts are in high dudgeon. A few even resort to language of the most impolite sort. They are not content merely to disagree. They impugn the judges' character and competency. If we were to believe these posts, then there isn't a decent panel of band contest judges to be found anywhere in America.

Imagine if these allegations were true. The implications boggle the brain. Oh, immitigable woe! How would we have come to a state where the finest music colleges all across America were turning out a steady stream of band directors who, when because of their success, they are considered  by their peers to be fit to be judges, become biased, venial, stupid, or worse? By what misbegotten plan would the moral fabric of America have been so rent that even band was riddled by incompetence and corruption?

But sink not into despair. These complaints are mere teenage trumpery.  They are 99.99 % nothing but adolescent asininity. A quick review of writings on adolescence by eminent thinkers from Socrates to the present reveals that adolescents have always been prone to rash judgments.  Their opinions are more the result of hormone-distorted emotions than of sound reasoning. That is why they do not let 14-year-olds serve on juries. It is perhaps best not to pay too much attention to such utterances. Given time, the kids will grow up.

And…yet…,

There lurks a nagging suspicion. These judge-bashing band members may not be expressing their own original ideas but rather those they have picked-up from adults. Not just any adults but perhaps even from their parents. And their parents are band parents. That is a disquieting thought. The mind yearns to reject it out of hand.

And………yet……,

Honesty compels the reluctant admission that, perhaps, a few band parents are judge bashers. OK, it is not just a few, it is more than a few, maybe even quite a few. However many it is, it is too many. A "Kill the Umpire" mindset may be tolerated in athletics, but not in band, never in band. Band is the last great hope of humanity. (That may be a bit of an exaggeration but band is at least sort of important.)

Why You May Not Agree with Marching Contest Judges (and why it is not the judges' fault)

As stated above, some marching show spectators (especially band members and their parents) find that they sometimes disagree with marching contest judges' decisions. That is OK. They sometimes attribute this to incompetence or venality on the part of the judges. That is not OK. It is not true for a one thing. You may run across a less than ideally astute judge, but only very rarely. Contest organizers seek out the best judges available. Use of multiple judges (3 to 8) helps insure that one bad judge does not skew contest results. Venality in judges is even rarer. Judges are honorable. If they weren't honorable they could certainly find a better way to exploit being dishonorable than judging marching contests. Politics comes to mind. There is a good reason why the great criminal masterminds have never paid any attention to band. There is not much money in it. By the time bands get to contest, band parents' bank accounts are too depleted to fund attractively large bribes.

Think about what is involved in judging a marching contest. Making judgments in artistic matters involves every known shade of gray plus a few shades whose existence science has only inferred. Many great musicians scoff at the very idea of musical contests. In athletics, things are obvious. Either the high jumpers clear the bar or they don't. A four-year-old could competently judge a high jumping contest. In the arts, what is good is always a matter of opinion. That's why there are 50 different recordings of Beethoven's 9th Symphony on the market. As a great philosopher once wrote, "1+1=2 is fact, almost everything else is opinion."

There are many reasons you might be in honest disagreement with good judges.

Judges look for performance excellence not entertainment value.

Judges look at how well a show is performed. They pay little attention to how entertaining it is. A skilled performance of a dull show will score better than a mediocre performance of an interesting show. (Of course, the ideal is a great performance of a great show.) Audience members are impressed by entertainment value, as well they should be. They should not compare their opinion of a show, which was based on how well they were entertained, with the opinion of judges who were evaluating how well specific skills were demonstrated.

Contest guidelines require that bands demonstrate certain specific skills in performance.

For example, guidelines often require that bands demonstrate both loud and soft playing (BOA guidelines specify "all dynamic levels"). A band that only plays loudly, no matter how well they do it, will lose points. A loud thrilling show that brings the audience to its feet cheering and clapping may therefore score poorly. Another area that often lowers the scores of bands that look and sound very good is woodwinds. Increasingly, judging guidelines want the woodwinds to get equal time with the brass and the percussion. The days when clarinets could be seen and not heard are past.

Judges pay very little attention to size, uniforms, etc.

Just like an elephant, a 400 piece band is impressive just by being there. However, judges give no points just for being there. At most contests, judges ignore who has the nicest uniforms, the prettiest flags or the best props. It is hard when you have just spent $80,000 on uniforms and the band parents have worked for 4 weeks making props to realize that the judges are more worried about how your band's feet look.

Judges can only judge the one performance they see.

Judges do not know how hard the kids have worked. They don't know that the soloist got sick. They don't know that the band performed the show better last week. They don't know what special adversity your band overcame. If they knew, they might be even more impressed than you are. But they don't know.

Judges detect small mistakes.

Judges have the training and experience to detect tiny problems with intonation, tone quality, balance, ensemble, etc. These subtleties aren't even noticed by normal people. These small differences separate the very very good from the merely very good. Just because these differences are tiny does not mean that they are not real.

Contests that give rankings usually use forced rankings.

There are no ties. If 10 bands compete and earn almost identical scores, the judges will still rank them from 1 to 10. Usually the judges aren't directly involved in the ranking. They just give each band a set of scores. A computer figures out the ranking, breaking any ties by using criteria that the judges may not even know. One band must be 1st and one band must be 10th. It has to be this way. First place trophies are expensive and they only had money to buy one of them. This does not always mean that the 10th place band was a lot worse than the 1st place band. Rankings in "Consumer Reports" are always qualified by a statement like, "Differences in rankings of 5 places are not significant.". Marching contest results could truthfully carry similar advisories.

All artistic judgments are subject to subjectivity.

Even the best judges do not always agree. One judge may be more concerned with intonation while another will care more about ensemble. These are not evil biases. These are legitimate artistic differences. At some contests, you get judges who tend to pay less attention to the particular weak points of your band and at other contests you don't.

As we enter the 3rd Millennium the average level of school band performance is as high as it has ever been. It is still improving. As the real quality differences among bands decrease, the judge's job becomes harder and the rankings mean less.  To illustrate these points let us take part in the Red Rubber Ball Contest.

Judging the Regional Red Rubber Ball Contest.

Welcome fans of the Red Bouncing Sphere! In today's contest we will select the best local red rubber ball to represent our region in next week's State Red Rubber Ball Championship.

Start with 6 identical red rubber balls. Number them so you can tell them apart. Then designate some people (students or band parents will do) to be judges in the Regional Red Rubber Ball Contest. Choose your judges by any criteria you think appropriate.  

Let the contest begin! Have each judge separately rank the red rubber balls from the best to the worst with no ties (a forced ranking). The judges are not to look at the numbers. Compare their rankings. Do the judges' results agree? No. Apply a typical contest formula (such as throwing out each ball's high and low ranking and averaging the rest). Make a final ranking.

Now post the official results of the Regional Red Rubber Ball Contest. Is there any difference between the 1st ranked ball and the 6th ranked ball? No, they are identical. But it doesn't matter, send the 1st ranked ball on to the State Red Rubber Ball Championship.  The other red rubber balls will stay home and become the subjects of ridicule on the alt.rec.arts.red.rubber.ball newsgroup.

There is a second part to our contest.  Have the same judges rank the balls in order of the identifying number (from 1 to 6) on the ball from lowest to highest. Do the judges' results agree? Yes, they agree exactly.

Were the judges less honest or more biased in the first part of the contest than in the second? No, you used the same judges in both cases. The differences among the things being judged were obvious in second case and non-existent in the first. All real judging situations fall between these two extremes.  As your band goes higher in competition, all the red rubber balls or bands look more alike.  It is then much harder to rank them according to any rational criteria.

There is no perfect way to combine judges' scores and rankings. Every method is a compromise. This brings us to-

Fun with Band Math!

Here is an example. Assume three judges were asked to score bands from 0 to 100. The judges' sheets show this result-

Band A: Raw Scores 82, 85, 80, Averaged Score 82.3, Judge's Rank, 1, 1, 2, Combined Rank 1st
Band B: Raw Scores 80, 83, 85, Averaged Score 82.7, Judge's Rank, 2, 2, 1, Combined Rank 2nd

If you give 1st place to the band with the higher score then you give 2nd place to the band that 2 out of 3 judges thought should get 1st place. If you give 1st place to the band that 2 out of the 3 judges thought was the winner then you give 2nd place to the band with the higher score. If you are going to rank them, you must do one or the other.

Either option seems totally reasonable when you state it alone. "The band with the highest score is the winner." "The band ranked 1st by a majority of the judges is the winner." Who would argue with either one?

The judges handed in their score sheets and then left for dinner. The authors of the contest rules didn't foresee this situation. They assumed that the top ranked band would naturally also have the highest score. (They were musicians, not mathematicians.) The final decision is up to you, yes, YOU who didn't even see either band perform.

You can't declare a tie since you only have one first prize to give, one very nice but indivisible prize. The crowd is getting restless. Quick, tell us, who should get 1st place, Band A or Band B? (Be prepared to explain your decision, in person, to the parents of the band that gets second place and no nice prize.)

More Fun with Band Math!

A problem with direct scoring is that a judge who scores over a wider range than the other judges can dominate the average. Here is an example.

Band A: Judge 1's score- 65, Judge 2's score- 68, Average- 66.5

Band B: Judge 1's score- 70, Judge 2's score- 69, Average- 69.5

Band C: Judge 1's score- 95, Judge 2's score- 60, Average- 77.5

Band D: Judge 1's score- 50, Judge 2's score- 65, Average- 57.5

Band E: Judge 1's score- 55, Judge 2's score- 63, Average- 59.0

Band F: Judge 1's score- 85, Judge 2's score- 62, Average- 73.5

Rankings-

Judge 1- C F B A E D
Judge 2- B A D E F C
Average of Both- C F B A E D

Wow! Judge 1 and Judge 2 disagreed completely on the ranking of the bands but the final ranking of the averaged scores is identical to Judge 1's ranking. Judge 2 might as well have stayed home. This crazy outcome happens simply because Judge 1 spread his/her scores over a wider range than Judge 2. Without a great deal more information, it is impossible to say that either Judge 1 or Judge 2 is right or wrong. We can only note the result. Judge 2's scores ended up not mattering. That is bad news for Band B.

(It is especially bad news for Band B since I happen to know that Judge 2 was THE WORLD'S GREATEST EXPERT ON MARCHING BANDS (you know whom I mean) and that Judge 1 was my great aunt Agnes. Aunt Agnes was a last minute substitute for an absent judge. She is a dear lady. Sadly, she has become rather nearsighted and hard of hearing. Still, she is very hale for a woman of 98 and has lost none of her feistiness (she was a suffragette and earned a BA in biochemistry). This was the first time she had seen a marching band since she saw John Philip Sousa's Band back in 1919. She was very excited about getting to be a judge.)

If the rankings of the two judges are combined in Texas UIL fashion then the results are-

Band A: Judge 1's rank - 4, Judge 2's rank - 2, Total- 6

Band B: Judge 1's rank - 3, Judge 2's rank - 1, Total- 4

Band C: Judge 1's rank - 1, Judge 2's rank - 6, Total- 7

Band D: Judge 1's rank - 6, Judge 2's rank - 3, Total- 9

Band E: Judge 1's rank - 5, Judge 2's rank - 4, Total- 9

Band F: Judge 1's rank - 2, Judge 2's rank - 5, Total- 7

Combined rankings of Both Judges- B A (C,F tie) (D,E tie)

Both judges had an effect on this composite ranking. The ties possibly are a problem.

Well, have you given up any hope of finding a contest scoring method that makes sense? No? OK, you asked for it-

Even More Fun with Band Math!

Think things will be all right if you combine the judges' rankings? Here is an example (using 3 judges but that same thing can happen with more).

Band A 1, 1, 5, Total- 7, Composite rank- 2

Band B 2, 2, 2, Total- 6, Composite rank- 1

Band C 3, 4, 1, Total- 8, Composite rank- 3

Band D 5, 3, 3, Total- 11, Composite rank- 4

Band E 4, 5, 4, Total- 13, Composite rank- 5

So, 1st place is going to go to a band that ALL the judges agreed belonged in 2nd place. The band that 2 out of 3 judges thought was the best will only get 2nd place. But wait, it gets better. In this contest in addition to a ranking, each band also gets a Division score. The first two judges gave Band A a "Division 1" and Judge 3 gave them a "Division 3". By the contest rules, a band gets whatever division score at least 2 judges agree on. Band A gets an overall "Division 1". All the judges gave Band B a "Division 2" so Band B gets a "Division 2". Imagine that sublime moment when the announcer tells the spectators that Band A got 2nd place with a "Division 1" performance and that Band B got 1st place with a "Division 2" performance.

Band Contest Urban Myths

Judges prefer classical music. This is a popular complaint with people whose bands don't play classical music. It probably results from the fact that most classical (i.e. serious) music is more technically difficult than most popular music. A band which does a good job performing "The Rite of Spring" has done a more difficult thing than a band that does a good job performing "The Sound of Music". It is still OK to prefer "The Sound of Music" but recognize that it can't showcase as wide a range of musical skills as some other music.

Judges are influenced by a good band's reputation. A popular complaint with people whose bands are new to a particular contest. Perhaps some judges are so influenced but this is as likely to cut one way as another. The judges may (perhaps even unconsciously) expect more from good bands and judge them harder. It is just not practical to prevent judges from knowing the identity of the bands they are judging in a marching contest.

The (insert name of any band that outranked your band at contest) Band violated practice time rules. This charge, when not completely unfounded, is usually the result of misinformation. The practice schedule for both a non-varsity and varsity band may have been mistaken as being that of the varsity band only. What appears to be an early start to practice may only be a legal basic-skills training period scheduled very late in the summer. The seemingly improper practice may be for a parade or other appearance for which additional practice time is permitted. Significant practice time violations are unlikely because so many people are involved with a marching band. It is very hard to keep secrets. Someone is going to blab.

Judge "Smith" was offended by us a few years ago and has it in for us. When not totally unfounded, it invariably turns outs that the judge just happens to have the same name as the mistreated person. There are many "Smiths" out there. The vast majority of contests take pains to minimize possible conflicts of interest.

The way they calculate the ranking is unfair. It may be meaningless from a rigorously mathematical point of view. It is not unfair, as long as it applies the same way to all the bands.

Possibly True Band Contest Complaints

Have to include a few of these just to show that there are some.

The judges' guidelines penalize our type of show. Entirely possible. If so you have many options. Do a show that is in-line with the guidelines. Try to get the guidelines changed. Go to a different contest where the guidelines favor the kind of show you do. And, the ever popular, sit around and complain about it while doing nothing. For maximum effect, BP'00's recommends doing the latter option with friends in a nice restaurant.

The contest is too far away./The contest ran too late./The weather on contest day was terrible./ We feel your pain and hope you could at least afford comfortable buses.

Copyright 1999 by George Yenetchi Original Article Here