Scrutineering Question

[hbetx9]

I have a question about the scrut sheets, or in so much as the algorithm for determining winners. For example

Judges ||| Cumulative ||| placement
3 6 6 1 1 1 5 ||| 3 3 4 4 5 7 7 7 ||| 2.0
6 4 4 7 4 3 4 ||| - - 1 5 5 6 7 7 ||| 4.0
4 7 2 2 7 2 1 ||| 1 4 4 5 5 5 7 7 ||| 1.0
7 8 5 8 5 7 3 ||| - - 1 1 3 3 5 7 ||| 7.0
5 1 1 5 6 4 7 ||| 2 2 2 3 5 6 7 7 ||| 5.0
2 5 8 3 3 6 6 ||| - 1 3 3 4 6 6 7 ||| 6.0
1 3 7 6 8 8 8 ||| 1 1 2 2 2 3 4 7 ||| 8.0
8 2 3 4 2 5 2 ||| - 3 4 5 6 6 6 7 ||| 3.0

How come the first and second place couples didn't get switched. As I understand, we look at the first place column, determine the highest number ( if possible ), then eliminate that row, and move to the second place column to determine 2nd, and so forth. I've noticed a couple instances like this one where the anticpated couple does not take the placement under this algorithm, what is my error in understanding?

[danceforfun]

As you see with 7 judges you need a majority of 4. With the first place you have only 3 for the first couple. So you go to the first and second places. Here you find a majority of 4 for the third couple who actually wins this dance. So you go on to 1.-3. places where the first and the last couple have 4. So you consider the 1.-2. places (both 3) and then the 1. places where the first couple has got 3. So the first couple is 2nd. So you go on with all other places.

This is called skating system. Only if two couples have exactly the same number of places (and a majority) the dance will be shared.

It´s a little bit difficult, but after a while quite easy to understand.

[Another Elizabeth]

Mathematically, the system is also sometimes called the median ranking system. One way to look at the two couples you mentioned is that the median rank received by the first place couple is a '2', and the median rank received by the second place couple is a '3'. (The third place couple also has a median rank of '3', at which point you look at the marks making up the "top half" of the marks received.)

[NielsenE]

This looks like the output from the scrutineering program I use. I'm not sure why the designer implemented it the way he did, but it some of the other software, or by hand, when you assign a place, you'll see a line through the row. Such as

Judges ||| Cumulative ||| placement
3 6 6 1 1 1 5 ||| 3 3 4(6)--------- ||| 2.0
6 4 4 7 4 3 4 ||| - - 1 5-----------||| 4.0
4 7 2 2 7 2 1 ||| 1 4 -------------- ||| 1.0
7 8 5 8 5 7 3 ||| - - 1 1 3 3 5--- ||| 7.0
5 1 1 5 6 4 7 ||| 2 2 2 3 5------- ||| 5.0
2 5 8 3 3 6 6 ||| - 1 3 3 4------- ||| 6.0
1 3 7 6 8 8 8 ||| 1 1 2 2 2 3 4-- ||| 8.0
8 2 3 4 2 5 2 ||| - 3 4(9) ------- ||| 3.0


Which helps to illustration when the place was awarded. I'm working on a new scrutineering package, and making sure it outputs like this is one of many things on my list....

[danceforfun]

We in Germany have a program called "Top Turnier" that uses pdas for the judges. So it is possible to be very fast in larger tournaments (100 couples and more).

You put the pdas into the cradle and trandfer all marks given by the judges within seconds. It is used at "Hessen tanzt" with 8 competitions simultaneously and most of the national championships. I think it is available in english, too.

[NielsenE]

There's a similar system in the US, as well. And the new product I'm working on, includes that functionality as well. However, I'm always a little skeptical of the true time benefits of a PDA system -- I'm almost always (99%) done with one event before the next event finishes. So there isn't any scrutineering delay. Now I do typically take slightly longer than an event to do a final, and couldn't do the instant awards that the PDA based systems allow, however I tend to prefer the awards blocks (as long as the awards aren't held hostage for hours). If there's enough events in the session (~10) you can go straight into the awards and I'll have the last event ready by the time you get to it...

The other side, at least among the events I tend to work (the new england collegiate scene), is that only one organizer seems interested in PDAs if it means an increased cost, and given the expense of acquiring a large number of PDAs the viability of the PDA based systems doesn't seem to be very promising. It would make my job easier, but I really don't see the benefits it offers to a competition over what a fast scrutineer can do in the first place.

Ie, recalls ready in seconds versus ~1-4 minutes depending on round size doesn't change things greatly compared with the results in 1-4 minutes versus 10-15 as some slower scrutineers seem to need in terms of flexibility of event scheduling, etc. Both systems are trivial to hook up to any sort of automated projection system, etc

[Laura]

However, I'm always a little skeptical of the true time benefits of a PDA system -- I'm almost always (99%) done with one event before the next event finishes. So there isn't any scrutineering delay.
A friend of mine who is a scrutineer also reports a similar experience.

It would make my job easier, but I really don't see the benefits it offers to a competition over what a fast scrutineer can do in the first place.
In the case of O2CM, the system I've been using (and which we used at USA Dance Nationals in 2005 and 2006), the benefits aren't so much on the actual scrutineering of events end, but rather on the registration and posting of marks end. It was so convenient for me, as Registrar, to be able to instantly process adds/drops/changes during the event. Also, one "page" in the system shows the schedule of events, automatically highlighting exactly what event is running and I think also adjusting for time delays or speed-ups, so that when some random dancer comes to me (when I've been busy processing paperwork and not paying attention to the event) and asks "what event is this" I can just look at the screen and tell them. Also, the marks come back to me right after they've been tabulated by the scrutineering system, so I can print them out and post them or stick them in "the book" as soon as the awards are announced. So, from the standpoint of interfacing quickly with the competitors to service their requests, this kind of system makes everything faster and easier and more accurate. No more running back and forth to the Chairman or Scrutineer with adds or drops. No more waiting for someone to have the time to print the marks and run them to me so I can hang them up. The system has an AIM-like "chat" feature built in so the Chairman, Scrutineer/Tech Person, and Registrar can all pass information around instantly even though they are in different parts of the ballroom. When the Deck Captain checks someone in, it shows up on all our screens so we can tell right away who is missing, and the Registrar can indicate that the person has been by to pick up their number, or can say that the person hasn't shown up and and so the Deck Captain can scratch them from their PDA in the on-deck area, Etc. etc. The computerized PDA systems save time and effort behind the scenes, but not necessarily with the act of scrutineering an event.

We used O2CM for the first time last weekend without its developer being present. I think this might have been the first time ever that O2CM was used to run a comp without the developer! We ran into two major snafus, but fortunately our team knows what they were doing so we made everything work, and we reported the bugs back to the developer for him to investigate and deal with. These new computerized systems are definitely the wave of the future, but are also still very much in the early-adopter phase. Competitions can definitely be run without them (and have been for what 60 some-odd years now) so I can certainly understand why there is lack of need and/or interest in some quarters. Still, as an organizer, even with the bugs in this early phase I am quite enthusiastic about using them. A year or so ago I wasn't, but after seeing it in use at 2005 USA Dance Nationals, and using it for 2006 USA Dance Nationals and then last weekend, I prefer them now.

[avab]

The line drawn across the placements dates back to doing the math by hand. If you draw the line across when the placement was determined, it's easier to not figure those marks into the next placement. It's a visual convention that makes alot of sense.

The difference between the German PDA system and o2cm is that o2cm doesn't require the pdas to be replaced on the cradle to send marks into the server. The German judge who was at USA Dance Nationals this year has already asked about using o2cm in Europe to compete with the current system. He said that this system is much more efficient.

While using pdas doesn't necessarily speed up the calculation much over manual input of marks (depending on the speed and accuracy of the scrutineer), it does, however, reduce judge error and hence scrutineer confusion (such as the judge writes down 194 but there is no 194 on the floor; the scrutineer has to send the sheet back out to the judge for clarification which slows down the process). Judges have more time to look at the dancing. MCs don't have to tell the judges who is competing against whom because that's clearly visible on the pda. This definitely saves time. As the advent of compmanager made larger comps feasible, so the pda based systems also increase the number of entries that can be handled.

And, yes, it is slightly more expensive to run than compmanager in its current form. This is because there is equipment to rent. The scrutineer or technician must invest in a number of pdas, ac cords, extra batteries, three computers, a router, extension cords, styluses for judges, etc. as well as packing cases to move all the equipment around. But the licensing fee is less for o2cm than compmanager, and you get the advantage of a fully functional internet registration and reporting feature. But compmanager is also working out a pda based system, so the costs of using that will increase to accommodate equipment rental as well. Even if all the judges in the US bought pdas, it makes sense to use models that have been proven to work with the system in question. Hence rental seems to be way it will be for a while at least.

[chandra]

recently in the west coast swing world people have been using a similar system. The judges all get Tablet PCs to work with. Judges I have talked to say the computers give them more judging options. They are also neate for registration, because their registration info is all in a database from comp to comp, so when i signed up for one comp, I just tell them my last name, and they already have my competitors ID, competition level, address, everything on file.
Also when there is a scratch or an addition at competition time its easier, because-the name can just be added by the score keeper, then the judges "r e f r e s h "* and the name shows up! Also, the tablets had solitaire and internet, which the judges seemed to enjoy. (They swear they arent playing solitaire when they should be judging you!)

*I think I need to go look this up in urban dictionary _^ whats wrong with r e f r e s h ?

[NielsenE]

Yeah, I'm currently investigating about 4-5 different form factor devices, ranging from standard PDAs up through tablet PCs looking into the usefulness of the interface for the judges, battery life, and weight over a long day of judging.

I also agree that the major gains for the competitions is the networked/communicating stations between the registration desk, scrutineer, deck captain, MC, etc. The PDA piece is not as important. (I already provide the internet registration/reporting in my market)

[DrDoug]

Even if all the judges in the US bought pdas, it makes sense to use models that have been proven to work with the system in question. Hence rental seems to be way it will be for a while at least.

I would guess (and hope) that security considerations would preclude letting judges use their own PDAs. Out of mild curiosity, what level of precaution do current systems take against being hacked?

[saludas]

The ultimate advantage to using these PDA based systems is for the competitor - 'instant' results, as soon as the event ends. It is SO frustrating to remain in costume for an hour after your event, to wait for 'awards'... and for the audience, which by the time awards are givine, has forgotten who was dancing etc. Combined with the strange awards situation where the competitors have their backs to the audience and are announced by their numbers, it makes for a very insular-looking and excluding situation for people who paid to see the dancing competition.

As the comp owners 'get with the program', I'm eagerly awaiting the 'tipping point' when it will not be viable to have a comp WITHOUT instant results - the point at which people simply do not attend a comp if it doesn't have this.

[danceforfun]

There's a similar system in the US, as well. And the new product I'm working on, includes that functionality as well. However, I'm always a little skeptical of the true time benefits of a PDA system -- I'm almost always (99%) done with one event before the next event finishes. So there isn't any scrutineering delay. Now I do typically take slightly longer than an event to do a final, and couldn't do the instant awards that the PDA based systems allow, however I tend to prefer the awards blocks (as long as the awards aren't held hostage for hours). If there's enough events in the session (~10) you can go straight into the awards and I'll have the last event ready by the time you get to it...

The other side, at least among the events I tend to work (the new england collegiate scene), is that only one organizer seems interested in PDAs if it means an increased cost, and given the expense of acquiring a large number of PDAs the viability of the PDA based systems doesn't seem to be very promising. It would make my job easier, but I really don't see the benefits it offers to a competition over what a fast scrutineer can do in the first place.

Ie, recalls ready in seconds versus ~1-4 minutes depending on round size doesn't change things greatly compared with the results in 1-4 minutes versus 10-15 as some slower scrutineers seem to need in terms of flexibility of event scheduling, etc. Both systems are trivial to hook up to any sort of automated projection system, etc

The time effect grows exponentially with the couples taking part in the competition. Remember that most German competitions have a lot more contestants than the US. With 10-20 couples it takes about 10-15 minutes by hand and 5 with pda. When there are 50 contestants it takes 20-30 minutes by hand and 5-10 with pda. With 100 contestants it can take 45-60 minutes until you get the results by hand.

The biggest problem is the risk of mistakes transferring the marks by hand. So pdas make sense.

[NielsenE]

The time effect grows exponentially with the couples taking part in the competition. Remember that most German competitions have a lot more contestants than the US. With 10-20 couples it takes about 10-15 minutes by hand and 5 with pda. When there are 50 contestants it takes 20-30 minutes by hand and 5-10 with pda. With 100 contestants it can take 45-60 minutes until you get the results by hand.


I'm well aware of the time required for large events, most of the events I scrutineer start with 100+ couples.

[avab]

I can only speak to the security with o2cm. The local network is restricted, so all the equipment has to be registered with the router. When a recognized pda is logged into the system it also has to manaully be accepted by the operator. This should be essentially unhackable, since the technician in charge would see someone trying to enter the system. Of course, someone will eventually take this as a challenge...

[chandra]

The ultimate advantage to using these PDA based systems is for the competitor - 'instant' results, as soon as the event ends. It is SO frustrating to remain in costume for an hour after your event, to wait for 'awards'... and for the audience, which by the time awards are givine, has forgotten who was dancing etc. Combined with the strange awards situation where the competitors have their backs to the audience and are announced by their numbers, it makes for a very insular-looking and excluding situation for people who paid to see the dancing competition.

As the comp owners 'get with the program', I'm eagerly awaiting the 'tipping point' when it will not be viable to have a comp WITHOUT instant results - the point at which people simply do not attend a comp if it doesn't have this.
Wow, you guys get results an hour after your comps? We always wait till either the end of the day (ussually like 12:00am if they do awards everyday) or all the weekend at once midday sunday... I guess thats cause we are soooo much smaller? (way less events I should say, seems like we have way more dancers in each event though)

[saludas]

Wow, you guys get results an hour after your comps? We always wait till either the end of the day (ussually like 12:00am if they do awards everyday) or all the weekend at once midday sunday... I guess thats cause we are soooo much smaller? (way less events I should say, seems like we have way more dancers in each event though)

It's at the end of the 'session' - which can be HOURS away, true. But I think we are talking about different things. I'm talking about PLACEMENTS and perhaps you are talking about judges' scores?

[waltzgirl]

I think some comps announce results at the end of the comp by choice. It creates a different atmosphere, more focused on enjoying the dancing for its own sake, less "how did I (he/she/they) do"?

[Laura]

It's at the end of the 'session' - which can be HOURS away, true. But I think we are talking about different things. I'm talking about PLACEMENTS and perhaps you are talking about judges' scores?
That's just poor planning on the part of the competition organizers, and has nothing to do with what scrutineering/registration system they choose to use. We used CompManager for years and would schedule awards breaks every five events or 45 minutes to an hour. Of course, getting them right away is even better in some people's minds, but waiting less than an hour doesn't seem too bad. Anything over an hour is quite annoying in my opinion. Especially if there are kids' events involved!

[Throwaway Overshare]

NDCA comps can't release the judges marks until the end of the competition.

[danceforfun]

In Germany we have all the marks and Information available in the internet:
http://www.topturnier.de/dmstd2006/
especially:
http://www.topturnier.de/dmstd2006/tabges.htm

The biggest advantage is, that the information is available instantly. It takes only a few minutes between the rounds and you have the result. And as soon as it´s official the final results are avialable. It´s only to the organizators to publish it fast. The marks are allowed after the finale only.

[Joe]

Hell, you could probably use cell phones at some point, and then the judges could literally just phone it in...

[avab]

As far as I know, the rules about releasing judge's marks to competitors is the same around the world. Marks are not released until after the results of that particular event are announced. Sometimes it is more efficient to print marks all at once, say at the end of a session. There is no rule in the NDCA stating that marks aren't released until the end of the entire comp.

Like the German system, marks from o2cm are uploadable to a website immediately (unless you are technologically challenged like me). Since an active internet connection could screw with the connection of the pdas, it is common to wait until the end of the entire day before uploading. Marks from Nationals were uploaded every evening within a half-hour of the end of the evening.

[ACtenDance]

I think it would be nice to see marks displayed as they're made. It would be a nice addition to o2cm if the marks could be displayed by an overhead projector as the judges enter them.... of course on a screen where the judges could not see what the other judges were marking but the audience could see it. I think it would add another dimension of enjoyment for an audience.

[Laura]

Well, maybe not as the judges are making them, but after all the judges have marked one dance and hit "send" then they could show them. That's called "open marking" and is used from time to time -- I've seen it done (with big cards rather than a computer screen) in some events at the San Francisco Open.

I'm not a fan of open marking as dancer, but it's fun as a spectator!

[Joe]

"Ten!"

[Laura]

Or, we could just break out an applause-o-meter :)