PDA

View Full Version : Competition Result forecasting


tunape
03-22-2007, 05:45 PM
okay, for a new test project, I'm trying a few statistical models for predicting the outcome of comps. You can see the forecast for the currently "active" comps(pulled from Eric's BRR site):
http://ballroom.mit.edu/Analytics/Forecast/

The model is very simple, and do not get upset if you are not the way it "would be".(it places us near the bottom). The results are from BRR only, so about 21 comps(not including Harvard Invitational).

but basically, it checks very couple in an event against every other couple to see the probability of beating that couple. To win the comp, you have to beat all couples. Hence, the probabilities are very small. Also, there's a lot of new couples(or partner switching) which the historically we do not know about, and assume a 50/50 chance of beating them. Also, it doesn't give weight for more recent comp results. All results are treated equal

Finally, I'd be happy to discuss more elaborate models with anyone. email mitbdt-www@mit.edu, or pm me.

tanya_the_dancer
03-22-2007, 06:14 PM
okay, for a new test project, I'm trying a few statistical models for predicting the outcome of comps. You can see the forecast for the currently "active" comps(pulled from Eric's BRR site):
http://ballroom.mit.edu/Analytics/Forecast/

The model is very simple, and do not get upset if you are not the way it "would be".(it places us near the bottom). The results are from BRR only, so about 21 comps(not including Harvard Invitational).

but basically, it checks very couple in an event against every other couple to see the probability of beating that couple. To win the comp, you have to beat all couples. Hence, the probabilities are very small. Also, there's a lot of new couples(or partner switching) which the historically we do not know about, and assume a 50/50 chance of beating them. Also, it doesn't give weight for more recent comp results. All results are treated equal

Finally, I'd be happy to discuss more elaborate models with anyone. email mitbdt-www@mit.edu, or pm me.

OK, shouldn't the sum of all probabilites to take 1st place in any given category be equal to 1 (I mean, someone WILL be 1st)?

tunape
03-22-2007, 10:12 PM
OK, shouldn't the sum of all probabilites to take 1st place in any given category be equal to 1 (I mean, someone WILL be 1st)?

well.. not quite... each couple has a distribution of winning various places... so for example, couple A has a 10% winning 1st, 25% winning 2nd, and soforth. right now, it only post the probability of each couple winning first, not who will win first... if that make sense.

it's not necessarily a good model, but I'm not a big fan of ranking systems like the one used on dancesport info or other sights since they rely on somewhat arbitrary multipliers.

wooh
03-22-2007, 10:18 PM
:p Now wondering if I'll ever see my husband getting as excited about a fantasy dancesport team as he does about his fantasy baseball teams.:p

Katarzyna
03-22-2007, 10:30 PM
I think the system doesn't match couples with slight spelling variations of the name :)

tunape
03-22-2007, 10:32 PM
:p Now wondering if I'll ever see my husband getting as excited about a fantasy dancesport team as he does about his fantasy baseball teams.:p

hahah! Fantasy team match! 4 couples, 4 styles. Friends are saying I'm trying to start a betting pool on comp results. :)

tunape
03-22-2007, 10:34 PM
I think the system doesn't match couples with slight spelling variations of the name :)

yeah, I'm working on that part. Some names badly parsed names. programmed it in 2 days, there's still a few bugs... but I'm hoping it will be a start, and have people join in building a better system.

tunape
03-22-2007, 10:39 PM
hmm... the system seems to penalize those who compete, but don't do well. so... if you're not going to win, don't compete at all(and get 1/2 chance to beat any other couple). hmm...

Katarzyna
03-22-2007, 10:44 PM
this sounds like a fun idea, why did you decide to build it?

9dncr
03-22-2007, 10:50 PM
Man, this seems like a fun and unique idea but a whole lot of thinking and time, I wish I could play on the computer at work! :)

tunape
03-22-2007, 10:51 PM
this sounds like a fun idea, why did you decide to build it?

to figure out the probability of outcomes at comps. It's not quite a ranking system, but even a ranking system which has more data for the collegiate scene for various levels would be interesting and useful.

there are standard estimation models like:
quality of couple = (coefficient) * (factors, eg. previous results, number of couples beat, size of event, etc...)

perhaps bayesian method would work better?

hbetx9
03-22-2007, 10:52 PM
I think this will be a very nice system after some revision and road testing, thank you for starting such an ambitious project. Note that the result tracker is actually incorrect in its attributions ( I can site personal examples, however I'm sure that there are many others, PM me if you want examples ), so those viewing should keep this in mind.

Katarzyna
03-22-2007, 10:54 PM
I see, this seem like a good idea, it will be probably easier to estimate with the higher level couples as their results tend not to vary quite as much as results in lower levels.. you also need quite a bit more resuts for this to be more accurate.. I think ranking systems such as point system on dancesport info give a good idea about a result in top levels..

bayesian method = No idea, never heard of it?

tunape
03-22-2007, 10:58 PM
I think this will be a very nice system after some revision and road testing, thank you for starting such an ambitious project. Note that the result tracker is actually incorrect in its attributions ( I can site personal examples, however I'm sure that there are many others, PM me if you want examples ), so those viewing should keep this in mind.

the system pulls results from ballroomregistrar(BRR). whatever BRR reports, it uses. However, I'd be happy look into any inaccuracies.

callen
03-22-2007, 11:03 PM
I think ranking systems such as point system on dancesport info give a good idea about a result in top levels..

Yes, the dancesport info site's rankings are pretty good indicators. You should be able to find many of the collegiate couples listed there, too, since the site incorporates all competitors in every event they find. The IDSF's point system is pretty poor except for the top few couples; it's based too heavily upon simply entering IDSF events.

Christopher

Katarzyna
03-22-2007, 11:07 PM
but dancesportinfo only includes the top levels, it doesn't track syllabus events.. it would be interesting to have a comprehensive system like that.. I agree, IDSF doesn't really help you predict much beyound the top few couples..

hbetx9
03-22-2007, 11:14 PM
but dancesportinfo only includes the top levels, it doesn't track syllabus events.. it would be interesting to have a comprehensive system like that.. I agree, IDSF doesn't really help you predict much beyound the top few couples..

You'd be surprised they don't track well however dancesportinfo did manage to capture the last BU competition, if you do a search with the key words "Boston University" some of the open events as well as the newbie events were tracked. This of course doesn't change your point, but I found it odd

tunape
03-22-2007, 11:42 PM
Yes, the dancesport info site's rankings are pretty good indicators. You should be able to find many of the collegiate couples listed there, too, since the site incorporates all competitors in every event they find. The IDSF's point system is pretty poor except for the top few couples; it's based too heavily upon simply entering IDSF events.

Christopher

dancesportinfo does not include lower syllabus levels, and they pick up scattered results from collegiate comps. Despite all this, I do hope to grab results from dancesportinfo to incorporate into the system eventually.

The ultimate goal for the project is to start some sort of "open source" project to develop a fair system, which anyone can view, and make improvements.

The trouble with dancesportinfo, also, is that their system is "closed." No one knows exactly how it is calculated, or at least it's not public information. There's a high level description of it here: http://dancesportinfo.net/Desc_Rating.aspx

Towards the bottom, they talk about using the ELO(for chess) system, and giving weights to events based on:
1) if strong couples compete it in(average prior points)
2) there are more couples
3) there are more couples from other countries

these are given arbitrary weights to calculate the points.

I'm hesitant about these systems because they seem to be less statistically rigorous(unless they publish more specs on how the weights are calculated, etc...).

I'm hoping to develop a more rigorous system which does not need to make arbitrary assumptions. Or if anything, at least build an "open" system where the methodology is fully published.

Finally, their system still does not provide probabilities of winning, or placing at a certain level at an event.

tunape
03-22-2007, 11:43 PM
You'd be surprised they don't track well however dancesportinfo did manage to capture the last BU competition, if you do a search with the key words "Boston University" some of the open events as well as the newbie events were tracked. This of course doesn't change your point, but I found it odd

it's odd that they caught champ standard, champ latin, and the newcomer events, but not everything else in between.

EDIT: our newcomers have more points than we do on their system!(because amateurs start off at 1500, and our competition results have shown that we are not quite "1500" material yet after many years of dancing)

Musique
03-23-2007, 02:29 AM
to figure out the probability of outcomes at comps. It's not quite a ranking system, but even a ranking system which has more data for the collegiate scene for various levels would be interesting and useful.

there are standard estimation models like:
quality of couple = (coefficient) * (factors, eg. previous results, number of couples beat, size of event, etc...)

perhaps bayesian method would work better?

Bayesian method definitely will be helpful.

Here's a nice paper on Bayesian ranking method
http://www.math.u-psud.fr/~lavielle/monolix/documents/main-BA-ranking0901.pdf

Given the frequent partner swap in collegiate circus, you might want to consider rank individuals too. If you want to develop a ranking system.

But how do you define the "probability of outcome" ? Placed first, 2nd? or prob of beating other couples? Say, if couple A has a 60% chance beating couple B, B has 60% chance beating C, C has a 60% chance beating A...

Let's discuss in MIT, if we get the chance(:

RIdancer82
03-23-2007, 02:50 AM
oy... almost 4am.... I should have known better to open a thread started by you, tunape ;-).... especially one with this title, lol. will read it once I wake up when I will be able to actually understand and absorb it all

Joe
03-23-2007, 06:32 AM
LOL, I am ranked at the bottom, below a metric ass-ton of people whose butts I'd kick. At least I'm in good company...

hbetx9
03-23-2007, 10:05 AM
LOL, I am ranked at the bottom, below a metric ass-ton of people whose butts I'd kick. At least I'm in good company...

The result tracker does not contain perfect information, specifically for a lot of couples, it has no comparison results and so would rank that person 50/50 against anyone else in the field. ( that is if I'm understanding tunape's algorithm correctly ), however over time as more results get loaded into the system the problem should correct itself. This is the issue with a 'new' ranking system is that everyone pretty much starts out as 'new' and so it takes a bit of time for things to normalize to match our intuition.

A Bayesian model I think will be necessary for updating, however this is a good first go :)

Musique
03-23-2007, 10:22 AM
:D The ultimate geeks' thread!

tunape
03-23-2007, 10:31 AM
But how do you define the "probability of outcome" ? Placed first, 2nd? or prob of beating other couples? Say, if couple A has a 60% chance beating couple B, B has 60% chance beating C, C has a 60% chance beating A...


I was going to post a simple example, but you beat me to it. so, say we have the following historical results for 3 couples(sorry, I had worked it out using my numbers, so just copying from my notes):

A beats B 60% of the time
A beats C 80% of the time
B beats C 60% of the time

For A to take first, it's the joint probability of beating both B and C = 0.6*0.8 = 0.48.

For A to take second, it's either:
1) A beats C, and loses to B = 0.8*0.4 = 0.32, or
2) A beats B, but loses to C = 0.6*0.2 = 0.12
for a total probability 0.44

and so forth. The trouble is that two couples may have equal probability of having the same place(s), and it's not consistent overall.

in terms of computation speed, it's easy to calculate 1st place and last place(order N^2), but much harder for other places.

tendancer
03-23-2007, 10:32 AM
to figure out the probability of outcomes at comps. It's not quite a ranking system, but even a ranking system which has more data for the collegiate scene for various levels would be interesting and useful.

there are standard estimation models like:
quality of couple = (coefficient) * (factors, eg. previous results, number of couples beat, size of event, etc...)

perhaps bayesian method would work better?


Couple of thoughts...

Can you normalize the #s to 1? trivial I'm sure...

How much effort would be involved to add the judging panel as a factor. e.g. I see a judge that historically gives me 5s/6s when she sees me in the final, that factor in the probability of results too...and the impact of that varies of course depending on the size of the panel. The effort there I'm sure is nontrivial (and probably involve lots of assumptions) but would make this tool quite interesting and does something I'm sure few existing tools already perform.

Of course, that's a whole diff can of worms. Then maybe going forward one has to factor in

* who takes lessons with the judging panel
* which couple is better looking
* whose coach has more pull
* whose coach are friends with the judges
* who happen to be the judges kids
* who are friends with the judges' kids

and if used on pro heats in ndca comps

* who brings a ton of pro/am students to the comp
* what did XXX said about them in Dance Beat

j/k...maybe...:)

Ithink
03-23-2007, 10:38 AM
My brain hurts!

star_gazer
03-23-2007, 10:45 AM
If you are using results from dancesportinfo, I've noticed that sometimes the results are incomplete. There are a couple comps for which I noticed that only final was posted when there were over four rounds. And sometimes couples that don't show are included in the first round. And when US couples go overseas...there are some serious changes in the rankings.

tunape
03-23-2007, 12:11 PM
I've added an "about" page to summarize the questions and discussion here. Thanks everyone for the comments and feedback(esp. musique[check your pm] for the pointer).

so far, the intermediate and advance levels at MIT seem to match my personal qualitative placements. this probably make sense because there's a lot of data for these couples on BRR, and they don't tend to switch partners often(except for the Brown team)

tunape
03-23-2007, 12:19 PM
oy... almost 4am.... I should have known better to open a thread started by you, tunape ;-).... especially one with this title, lol. will read it once I wake up when I will be able to actually understand and absorb it all

hehhe... don't read dance-forums at 4am! go to bed. :)

hbetx9
03-23-2007, 01:36 PM
I've added an "about" page to summarize the questions and discussion here. Thanks everyone for the comments and feedback(esp. musique[check your pm] for the pointer).

so far, the intermediate and advance levels at MIT seem to match my personal qualitative placements. this probably make sense because there's a lot of data for these couples on BRR, and they don't tend to switch partners often(except for the Brown team)

What styles are you checking against and what personal qualitative placements are you talking about?

tunape
03-23-2007, 01:43 PM
What styles are you checking against and what personal qualitative placements are you talking about?

just my personal feeling and intuition. it's not necessarily relevant to the discussion, but perhaps a verification of the system. Of course, we will only know from the actual outcome.

hbetx9
03-23-2007, 01:47 PM
I see, I really can't say enough about how useful this system will be once accurate, which would require making the result tracker on BRR more accurate, however Eric has quite a lot on his hands. Do you welcome error reporting at this stage or is it still in a prototype phase?

hbetx9
03-23-2007, 01:50 PM
To test accuracy you may want to post the comparison of the prediction with the results from HC and MIT

tunape
03-23-2007, 02:01 PM
Do you welcome error reporting at this stage or is it still in a prototype phase?

I certainly welcome all errors and feedback.

The system is classified into two main parts:
1) importing tools(for results and registration)
2) forecasting

#1 can definitely always be improved since my code is not doing a perfect job of parsing results from BRR, and matching nicknames with each other. Eventually we can pull in results from other sites too for more data.

#2 is the more theoretical/mathematical part which is where I want to investigate further.

tunape
03-23-2007, 02:05 PM
To test accuracy you may want to post the comparison of the prediction with the results from HC and MIT

I think the Bayesian methods may allow for updating and learning - but I haven't read the paper that Musique posted yet.

There's also a few AI learning algorithms where you train the system to automagically make better and better predictions.

If anything, I'd say there are too many algorithms to try, and I would invite contributors to try them now that we have the data.

Katarzyna
03-23-2007, 02:36 PM
1) importing tools(for results and registration)

#1 can definitely always be improved since my code is not doing a perfect job of parsing results from BRR, and matching nicknames with each other. Eventually we can pull in results from other sites too for more data.
.
No kidding, looks like we have 0 results in BRR..

good luck with this project thogh..

tendancer
03-23-2007, 03:22 PM
Something's bugging me:

A beats B 60% of the time
A beats C 80% of the time
B beats C 60% of the time


A 1st = .48
For A to take second, it's either:
1) A beats C, and loses to B = 0.8*0.4 = 0.32, or
2) A beats B, but loses to C = 0.6*0.2 = 0.12
for a total probability 0.44
A 3rd = .4 * .2 = .08
total = 1

B 1st = .6 * .4 = .24
B 2nd = beat a lose c = .4 .4 = .16
beat c lose a = .6 .6 = .36
total probability = .52
B 3rd = .4 * .6 = .24
total = 1

c 1st = .2 * .4 = .08
c 2nd = beat a lose b = .2 * .6 = .12
beat b lose a = .4 * .8 = .32
total = .44
c 3rd = .6 * .8 = .48
total = 1

These are all well and good, but...


a b or c get 1st = .48 + .24 + .08 = .8
a b or c get 2nd = .44 + .52 + .44 = 1.4
a b or c get 3rd = .8

Even if you normalize the sum to 1, that still doesn't make sense, in a working model all 3 values should be equal no? or I am missing something

tunape
03-23-2007, 03:31 PM
Something's bugging me:
These are all well and good, but...


a b or c get 1st = .48 + .24 + .08 = .8
a b or c get 2nd = .44 + .52 + .44 = 1.4
a b or c get 3rd = .8

Even if you normalize the sum to 1, that still doesn't make sense, in a working model all 3 values should be equal no? or I am missing something

yes, this is correct. It's definitely a problem with the model. It currently says *given a couple*, what is their probability of placing 1, 2, and 3(regardless of other's outcomes). There's some double accounting in that for A to take second, B has to take first, or C has to take first. But when you calculate the probabilities for B, it doesn't take A's placement into consideration(maybe it should!). If that makes sense...

discussing it is definitely helping to iron out the problems with this. This algorithm is definitely not typical to ranking/placement algorithms out there.(maybe for these reasons).

(more later, have to run to practice)

tendancer
03-23-2007, 03:36 PM
yes, this is correct. It's definitely a problem with the model. It currently says *given a couple*, what is their probability of placing 1, 2, and 3(regardless of other's outcomes). There's some double accounting in that for A to take second, B has to take first, or C has to take first. But when you calculate the probabilities for B, it doesn't take A's placement into consideration(maybe it should!). If that makes sense...

discussing it is definitely helping to iron out the problems with this. This algorithm is definitely not typical to ranking/placement algorithms out there.(maybe for these reasons).

(more later, have to run to practice)

Actually now that I think about it I *think* I know why: the question kinda answered itself...all the %s for 1st place just needed to be divided by 0.8. (and 2nd place by 1.4 etc...then the cross comparisons should match).

So actually, the odds of e.g. A winning is not 48%, but 0.48 / .8 = 60%. B is .24/.8 = 30%, and C = 0.08 / .8 = 10%...that ought to work. Seems counterintuitive though...guess when I have free time I can write up a quick program to see what the #s turn out empirically based on these odds.

tendancer
03-23-2007, 04:07 PM
eureka...only after I wrote a script to test this did I notice the loophole

I rolled 100k times, and ended up with:
A won: 48244
B won: 24013
C won: 8074

so question is, where hell did the other ~20k rolls go?

those are cases where e.g.
A beat B
B beat C
C beats A

which is an impossible scenario, but allowed by this numeric model.

So, that model needs fix.

(in case you are interested, when I force the script to re-roll when an impossible scenario surfaced, results were ~60-30-10)
A won: 60009
B won: 29939
C won: 10052

Now I can go back to doing real work...

tunape
03-23-2007, 11:11 PM
eureka...only after I wrote a script to test this did I notice the loophole

I rolled 100k times, and ended up with:
A won: 48244
B won: 24013
C won: 8074

so question is, where hell did the other ~20k rolls go?

those are cases where e.g.
A beat B
B beat C
C beats A

which is an impossible scenario, but allowed by this numeric model.

So, that model needs fix.

(in case you are interested, when I force the script to re-roll when an impossible scenario surfaced, results were ~60-30-10)
A won: 60009
B won: 29939
C won: 10052

Now I can go back to doing real work...

Thanks for the simulation. But yes, there's transitivity issues in the ranking system as you pointed it, making it not "consistent"(impossible placements, and/or duplicate placements). The system doesn't assume any type of transitivity. Most other placements systems assume "independence of irrevelant alternatives"(IIA) which still has this problem. I think this is why many use a ranking system(like on dancesportinfo) instead.

Bayesian methods and/or Markov processes may do solve the IIA and transitivity problems.... but haven't worked it out yet.

Musique
03-24-2007, 12:56 AM
I think this is the problem. A beating B is not independent of A beating C.
Let's formulate this problem in terms of distributions.

Random variable A B C has a joint distribution, and they represents "the level of dancing" or three couples.

The chance that A couple beats B couple is the probability that random variable A is greater than random variable B, namely Pr(A>B).

The chance that A gets the first place is
Pr(A>B, AND, A>C) != Pr(A>B) * Pr(A>C),
because the two events are not independent.

The point is that this situation requires an understanding of the full joint distribution, only knowing the margines is not sufficient.


Thanks for the simulation. But yes, there's transitivity issues in the ranking system as you pointed it, making it not "consistent"(impossible placements, and/or duplicate placements). The system doesn't assume any type of transitivity. Most other placements systems assume "independence of irrevelant alternatives"(IIA) which still has this problem. I think this is why many use a ranking system(like on dancesportinfo) instead.

Bayesian methods and/or Markov processes may do solve the IIA and transitivity problems.... but haven't worked it out yet.

callen
03-25-2007, 04:25 PM
Let me offer a hand. First, I'll explain the "counterintuitive" part, so hopefully it won't be so counterintuitive. Second, I'll explain why what's showing up in the results here still doesn't match the input parameters as given here. Third, I'll explain how you want to adjust things to fix this problem. Fourth, I'll give suggestions on how to proceed.

Note: for all of this I will use the recent example of A beating B 60% of the time, B beating C 60% of the time, and A beating C 80% of the time.

1:

The way you are running the probabilities is as independent contests, treating those probabilities as the chances of winning the independent contests. Think of it like drawing cards or flipping coins against each other, but with the odds not being 50-50. First you have A compete against B, then you have B compete against C, then you have A compete against C. That's how you manage to have the possibility of A>B>C>A.

This is OK. All you have to do is remove the impossible results and renormalize, as you guys figured out. (Though that 1.4 isn't right; they're all 0.8.) If you simply multiply all the probabilities, you'll get:

A>B>C: 288/1000
A>C>B: 192/1000
B>A>C: 192/1000
B>C>A: 48/1000
C>A>B: 48/1000
C>B>A: 32/1000
A>B>C>A (impossible): 72/1000
B>A>C>B (impossible): 128/1000

So there are 200/1000 impossible results. Multiply all results above that by 1000/800, and you have renormalized things:

A>B>C: 36%
A>C>B: 24%
B>A>C: 24%
B>C>A: 6%
C>A>B: 6%
C>B>A: 4%

2:

Let's look a little closer at the input parameters and the results. A beats B 60% of the time. That means the sum of the probabilities of A>B>C, A>C>B, and C>A>B is 60%. But when we add them we get 36%+24%+6%=66%. Oops. That's because the original numbers were used as odds of winning individual contests, and some of those contests have been removed. To do it properly, first I'll define some variables for the different probabilities:

A>B>C: P1
A>C>B: P2
B>A>C: P3
B>C>A: P4
C>A>B: P5
C>B>A: P6

The percentages give us the following system of equations:

P1+P2+P5=0.6
P1+P3+P4=0.6
P1+P2+P3=0.8
P1+P2+P3+P4+P5+P6=1

Hopefully you already see the problem. This is an underdetermined system. If you don't believe me, check these two results: (30%, 20%, 30%, 0%, 10%, 10%) and (25%, 25%, 30%, 5%, 10%, 5%). But at least now we have results that match the input parameters.

3:

How do we fix things? The easiest way is to avoid the underdetermined system. Rename the input parameters as the probabilities of winning in direct contest instead of the probabilities of overall victory. This allows you to use the method you're using.

Now that we've done this, we can go back to the renormalization. Since you're using a computer, it's not too hard to renormalize. You'll have to do it at every level over two couples, though. Do the following. Find the probability that A>(B,C,D,E,F,...) as if it's independent. Do this for each of teh others winning, too: B>(A,C,D,E,F,...), etc. Add all those and renormalize. Now do the same for the remaining contestants within each field. In the end you'll be able to simply multiply all the renormalized probabilities together normally.

4:

First, what you really want is the expected outcome for different couples. I would suggest determining an average and a standard deviation or something similar, using the probabilities as the weights for the positions.

Second, figuring out how likely a couple is to beat another couple is not so trivial. If you only look at final results, you can miss how close some couples may have actually been. For example, in the final of Tri-State, none of the three of use who were 4th-6th got a majority of 4ths in any dance as I recall. However, almost always all three of us got a majority of 5ths. A mark here or there could have drastically altered things. You wouldn't see that from looking at 4, 5, and 6, nor would you from looking at (4,4,4,4,4), (6,5,5,6,5), and (5,6,6,5,6). If possible, a better way would be to incorporate individual marks, dances, and events. Perhaps placing even weight on all three categories would be useful? This is the real killer in the project; that's a lot of data.

Christopher

tunape
03-25-2007, 10:49 PM
Second, figuring out how likely a couple is to beat another couple is not so trivial. If you only look at final results, you can miss how close some couples may have actually been. For example, in the final of Tri-State, none of the three of use who were 4th-6th got a majority of 4ths in any dance as I recall. However, almost always all three of us got a majority of 5ths. A mark here or there could have drastically altered things. You wouldn't see that from looking at 4, 5, and 6, nor would you from looking at (4,4,4,4,4), (6,5,5,6,5), and (5,6,6,5,6). If possible, a better way would be to incorporate individual marks, dances, and events. Perhaps placing even weight on all three categories would be useful? This is the real killer in the project; that's a lot of data.

Christopher

Hi Christopher,

Thanks for the excellent comments! I am fully aware of the impossible situations(and erred probabilities) that you pointed out in #1 and #2. There's some great feedback and suggestions(including yours) to address the issues which I will definitely look into.

You also made a good point which had not been suggested yet - to look at the individual dances to determine "closeness" of placement. We certainly have all the data on marks by round, by events, and by which judges for all the events on BRR! However, I will reserve the right to censor any data which goes down to judges' biases since it would only seem to cause political battles rather than be constructive. That said, it may still be useful to see the percentage or distribution of judges' marks for the given dances!

Definitely a lot to think about! thanks!

Katarzyna
04-05-2007, 04:42 PM
So curious how is this project coming along?

Kitty
04-05-2007, 05:34 PM
So curious how is this project coming along?

you mean, you want to know from user's point of view and without technicalities?:-)

tunape
04-05-2007, 08:26 PM
So curious how is this project coming along?

I got a lot of really great feedback from df members, and others. We have a number of interesting directions and technical things to try, but on my end, I have to study for exams for the next 2 months or so... so, it's going to be on halt until then. will pick up during the summer.

Katarzyna
04-06-2007, 12:16 AM
you mean, you want to know from user's point of view and without technicalities?:-)Absolutely :)

Thank you Tunape