BAOC Score calculation methodology

Note: (BAOC 2008 Sprint season is used as example but methodology is not limited to it)

External links:

http://www.attackpoint.org/discussionthread.jsp/message_149323 discussion about GV calculation

http://www.geocities.com/nikdangerdvm/03methods.html - original write-up of the GV algorithm calculations.

Methodology was called by many people as cumbersome and I will try to explain here in my words. I will quote the Methodology where applicable and explain practical application details

Originally Adopted for USOF from Wyatt Riley's BAOC ranking rules, 1999. One more time adopted by S. Kretchetov starting from here http://www.geocities.com/nikdangerdvm/03methods.html 

Algorithm flow and definitions

There are four items here and they all depend on each other. Must be consumed as a whole dish. Don’t get disappointed until done reading.

NOTE: Algorithm makes no sence to implement if there is only one race. Two or more races must be present. 

1. Runner Local Power (RLP) = Race Gnarliness Value (GV) divided by Runners Time (RT) Example: 
   • Race GV is 5000
   • your time was 1:40:00 or 100 minutes. 
   • Your RLP for that race is 5000/100 = 50. 
    
2. Runner Global Power (RGP) = plain average of Runner Local Power (RLP) over all races
   Example
1. Your Race#1 RLP = 50
2. Your Race#2 RLP = 70
3. Your RGP = (50+70)/2 = 60
3. Runners Race Difficulty (RRD) = Runner Global Power (RGP) multiplied by Runners Time (RT)
   Example: 
   • If your RGP = 60
   • your RT = 80 minutes
   • your RRD would be 60*80=4800
    
4. Race Gnarliness Value (GV) = harmonic average (reciprocal of average of reciprocal values ) of Runners Race Difficulties (RRD) for all runners in current race.
   This value only has sence if there are more then one race.
   Example: 
• John has RGP = 100 and runs RACE in 30 min. (RRD = 100 * 30 = 3000)
• Mary has RGP = 60 and runs RACE in 100 min. (RRD = 60 * 100 = 6000)
• 1/ GV = ½ (1/(100*30) + 1/(60*100)) = 1/4000 
• GV = 4000

Some observations:

• If you are on average twice as fast as somebody, you should end up with about twice their score. 

Practical Implementation details.

• Input data for the algorithm are three columns: Person, Race and Time. No other information is needed 
• All valid finishes of the season must be used. Valid finishes are times (not OT, DNF, MSP, etc...). 
• Algorithm is recursive over items 1-4. Must be stopped after next iteration changes results by negligible amount. 
• Initial conditions are only Runner Local Power (RLP) which can be set to any positive nonzero number 
• Algorithm converges to the same point independently on initial conditions. For simplicity all runners can be given RLP = 100 at first step 
• Algorithm is linear relative to initial conditions. Meaning that if all initial RLP are increased by factor F, the resulting GV, and RGP will be increased by the same factor. 
• The only nontrivial output of the algorithm is set of GV, which is one number per race. The rest of the results can be immediately derived from GV and Runners Times (RT) 

 Analysis of the results:

Anywhere below when I say Score I mean RGP Runner Global Power. If some other scoring system is used I say USOF Score or BAOC Score. 

As was mentioned above, algorithm is linear and resulting values can float anywhere from 1 to 100 or even 100000. Only ratios will preserve: If you run twice faster then me, your score will be twice larger then mine. So the question of normalization arises.

• Original proposal of the USOF officials was to take top 3 runners and make average of these three runners scores equal to 100. This method has a serious drawback. 
• If we have one runner who is 10 % faster then two closest competitors, he will always have ~107 points (and the top 3 will always look like (107+ 97+96)/3 = 100
• If suddenly we got another fast runner of the same caliber, situation changes and we have now trio of (104 +104 + 92)/3 = 100. 
This is actual situation and happened all the time in last BAOC season.
Such fast and pointless recalibration moves everybody in the rest of the pack and makes actual value of a score not very valuable. 
• I propose the following way to normalize scores.
• Take all scores from people who ran at least once and normalize scores so that average score of all runners would be 50 ( Reasonable?)
• Technically, this can lead to all kind of unpredictable results. F.e if our fastest runner is 4 times faster than average member of a club, he would get score of 200 (50*4).
• But practically, I took all ~300 people who ran BAOC Sprints in the 2008 and plotted them on the Ranking curve presented below. 
• Curve is plotted in college-like % below style.
• Along X-axis is runner's score normalized as explained and along Y-axis is percentage of people who have score below.
• Chart shows that 
• Runner with score 30 will pass 15% of fellow runners if all running same track.
• Runner with score 80 is better then 95% of the runners

• The curve is almost ideally symmetrical around 50% point. It means that approximately half of the runners are faster then average and half of the runners are slower then average
• It happened so that 3 fastest runners in the list are approximately 2 times faster than average runner and therefore have scores around 100
• And it also happens that average score of this 3 runners is 99.
   

	Name	Score
1	Eddie Bergeron	106.9112
2	Erik Nystrom	95.5410
3	Sergey Velichko	95.1596

• Note: I did Nothing to make it happen. 

Benefits of 50% average score normalization

• Produces very stable scores if large number of people is taken into account. 
• Does not depend on top 3 runners performance.
• Allows effective comparison of the top Club runners relative to the Outside World ( Provided that at least one A-meet happened during the season)
• Example: 
  Our top qualified runners of last season Rex, Francois and Steve Greg currently have score of 78.0, 85.5 and 74 correspondingly (see table below), which puts them at top 10% (see chart above) of all runners. They would have scores 99, 108 and 93 if normalized using old method.
• How will runner perform on national Level? Here is the answer: Compare predicted and actual scores for the BAOC runners at the last year's Northstar Event.
  Note that Both Francois and Rex run close to their scores during the event where a lot of high level runners were present. (Ben apparently is still underestimated still in our Sprint Series) If normalized to 100 points for the best runner, our runners would never knew their real level. 
• In the meantime it also helps middle pack runners, such as Toby and Alex, realize their real USOF level (~60) instead of ~80 points as they would have otherwise.

• If applied to the rest of the courses (White -Blue), this sytem, allows to estimate GV of the courses more accurately while preparing for the A-meets
  (The "ideal" GV at the lower end of the USOF standard are: 1500 for Sprint, 4500 for Brown, 5000 for Green, 6000 for Red, and 7500 for Blue.

BAOC and/or USOF "practical" Score calculation

• USOF system proposes following
  
  • Runner Local Power (RLP) calculated according to the above described USOF procedure. Normalization doesn't matter
  • If you run 4 or fewer races, then all of your races are averaged together. If you run more than four races, you get to throw out one low score for every two additional races. 
  • For example, if you run 6 races, then you get to throw out (6-4)/2 = 1 score (your lowest) and the remaining (top 5) scores are averaged.
  • The scores are normalized (multiplied by a constant) so that the top three finishers average 100 points. 
    Note (SK): This normalization only takes into account Those Who Qualified. Fast runner with only one race would automatically be excluded.

• BAOC simplified system. (Proposed By Rex Winterbottom originally)
• Winner of each race gets 100 points. 
• Remaining runners get RT_winner/ RT_runner * 100
• Best 5 races count and are required for qualification (Including Course Setter Bonus)
• At the end of the season results are normalized so that top 3 got 100 points

• BAOC slightly advanced system. Used in 2008 (Combination of USOF and BAOC simplified, proposed by Shura Kretchetov)
• Runner Local Power (RLP) calculated according to the above described USOF procedure. Normalization doesn't matter
• Best 5 races count and are required for qualification (Including Course Setter Bonus)
• /At the end of the season results are normalized so that top 3 got 100 points

• BAOC/ advanced system. Proposed for 2009 BAOC Sprint (and other?) Series. Proposed for usage in USOF in part of score normalization (proposed by Shura Kretchetov)

• Runner Local Power (RLP) calculated according to the above described USOF procedure. 
• Scores normalized so that average score for all runners is 50
  
• Best 5 races count and are required for qualification (Including Course Setter Bonus)
• At the end of the season no normalization is done. Results show actual runners performance relative are normalized so that top 3 got 100 points

FAQ

• Current system normalizes 3 top runners to have 100 on average, which makes one or two of them always above 100.
• In the system I'm proposing,t he only reference point is average person, who will always have score 50. If somebody runs 3 times faster than average he will have score of 150.

APPENDIX