How to compute the Median?
I added computation of a median to computation of the mean
for comparing puzzle solving times while excluding the
over-importance of outlying values.
I want to be able to rank puzzle solving times in 3 ways:
How does puzzle P compare with puzzle Q ? This is used for
ranking puzzles within a book, where I put the easiest first.
How does solver X compare with solver Y ? This is used by
competitive players, who want to be first.
How does solver X's position in the group change over time?
Again, this is used by competitive players.
Of course, I do not want emphasis on timings and publication of them
to discourage any noncompetitive players.
I have changed the rules over the 20 plus years I have been running the website.
At the beginning, the puzzles had so many errors that I considered up to 5
wrong squares to constitute a solution. At some point, because of security breaches
and other factors I required membership for access to the daily classic.
Then I relented because I wanted the puzzles to get their widest audience,
so I created another membership class which allowed solving most puzzles a year
or more later than the original release date.
Who the players are, whether they can cheat by using hints, and when hints are
available to player X - I can't even remember now what all the rules were
at any given time.
As far as the puzzles themselves, double crostics on paper typically had longer
clues. Some puzzles had misspellings or other errors originally; I may have
corrected these after the puzzles were released online. Then are the timings valid
in any sense?
Here are Forum entries that are relevant:
2014 Mar 24 Competitive solving and What Hints?? SusanVanHemel
Wow, I never knew hints were available. How do I choose whether to see them? I really enjoy competitive timing, though I
am also often subject to interruptions by real life. I think Sue's March 9 suggestion is good.
2014 Mar 9 Strict vs Relaxed SueGleason
Here is a proposal which I think will work for everyone. Relaxed players will continue to get hints whenever they like.
Strict players will have access to hints one hour after beginning a puzzle. I am already showing the players' preference in
the results. Players will generally stick with their chosen strictness option; only I will be able to change it.
2014 Mar 8 Competitive timings Ron Stoner
Count me as a "relaxed solver." I usually solve on a laptop using a hunt-and-peck typing style while watching TV newscasts,
letting dogs in and out of doors, sipping coffee, etc.. Still, I find it interesting to see how quickly other solvers can
breeze through, so enjoy seeing the timings, especially for puzzles I have constructed.
2014 Mar 7 Should I keep the contest? SueGleason
How many people want me to keep the competitive timings?
2014 Mar 2 Puzzle for today - dixie SueGleason
There was a careless mistake in the version of the puzzle, dixies, as I originally posted it. It seems to be a typical
mistake that I have often made when I try to correct a puzzle. Something is wrong with my procedure and I often tend to
forget to redo the cross-referencing step after making changes. In any case, I have now posted both a corrected version
of "dixies", and Buff's original "dixie". If you have time, please solve both, and let's discuss the differences.
2014 Feb 27 Cumulative Results SueGleason
We have now published 1436 puzzles by Mel. In 2013 there were 172 by Mel, 80 byKWatkins, 58 by Buff Toneman, and
47 by Julian Schrock. I have just posted the cumulative results for 2013 solvers, along with earlier years. Congratulations
to the speedsters: Naomi Goldblum , followed by Daniel Levine, Sandi Macan, and Pete Volin, each with over 100 top 3
finishes.
2014 Feb 27 Strict vs Relaxed SueGleason
I am going to disentangle the results of the speed solvers and the relaxed players. I do not want to diminish the enjoyment
of the relaxed solvers by curtailing their use of hints. On the other hand, their results should not appear mixed in with
those of the speedsters. Let me know if you seem to be in the wrong category.
2014 Jan 26 Competitive timings SueGleason
At this point in time, we have been allowing any user of the HTML players to "cheat" when solving any puzzle. I suppose I
should change this, in order to restore the accuracy of the timed competition. I think I would like to change the contest
period for the acrostics to a single day rather than a week.
2013 Mar 7 Timing info SueGleason
People using the new HTML player to solve acrostics may experience strange results. I believe I have allowed the use of
hints even with new puzzles, making for very short reported times. In addition, often the times are not reported at all.
Please post here to let everyone know what you have experienced. What should I do in the future?
2006 Apr 10 Monthly Reports SueGleason
I have revised the cumulative results report from completed months. It is the last report on the timings page. You may set
the date to an early month if you like. There are several differences from the old report. First, they run ad hoc and much
more efficiently. The time of the median player is shown, rather than the average time of successful players. So for a very
difficult puzzle like juggernautc, the numbers are very different. 59 people tried the puzzle but only 35 finished within
the contest week. The median time was 101 minutes, although the average of the under 9999 players 52 minutes; an enormous
disparity. Also, previously the numbers in the main grid reflected the sequence of submission by date and time. Now the
numbers reflect speed (1 is fastest). There is an inconsistency in the handling of the short word puzzles, which were
classified as a separate variety puzzle for a while during 2003-2004, and therefore omitted from the new style reports.
2004 Nov 10 Read the footnotes - mean vs median SueGleason
Average times are based on puzzles completed under 9999 seconds (single session). Thus the outliers are excluded.
2004 Nov 10 Mean time SusanVanHemel
Not to quibble, but for a distribution like this, where some "outliers" have very long times (those who leave the puzzle
open and go out to a meeting, e.g.) the median would be a better indication of "average" solver's performance than the mean.
The median score is that score with equal numbers of scores above and below it, derived from rank order of times, not from
actual times. The mean is greatly skewed by a few high scores while the median is not.
2004 Nov 9 New timings page Joe Buccino
Thank you for the extra information. Now I'll be even more competitive. :-)
2004 Nov 5 Rank on Timings Report SueGleason
If my rank is 4 on puzzle X, 3 people beat me. Three people did puzzle X perfectly and faster than I did.
I am comparing my time to the field of perfect solvers. Whether or not my solution was perfect.
2004 Nov 4 Carolyn Roosevelt
Hope I'm not obfuscating, but has 'rank' changed its meaning? or does it still refer to better perfect solutions?
2004 Nov 3 Timings page Joe Buccino
I love it. Could we also see "best time" and "mean time" so that we can compare our time?
2004 Nov 3 New timings info SusanVanHemel
I like it! What are "winners" -- all finishers with acceptable error counts?
March 19,2021