1. Forum
  2. Usenet
  3. REC.GAMES.BACKGAMMON

  • Campaign for the fair evaluation of match performance

    From [email protected]@21:1/5 to All on Thu Jul 27 10:37:52 2023
    We all know the role that luck plays in the game.
    But, assuming that we can rely on the bots (a very reasonable
    simplifying assumption in my view), how should we assess
    match performance?
    The answer is actually blindingly obvious -- simply observe how
    much MWC (match-winning-chances) a player sacrifices.

    The reasons for introducing much more complex metrics involving
    EMG normalization are entirely bogus and spurious.
    So a player would get penalized far more for a mistake at DMP than
    for the same mistake in the first game. What on earth is wrong with
    that? At DMP, it's a more serious mistake. What if the reason for
    the DMP mistake is that the player is struggling with the greater tension? Shouldn't that lesser ability to cope with tension be penalized?
    Does anyone think tennis is unfair in that the value of a point changes depending on the stage of the match?
    I saw a player, who was new to competitive backgammon, run himself
    extremely low on time for the final DMP game of his match, so he had
    nowhere near enough time to play the final game competently, made
    horrible checker blunders and lost it.
    He had played the first game slowly. Suppose he'd played the first game super-fast to blunder in that game and been able to play the others at
    a reasonable speed. Surely that would have been more sensible (although admittedly still not optimal), and yet the EMG theory does not recognise this. Under this EMG normalization theory, running yourself low on time doesn't get penalized enough.

    Paul

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Timothy Chow@21:1/5 to [email protected] on Thu Jul 27 23:07:53 2023
    On 7/27/2023 1:37 PM, [email protected] wrote:
    We all know the role that luck plays in the game.
    But, assuming that we can rely on the bots (a very reasonable
    simplifying assumption in my view), how should we assess
    match performance?
    The answer is actually blindingly obvious -- simply observe how
    much MWC (match-winning-chances) a player sacrifices.

    Bob Koca and I argued this point on BGOnline many years ago,
    so I basically agree with you.

    We got considerable pushback. The counter-argument that I found
    to be most reasonable is that if what you're trying to do is to
    estimate *how skillful each player is*, then EMG greatly reduces
    variance while (probably) introducing only a small amount of bias.

    Here's an illustrative example. Suppose that an 11-point match
    at some stage reaches 3-away/3-away. Player A blunders horribly
    in the first game of the match, but plays very well in the
    3-away/3-away game. By MWC standards, Player A plays very well.
    But now imagine that, hypothetically, the 11-away/11-away game
    and the 3-away/3-away game were switched. You might infer
    that in this scenario, Player A would blunder horribly in the
    3-away/3-away game but play the 11-away/11-away game very well.
    (This is certainly not a watertight inference. There are
    psychological factors at play---maybe Player A's blunders are
    a function not of the decisions that arise but of "nerves" in
    the first game of a match---plus what is "correct" at 3-away/3-away
    is not necessarily "correct" at 11-away/11-away. Nevertheless,
    I think it's a reasonable supposition.) In this hypothetical
    scenario, Player's A's performance is much worse, because
    blunders at 3-away/3-away are more costly than at 11-away/11-away.
    We could say that Player A was "lucky" that the tough decisions
    arose in the first game rather than later on in the match.

    In the long run, we expect that this sort of "luck" will even out,
    but it could take a lot of matches for that to happen. All those
    EMG shenanigans are a form of (biased) variance reduction, to try
    to get a more accurate estimate of skill from less data.

    I think this argument is reasonable as far as it goes. If you're
    trying to award BMAB titles based on performance, then using EMG
    is probably going to yield more sensible results---the benefits
    of variance reduction probably outweigh the drawbacks of bias,
    especially because there probably isn't going to be that much
    *systematic* bias (EMG will probably be biased in one direction
    for some matches and biased in the other direction for other matches).

    On the other hand, this still doesn't give us a good argument for
    using EMG for assessing *who played better* in a particular match.
    For this, I agree that MWC is still a more logical choice than EMG. Unfortunately, I think that people have gotten so used to EMG that
    they will probably balk if MWC and EMG give different answers to
    the question of who played better.

    ---
    Tim Chow

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Simon Woodhead@21:1/5 to All on Fri Jul 28 03:46:10 2023
    I think it's been posted here before, but JB wrote a seminal paper
    on this some years ago (I forget when):

    https://www.fortuitouspress.com/emg

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Timothy Chow@21:1/5 to Simon Woodhead on Fri Jul 28 08:26:07 2023
    On 7/27/2023 11:46 PM, Simon Woodhead wrote:
    I think it's been posted here before, but JB wrote a seminal paper
    on this some years ago (I forget when):

    https://www.fortuitouspress.com/emg

    That is indeed an excellent article, but it doesn't directly address
    Paul's point. Jeremy Bagai is still trying to come up with something
    which assigns roughly the same weight to "the same error" at different
    match scores, but Paul is arguing against that.

    Offhand, I'm unable to find the best place where Bob Koca makes the
    argument for using MWC (or VRR, variance-reduced result) to determine
    who played better in a match, but here's one old BGOnline post where
    he makes some relevant comments.

    https://www.bgonline.org/forums/webbbs_config.pl?read=173017

    ---
    Tim Chow

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From [email protected]@21:1/5 to Timothy Chow on Fri Jul 28 05:47:28 2023
    On Friday, July 28, 2023 at 1:26:13 PM UTC+1, Timothy Chow wrote:
    On 7/27/2023 11:46 PM, Simon Woodhead wrote:
    I think it's been posted here before, but JB wrote a seminal paper
    on this some years ago (I forget when):

    https://www.fortuitouspress.com/emg
    That is indeed an excellent article, but it doesn't directly address
    Paul's point. Jeremy Bagai is still trying to come up with something
    which assigns roughly the same weight to "the same error" at different
    match scores, but Paul is arguing against that.

    Offhand, I'm unable to find the best place where Bob Koca makes the
    argument for using MWC (or VRR, variance-reduced result) to determine
    who played better in a match, but here's one old BGOnline post where
    he makes some relevant comments.

    https://www.bgonline.org/forums/webbbs_config.pl?read=173017

    Thanks for all commenters on this thread.
    I haven't read the bgonline forum yet or looked at the Bagai article.
    I'm sure I'll look at these soon, though.

    The logic of the EMG crowd would seem to make sense though, as well
    as my and Tim's view.
    In fact I might be close to a somewhat analogous situation this Sunday.

    This Sunday I (and some friends) will be at Roehampton, London, spectating
    a mens tennis event.
    The median world ranking (ATP) of the players at the event is likely to be approx
    1500. Given that the world contains approximately 4 billion male humans, this is a very
    high standard in my opinion (1500th out of 4 billion).

    Despite this type of ranking impressing me, tennis is not as well funded as soccer or basketball etc., and only a few of such players are likely to receive funding and sponsorship.
    I'm sure there will be scouting at this event, where those with access to funds (or, more likely, employed by them) will be taking notes on who is worth supporting
    financially, and who can benefit more by professional coaching etc.

    Now (finally) comes the relevant point that connects with this thread.
    As the scouts watch the matches, and look at players with impressive technique,
    power, speed and accuracy, will they focus on (or even notice) the scores at which the
    impressive rallies are played?
    I kind of doubt it. I think that an excellent serve in the first game of a match will
    impressive the scouts about (or almost) as much as the same serve at match point.
    If my theory is right, then these scouts are adopting the EMG logic.
    I don't have a huge amount of evidence for my view but there is some evidence in that
    the scoreboard is changed quite rarely --- only once every two games.

    Paul

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Timothy Chow@21:1/5 to [email protected] on Fri Jul 28 23:04:17 2023
    On 7/28/2023 8:47 AM, [email protected] wrote:
    Now (finally) comes the relevant point that connects with this thread.
    As the scouts watch the matches, and look at players with impressive technique,
    power, speed and accuracy, will they focus on (or even notice) the scores at which the
    impressive rallies are played?
    I kind of doubt it. I think that an excellent serve in the first game of a match will
    impressive the scouts about (or almost) as much as the same serve at match point.
    If my theory is right, then these scouts are adopting the EMG logic.

    As far as technical skill is concerned, my guess is the same as yours.

    But scouts also look for mental toughness. For that, they may put more
    weight on mental toughness at match point than on mental toughness at
    set point for the first set.

    ---
    Tim Chow

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From [email protected]@21:1/5 to Timothy Chow on Sat Jul 29 01:58:36 2023
    On Saturday, July 29, 2023 at 4:04:21 AM UTC+1, Timothy Chow wrote:
    On 7/28/2023 8:47 AM, [email protected] wrote:
    Now (finally) comes the relevant point that connects with this thread.
    As the scouts watch the matches, and look at players with impressive technique,
    power, speed and accuracy, will they focus on (or even notice) the scores at which the
    impressive rallies are played?
    I kind of doubt it. I think that an excellent serve in the first game of a match will
    impressive the scouts about (or almost) as much as the same serve at match point.
    If my theory is right, then these scouts are adopting the EMG logic.
    As far as technical skill is concerned, my guess is the same as yours.

    But scouts also look for mental toughness. For that, they may put more weight on mental toughness at match point than on mental toughness at
    set point for the first set.

    From the standpoint of an avid spectator at these events who has spoken to some of the
    principals but isn't very well informed:

    I think it's a two-track process:
    The players are observed in competitive play with the scouts ignoring the score and looking
    only at tennis ability.
    Combined with that, match scores are examined, and reports are available giving the result
    of every single point. This match log is scrutinised to see who won the "big points".

    Paul

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)