More Information Increasing Accuracy?


More information increasing accuracy in Fantasy Football or other Fantasy Sports**?

(** this article is contained within Bush and Cherney’s Textbook on Fantasy Football Drafting – due by June 1, 2018, on Kindle)

By Dr. John Bush

Hall et al published in this article a discussion on this concept. The illusion of knowledge: When more information reduces accuracy and increases confidence. (2007) Crystal C. Hall, Lynn Ariss, Alexander Todorov, Organizational Behavior and Human Decision Processes 103: 277–290.

Their abstract lays out using NBA predictions and sports fans. They set up the experiments with Fans and they were given some stats on the teams. One group had no team names and the other group had the team names. The Team names gave a false confidence to winning picks. What happens was the group relied less on the stats because they were “familiar” with the team and “knew” that team. More familiar teams caused a failure in “using stats etc.” objective data to make bets and picks. The group that had the team names had a “false” belief in the strength of their confidence.

 In Fantasy Football/Sports, the issue is because we “know” the “worth” of a player because of whatever (Fill in the reason), do we then ignore the objective data?

Drew Brees is a first ballot hall of Famer, so he is always the best. Losing Jimmy Graham and Kenny Stills did not matter back in the day.  Our confidence is high, and we focus elsewhere in our draft. If we have 20 pieces of information, then we am knowledgeable about that situation? Cueing specific knowledge can increase the sense of expertise!

Paul Slovic, “Behavioral Problems of Adhering to a Decision Policy,” unpublished manuscript, 1973.

The quotes and figure above illustrate the issue.  Quoting “Eight experienced horse-race handicappers were shown a list of 88 variables. Each handicapper was given the data in increments of the 5, 10, 20 and 40 variables he had judged to be most useful. They predicted each race four times–once with each of the four different levels of information. For each prediction, each handicapper assigned a value from 0 to 100 percent to indicate the degree of confidence in the accuracy of his prediction.

 The average accuracy of predictions remained the same regardless of how much information the handicappers had available.

Three of the handicappers showed less accuracy as the amount of information increased, two improved their accuracy, and three were unchanged.

All, however, expressed steadily increasing confidence in their judgments as more information was received.

This relationship between amounts of information, the accuracy of the handicappers’ prediction of the first-place winners, and the handicappers’ confidence in their predictions are shown in Figure 5.

Info Fig


This discussion was found in Chapter 5 Psychology of Intelligence Analysis (1999) by Richards J. Heuer, Jr. “Do You Really Need More Information? “

  • Point 1. If you have an artificial sense of confidence via a lot of data then mistakes will happen. System 1 Errors
  • Point 2. More information past a small amount will not add to your accuracy in drafts but will elevate your sense of confidence. System 1 Errors. See Coherence discussion below. See Point 1.
  • Point 3. The time spent searching for too much information is wasted except for injuries or some black swan event. Looking at 20 websites for your rankings is wasted. Fake Pigskin is all you need :)
  • Point 4. So what is the amount of information you (YOU) need for winning?  See the amount of information discussed below.  I suggest you go back and read multiple hypotheses for some insight.
  • Point 4A. Keep records to record your journey. So, go back and ask yourself the whys for your mock picks. This is a way to see what information was driving your decisions.
  • Points 4B. What are successful players doing? In your leagues, the top group may be doing things that you do not. Why are they making their decisions?
  • Point 4C. Decide on a flowchart for each of your real drafts and save for next year to look back at. Be reflective in short.
  • Point 5. There are three types of forecasting techniques:
  • Point 5A. Subjective or qualitative where you rely on expert judgments
  • Point 5B. Time-Series Projections where you use quantifiable observations over time. – Record Keeping!
  • Point 5C. Causal Models where you emphasize causal/correlational relationships. The principal emphasis in forecasting is looking for patterns and fluctuations over time. Little tidbits.
  • Here is an example. “When the 1QB was hurt and the backup QB in place sometimes the WR 3 4 or 5 is elevated because they have spent time with the backup more than the QB1 in that team. Before the season determine if such relationships exist.” Who is on the practice team?
  • Point 6. Three things to do.
  • Point 6A. List all your assumptions about your player, team, defense strength etc.
  • Point 6B. Reverse each assumption. What is its opposite? Write down and consider your “Red-Cell Thinking/Devil’s Advocacy”. 
  • Point 6C. Ask yourself how to deal with each reversal assumption in a draft or waiver wire etc.

If you work on these things you can grade yourself in the offseason and do reflections on your journey. We are used to being reflective given our careers as a University Professors. We are used to delayed gratification for success. It is maybe easier for us.

If you hate to write, then use a voice recorder and document your drafts if you can. There is software that records your screen in a video/audio. Use that while drafting!

Imagine finding out you are really being sucky at picking late round sleepers but great at WR/Pitchers etc selections. That gives you an area to focus on.

Analysis of data suggests that more information does not increase our understanding unless the information increases our coherence of the data set.

Any Fantasy sports data you use must have meaningful relationships. Those relationships have to have some logical foundation. The increasing amounts of opinion-based information do not increase our coherence looking at all of the available sources.  That said many of these opinions from pundits are based on their view of the data.

In science, the worst sin can be summarized as “An Idea in Search of Data”.  In Fantasy Sports we must gather the data following a question first and only then have thoughts based on that journey!  

 Thus, if we cross the level of data that we can “handle” then it is suggested you prune your data and eliminate the less coherent parts!  This process will require effort and practice and it will use your slower System 2 thinking. Your fast and frugal System 1 will not like pruning.  However, record keeping can help establish your baseline of coherence and is the beginning of reference class data from future activities.

1) What is the minimum amount of information needed?

“Experts overestimate the importance of factors that have only a minor impact on their judgment and underestimate the extent to which their decisions are based on a few major variables.

In short, people’s mental models are simpler than they think, and the analyst is typically unaware not only of which variables should have the greatest influence, but also which variables actually are having the greatest influence.”

This discussion was found in Chapter 5 Psychology of Intelligence Analysis (1999) by Richards J. Heuer, Jr. “Do You Really Need More Information? “

I believe the above is the key here. Many pundits have opinions that fit their mental models in Fantasy Football but they on average have no real clue as to the hierarchy worth of their variables. Therefore, experts and non-experts spend more time than needed to build their model!

If one looks back at the extreme events in Fantasy Football/Sports, you will find these events have produced a sensational based remembrance that your System 1 recognizes as that data was important last time, so it must be important this time. Listen to the pundits discuss when their point was right and led to their success. They have fallen into the trap

I have ideas to answer this issue for myself. I will most likely deal with that over 2018 offseason. However, knowledge of the unknown is better for you than the dangers of the unknown unknowns!

2) What is the nature of that information for your mental model?

This is my Redraft Fantasy Football Example. Redo for Baseball etc.

I suggest the second question is can be more personal and you should keep a diary and see what works for you. You should break down all the information. This process never stops FYI.

Things I think I need but not in hierarchical order

  1. A) SOS
  1. B) Draft pattern of the league
  1. C) Value of Player

1) Team Usage

2) Strength of position

3) Attempts and Targets

4) New situations

5) Nature of Players around them

  1. D) My Projections both regular and injury
  1. E) Timing of Positional Picks (Draft flow)
  1. F) Playoff value
  1. G) ADP of players (see wisdom of the crowd’s concept)

How should Fantasy Football/ other Sports Analysis work?

A more accurate analogy for describing how Fantasy Sports Analyst should work is a medical diagnosis.

The doctor observes indicators (symptoms) (data collection) of what is happening, uses his or her specialized knowledge of how the body works to develop hypotheses (Forming a Hypothesis) that might explain these observations, conducts tests (Science-Based Testing) to collect additional information to evaluate the hypotheses, then makes a diagnosis. This medical analogy focuses attention needed by the Fantasy Football Analyst on their ability to identify and evaluate all plausible hypotheses. Multi-Hypothesis Approach!

A collection is focused narrowly on information that will help to discriminate the relative probability of the alternate hypothesis.

This discussion was found in Chapter 5 Psychology of Intelligence Analysis (1999) by Richards J. Heuer, Jr. “Do You Really Need More Information? “

What Heuer is saying and describing is the scientific method! I have been thinking this way since the 1980s! The problem is the lack of real scientifically structured hypothesis-driven investigations within Fantasy Football/Sports. Hence the reason that has driven Bush and Cherney to write a textbook containing our experiments!


Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>