Superforecasting for the Farm
How much is it humanly possible to forecast future events on your farm?
With some mostly commonsense tactics, you can improve your prediction skills by 10% or more, according to new research.
Prediction is something you have to do every day, whether you’re aware of it or not. You look at a weather forecast and wonder about the odds that it applies to your place. Will fertilizer prices be cheaper in the fall or next spring? What are corn prices going to do this winter? Will your calving percentage surpass 90% this spring? Do I have enough grain storage capacity for the crops this fall? And so on, every day…
How good all those predictions are no doubt has a lot to do with the long-term success of your operation. So, can you do better? Yes, easily 10% better with just 60 minutes of study, or as much as 50% or more better if you really work at it, according to the authors of a new book, Superforecasting: The Science and Art of Prediction.
A star of the book, Bill Flack, is living proof of how you can improve your ability to peer into the future with greater accuracy.
In Superforecasting, Flack, a retired USDA employee who lives in Kearney, Nebraska, is contrasted with famous media pundits and other experts who purport to predict everything from hot stock market picks to presidential politics.
The problem with pundits is, while they may tell a good story about a possible future, they are seldom held accountable for their predictions, which it turns out, are generally no more accurate than a chimpanzee throwing darts on the wall, according to Superforecasting author Philip Tetlock.
You might as well flip a coin as listen to a lot of these so-called experts, Tetlock says.
“The ability to tell a good explanatory story and your forecasting accuracy is rather weak,” he says. In fact, the more famous the pundit, the less accurate the forecast.
Tetlock, a professor at the Wharton Business School, and colleagues have labored for years to identify superforecasters like Bill Flack.
Forecast Farmland Prices in 2017
To improve your forecasting skills, join thousands of other aspiring superforecasters on Good Judgment Open. Put your predictions to the test on an exclusive Successful Farming question: What direction will farmland prices go?
FORESIGHT IS REAL
Superforecasting gleans lessons from decades of research, including a large-scale, four-year forecasting tournament involving ordinary people like Bill Flack.
In the tournament, sponsored by a U.S. intelligence organization, thousands of participants competing as teams were asked to make forecasts on complex international events. Would protests in Russia spread? Was the price of gold going to plummet? Would there be war on the Korean Peninsula?
Flack and some others became “astonishingly good” at forecasting these kinds of global events, even better – 30% better – than intelligence analysts equipped with classified information.
The results of all the research proved a couple things. “One, foresight is real,” the authors write. “Some people – people like Bill Flack – have it in spades.”
Folks like Flack do 50% or more better than the “crowd,” according to Tetlock. “Superforecasters could assign probabilities 400 days out about as well as regular people could about 80 days out,” he says.
Another finding is that anyone using the principles developed in the project has a good chance to improve foresight and business judgment, not just government analysts predicting geopolitical events.
The researchers also have spun off a forecasting business, Good Judgment, Inc., to train companies and individuals to make better forecasts, according to Warren Hatch, senior vice president for Good Judgment, Inc. and a superforecaster himself.
The techniques can be used to improve your ability to predict events in your farm business, too, Hatch says.
Taking a step to better forecasting can be achieved at a low cost.
One result of the research that surprised Tetlock was that a 60-minute tutorial given the tournament participants improved their accuracy by 10% each year.
An improvement of 10% may not seem all that dramatic, but it’s the “difference between a consistent winner who’s making a living or the guy who’s going broke all the time,” a leading financial analyst told the Superforecasting authors.
WALKING AND TALKING
At a meet-up with him on a Nebraska farm on a hot, summer morning in June, Bill Flack discussed how superforecasting might be used by farmers.
In getting acquainted, Flack said he preferred manual labor in his career at USDA so he could work outdoors. He’s pleasant company, able to speak on a lot of topics, seemingly curious about most anything, and, he’s an expert bird watcher. (We counted 27 species on the corn/soy/hay farm.) A first impression was that here is a guy who’s comfortable in his own skin.
Asked about how superforecasting works, he fielded questions on a mix of topics. What were the chances the Cleveland Cavaliers, down 3 games to 2, would win the NBA title? How would you figure the odds of an old silver maple tree toppling over on a nearby house in the next 10 years? Would farmland prices likely rise next year?
There was an 80% chance of rain the day before; it didn’t rain. A lesson in probabilistic thinking is there was also a 20% chance it wouldn’t rain. In forecasting nothing is certain, he pointed out.
Like weather forecasters, you should attempt to forecast probabilities. “Don’t think in terms of yes, no, or maybe. Is something 80% or 25% likely to happen?” he said.
“A lot of what we’re interested in forecasting are low-probability catastrophes — extreme droughts, crop-destroying hailstorms, crashes in farmland prices,” he said. “If we’re trying to insure against such an event, it’s important to know whether the probability that it’ll occur in a given year is 1% or 10%.”
A number of ag issues could lend themselves to superforecasting techniques, he believes. “Commodity prices seem like an obvious one. Farmland prices also seem like an area that warrants forecasting. Interest rates could be an area of focus. Federal legislation and regulation are still another area. Will certain provisions be included in the next farm bill? What ethanol requirements will EPA impose?”
One of the keys to tackling such issues is to ask the right questions – questions that have definite, useful answers. Not too easy and not impossible to know.
So, for example, he says, “Don’t ask ‘Will fuel prices increase significantly this year?’ Ask something like ‘Will the price of 10% ethanol, 87 octane gas at the Cenex on Highway 81 exceed $2.75 at any time on or before October 31?’ ”
Flack thinks farmers ought to consider forming forecasting groups, like they have done with marketing clubs. Teams perform better than individuals, 27% better, superforecasting research shows. In the four-year forecasting tournament, after the first season, Flack was recruited as a superforecaster and became a member of a team of about a dozen people.
“Each team member made individual predictions, but our predictions were visible to the other members of our team. We had a team forum in which to discuss each question, and we were scored as a team as well as individually. This helped a lot,” he said.
In getting started with making any forecast, you should start by setting the “base rate,” Flack says. “If you're forecasting gas prices, find out how often the price has reached a certain level in the last few years, or look at how often it’s changed by a certain amount. That gives a good number to start from, and that can be adjusted up and down according to Arab politics, refinery explosions, gas-tax increases, and so on.”
Make frequent, small adjustments rather than very occasional large ones, and keep looking for more information, Flack says. “On many questions, the people in my superforecasting group came in at least once a week and nudged their forecasts up or down by 1% or 2%. People who made really fine forecasts — changing, say, from 67% to 68% — did better than those who stuck to round numbers like 65% and 70%.”
Finally, it’s also important you keep score, Flack says. The Good Judgment project uses something called a Brier score, which rewards early accuracy in forecasts, frequent updates, and honest attempts to hit the mark.
“Scoring gives feedback. Without it, it’s too easy to say, ‘I was really close, or I’d have gotten it right if this extraordinary thing hadn’t happened.’ Scoring doesn’t just tell me if I’m right or wrong on a question; it gives me an idea whether I’m being over- or underconfident in my predictions, and I can take that into account in future forecasts,” Flack says.
After some walking and talking, it became clear that Flack embodies the traits of a superforecaster described in the book.
Among other things, superforecasters tend to be:
- Cautious — nothing is certain.
- Humble — the world is complex.
- Open-minded — beliefs should be tested.
- Curious — enjoy solving problems.
- Numerative – comfortable with numbers.
- Gritty — determined to solve problems.
Most important, according to Tetlock, a superforecaster has a commitment to “belief updating and self-improvement.” This trait is about “three times a better predictor [of becoming a superforecaster] as its closest rival, intelligence,” says Tetlock.
In the words of Thomas Edison, superforecasting appears to be 75% perspiration and 25% inspiration, he writes.
By late morning, the temperature had already reached 90°F. (as forecasted), and Flack and I were both starting to sweat a little. But our enthusiasm for the topic hadn’t waned. We checked a field windbreak for more birds and noted that the corn was already starting to curl from the hot, dry weather.
In trying to explain his success, Flack echoed the “perspiration” theme from the book. “I had good luck, lots of spare time to work on forecasting, more good luck, and did some hard work. Did I mention good luck?”
This is the humble part of superforecaster makeup perhaps, but it is true that your predictions can come back to earth. “Slumps are inevitable; Bill knows there is no guarantee he will continue to be a great batter,” Tetlock writes.
“I think I was doing a lot of the right things, but it would have been easy for some of my forecasts to have turned out much worse than they did – if, for example, some event had happened a few days earlier or a few days later, or if a stock index or other indicator had been less than 1% higher or lower than it actually turned out,” Flack says.
So, OK, good luck helps, and your luck may turn. When it does, you’d do well to follow Flack’s example of being in what Tetlock calls, “perpetual beta.” You “try, fail, analyze, adjust, and try again,” as a way to keep improving.
In other words, perhaps, if you want to be a superforecaster on your farm, you pretty much make your own luck.
Ten Commandments of Superforecasting
The book, Superforecasting: The Science and Art of Prediction, is available from Crown Publishers. It contains the Ten Commandments of Superforecasting, a summary of the 60-minute tutorial that improved forecasting tournament participants’ performance by 10%.