University of Cincinnati College of Law
University of Cincinnati College of Law Scholarship and
Publications
,$+/)+!$*&+ ),$!+!'&* '$$'.,$+/ '$)* !(
$$
Shaking Decision Trees for Risks and Rewards
Marjorie Corman Aaron
University of Cincinnati College of Law)'&%,,
'$$'.+ !*&!+!'&$.')#*+ 2(** '$)* !($.,,(,*
)+'+ !*(,+*'$,+!'&&)!+)+!'&'%%'&*
1!*)+!$!*)', ++'/','))&'(&**/+ '$$'.,$+/ '$)* !(+&!-)*!+/'!&!&&+!'$$'. '$)* !(
&,$!+!'&*+ *&(+')!&$,*!'&!&,$+/)+!$*&+ ),$!+!'&*/&,+ ')!0%!&!*+)+')'&!-)*!+/'
!&!&&+!'$$'. '$)* !(&,$!+!'&*')%')!&')%+!'&($*'&++ #& !)* ,,
'%%&!++!'&
)"')!')%&)'& #!&!*!'&)*')!*#*&.)*!*(,+*'$,+!'&0!&$$+
12 DISPUTE RESOLUTION MAGAZINE | FALL 2015
Shaking Decision Trees
for Risks and Rewards
By Marjorie Corman Aaron and Wayne Brazil
W
e are two long-time colleagues with many
years of work in the courtroom, in the
classroom, on the bench, and around the
mediation table. Our purpose here is to extend a
conversation between us that we hope will enhance
our readers’ appreciation of the power and the
limitations of decision analysis. We write together,
approaching this subject from different perspectives,
some wholly complementary and others reflecting
professionally respectful differences of view.
We hope that what follows will equip lawyers and
neutrals to make better informed judgments about
how to use decision analysis more instructively and
reliably — as well as how to identify circumstances in
which its superficial use can yield unreliable assess-
ments of risk and value.
Our topic centers on the theme of this issue of
Dispute Resolution Magazine: the role of numbers
in our corner of the legal subculture. Numbers have
huge psychological power, and this power is the
principal source of both the value and the danger in
decision analysis.
It is ironic that among lawyers, many of whom
turned to this profession because they felt so chal-
lenged by math, numbers have so much power. Maybe
lawyers, who are more comfortable with words, are
especially susceptible to measurability bias. We tend to
overweigh what is measured, counted, quantified —
and to underweigh what is not. Take something out
of the language of numbers, and we are less likely to
assign it importance for decision-making. Present that
same message in numbers, and we consider it signifi-
cant. Our clients are apt to do the same.
We wonder if this is because humans have a deep
need for certainty, or at least for some kind of reas-
surance. It may be rooted in our raw understanding
of how profoundly uncertainty pervades so much of
our existence. But the lure of quantification makes us
vulnerable to deception through the slightest manipu-
lation of numbers.
Of course, even with its numerical appearance and
mathematical operations, decision analysis provides no
certainty. In a legal case, it is based upon human esti-
mates. Thus, the numbers it yields are no more certain
than traditional case evaluation, delivered in prose.
The Pure Pluses
Decision analysis marries judgments (best profes-
sional guesses) to numbers. A fragile coupling — but
not for that reason to be shunned. On the contrary,
this union can yield great rewards.
Decision analysis, properly used, can constitute a
highly disciplined, rational, analytically demanding
and careful approach to decision-making — at least
when the thing about which we need to make deci-
sions is as elastic, dynamic, fluid, and mercurial as civil
litigation can be.
This is true because decision analysis exposes,
more effectively than any other tool, including a prose
summary, the number and character of the “risk piv-
ots” that civil litigation entails and clients and lawyers
must try to assess.
1
By exposing these in a graphic
presentation, decision trees also help clients and law-
yers understand the succession of and the dynamics
between the pivot points.
Just as important, carefully constructed decision
trees emphatically remind us that to fully comprehend
our litigation circumstance, we must assess each risk
pivot in relation to the others. Each may contribute
to larger cumulative risks. In this way, decision trees
succinctly illustrate the complexity, convolution, and
uncertainty that inhabit so much of civil litigation.
Lawyers and clients both seek to feel comfortable
with their decisions. Many need to be able to explain
and defend their choices to themselves and to others,
including shareholders as well as people higher on
the organizational chart. In our own work, we have
found that when used with appropriate refinement
and circumspection, the method’s numerical yields —
cumulative probabilities of possible outcomes and
overall discounted value — may provide people with
such comfort. Decision trees’ numbers can help clients
feel that their settlement decisions (yea or nay) are not
undisciplined or arbitrary but supported by a process
that provides logic and reasoning.
Important Precautionary Refinements
Decision tree analysis involves cumulating prob-
abilities. Put in the clients’ words, “Of all the ways this
case could play out, what’s most likely to happen?
What are my overall chances of getting nothing?
Of winning enough to cover my losses? Of getting
socked with a verdict that will bankrupt my business?”
The method is also used to derive a “discounted
value”: the sum of each possible outcome multiplied
by its cumulative probability. Given that these
results — cumulative probabilities and discounted
value — run on math and are often given meaning in
settlement decisions, anyone who wants to use deci-
sion trees effectively and properly needs to deeply
understand the process’s sophistication and limita-
tions. In that spirit, we offer the following discussion
of important cautionary refinements. Far from an
exhaustive list, it addresses some of our own concerns
about the method’s use.
Beware of biases when estimating the probabilities
and case outcomes.
2
Lawyers and clients are both subject to optimism
and partisan perception biases, notwithstanding
commitments to remain “objective.” Also relevant is
the anchoring bias; initial numbers unduly influence
our judgments.
These biases may be old news to our highly edu-
cated readers. The bad news is that, even when aware,
people tend to believe they are less susceptible to
these biases. But that’s just not true.
3
Research estab-
lishes that most lawyers are not terribly competent
at predicting how a judge, jury, or arbitrator will rule.
Attorneys tend to be overconfident and inaccurate.
Interestingly, research suggests that the risk of exces-
sive optimism increases with the complexity of the task
or the target of estimation — and forming “guess-
timates” about litigation outcomes is a notoriously
complex task.
4
Thus, we urge humility when estimating
probabilities on a decision tree. It is good practice to
try a range of probability estimates for critical risk piv-
ots. Even if your current estimate is 65% for a certain
event (say, liability), try calculating the tree with that
probability at 60% or 70%, or 55% or 75%.
The same advice holds for predicting verdict
awards. While plaintiffs and their counsel certainly
overestimate, research suggests that defense lawyers
are particularly prone to optimism when (under)
estimating awards.
5
Defense counsel are advised to
remember: the jury that finds liability is a jury that
favors the plaintiff. One of us served as a mediator
in a case where, in a caucus, we all waved away the
possibility of damages beyond a few million dollars.
The case proceeded to trial, and the jury awarded
damages of $40 million. Don’t fail to consider the
worst-case scenario.
Judgmental anchoring — a previously considered
number’s influence on a numeric judgment — also
critically impacts the decision analyst. Much as an
anchor pulls a boat in its direction, a first number —
that first guess or reference point, even if obviously
wishful — pulls subsequent numerical judgments up
or down. Anchoring is another robust, consistently
demonstrated phenomenon in research on psychology
13 FALL 2015 | DISPUTE RESOLUTION MAGAZINE
and decision-making, across domains, for novices and
experts, including lawyers. It is easy to see how a law-
yer or client could be anchored to a number generated
by his or her own biased guess, or by a recent high or
low verdict reported online or in the papers.
We’d like to think an intelligent lawyer would
adjust an early number for new information or further
thinking. Unfortunately, research confirms that, while
some adjustment occurs, most people adjust insuf-
ficiently from initial anchors. People estimate ranges
too narrowly, and they tend to remain confident
and optimistic.
Probability estimates must be true to their location
on the tree and must assess interdependence of
outcomes at risk pivots.
Effective estimates of probabilities at any given risk
pivot must reflect what the circumstances would be
on that particular branch of the decision tree at that
particular juncture, i.e., at the moment in time repre-
sented on the tree. In a tree that presents a risk pivot
at summary judgment, probabilities after “summary
judgment denied” should be estimated in that light.
After all, only after such a ruling will everyone know
that the judge found some merit to arguments about
a serious factual question.
To dig more deeply into the litigation weeds and
the litigator’s judgment, imagine a case involving
a hard-fought motion to dismiss a cluster of fraud
claims. Along each tree branch after the motion, the
next risk pivot might be labeled “liability or no liabil-
ity.” The litigator’s common sense knows to adjust
chances of liability based on whether the risk pivot
sits on a tree branch following a positive or negative
ruling on the fraud claims. After all (let’s assume), if
the fraud claims remain, the jury will hear additional,
inflammatory evidence that may also impact the odds
of its finding liability.
Thus, before working through a decision tree analy-
sis, defense counsel might have roughly estimated
the chances of winning a defense verdict at, say 50%
to 60%. But when constructing the tree, counsel is
compelled to recognize that these percentages are
credible only if the fraud claims are dismissed. Given
the judge’s revealed proclivities and the potentially
inflammatory evidence, counsel would be wise to
estimate that the chances of a defense verdict along
that path are much lower.
Under probability theory, an analyst can
determine the cumulative or joint probability of a
particular outcome by multiplying the likelihood of
one event by the likelihood of another event only
if the likelihood that each event will occur is truly
independent. In civil litigation, sometimes the same
important factor, or set of closely related factors,
can significantly affect the likely outcome at differ-
ent pivot points along a decision tree. When this is
the case, a decision analyst must be very careful to
assess the impact of the interdependence of the fac-
tors at each pivot point.
Basic probability theory agrees. Indeed, when
calculating cumulative probabilities, bedrock rules of
probability require deliberate adjustment if probabili-
ties along a path are not independent.
To discuss the question of independence in cumu-
lative probability, it’s worth illustrating how cumulative
probabilities work with a game involving serial jars
of marbles. The rules of the game are that to win the
pot of gold, you have to draw two red marbles (while
blindfolded), one from each of two jars placed in a
row. The first jar holds 100 marbles, 80 red and 20
black. The second jar also holds 100 marbles, but 50
red and 50 black. What happens on the first draw has
no impact on the draw from the second jar (except
that you won’t proceed to the second jar if you draw a
black marble from the first). In this game, the cumula-
tive probability of winning the end pot of gold is 40%:
80% (first jar) x 50% (second jar) = 40%. These two
independent probabilities are not affected by any hid-
den, shared factors. In other words, drawing that first
red marble does not have any hidden but powerful
effect on the odds that you will later draw another.
Returning to the jars of marbles: what if, as soon
as you drew a red marble from that 80/20 first jar, an
invisible hand altered the black-to-red marble ratio in
the second jar? That invisible hand changed the marble
mix in the second jar from 50 red/50 black to 70 red/30
black. Now, the cumulative probability of drawing two
red marbles is no longer 40% (the product of 80% x
50%); it is 56% (the product of 80% x 70%).
In the case example, the judge’s ruling on the
fraud claims functions as the invisible hand in the
marble jar. It changes the “marble mix.” The rules of
probability are satisfied only if players use the new,
altered probability.
14 DISPUTE RESOLUTION MAGAZINE | FALL 2015
Let’s look at another example to illustrate the chal-
lenge presented when the same factor affects the like-
lihood of outcomes at different risk pivots. In personal
injury cases, the same factor — what the jury thinks
of the plaintiff as a human being — can affect both
the likelihood that the jury will believe her account
of how the accident occurred (thus how the jury will
resolve the liability issue) and the likelihood that the
jury will be generous when it awards general damages
(a notoriously elastic determination). When the same
variable can play a significant role in the outcome
at two formally distinct risk pivots, a risk analyst who
is trying to determine the cumulative probability of
an ultimate outcome faces a very difficult task. She
must take fully into account her judgment about the
likelihood that the jury will believe (and believe in) the
plaintiff when she is developing her estimate of the
most likely zone of general damages.
What’s crucial here: Pay attention to the interdepen-
dence/independence of outcomes at the risk pivots
and stay on top of the rolling analytical logs. As reality
unfolds, return to earlier developed decision trees to
adjust estimates and structure based on new insights.
Take into account what has happened in the litigation,
unforeseen developments with evidence and witnesses,
and new information learned in discovery. A judge’s
comments at oral argument or in a written opinion
might call for some reevaluation. After all, the judge
may have been the first neutral to weigh in and will rule
on evidentiary motions at trial.
Reflect what triers of fact are
asked to decide — and how
they return verdicts.
The decision analyst is charged
with thinking carefully about how
judges and juries may rule. To do
that, the decision analyst should
consider what questions the triers
of fact will be asked, imagine their
possible answers, and estimate the
likelihood of their (determinative)
answers.
For that reason, the decision
analyst should be aware of the
importance of the form of verdict a
jury will use. Let’s assume that the
jury will understand the judge’s
formal instructions that in order to find liability, it must
first find both causation and negligence. Where the
jury will be given only a simple general verdict form,
should the decision analyst assess the probabilities of
each separately and multiply them to get the cumula-
tive probability of a liability finding? Probably not. After
all, when jurors return verdicts on general verdict forms
(without addressing specific questions), a litigator’s
experience suggests that despite the legal distinctions,
the jurors will slip unselfconsciously into a gut sense of
what’s right — of the justice they want to bring about.
If you don’t believe they will assess the negligence and
causation issues separately, but rather holistically, then
your probability estimate should be holistic. It should
reflect the way you believe the jury will approach the
question.
In federal courts, juries commonly return their ver-
dicts in the form of answers to special interrogatories.
Special interrogatories are designed to cabin deci-
sion-making sloppiness by compelling juries to make
separate findings about legally separable issues, e.g.,
to address separate components of multi-element
claims or defenses one component or one element at
a time. When the court thus parses and isolates sepa-
rate issues, it asks the jury to determine, separately
for each issue, whether the party bearing the burden
of proof has met its burden. To assess probabilities,
the decision analyst could ask the parallel questions:
What is the likelihood of the jury answering yes to
15 FALL 2015 | DISPUTE RESOLUTION MAGAZINE
each and every one of the questions required for a
liability finding?
Pay attention to your gut — and to the arithmetic.
What if the cumulative probability of a particularly
important result — liability or a desirable damages
award — ends up far, far from a lawyer’s gut sense?
Should we look to the gut or the math as the dis-
tortionist? The answer, of course, is that we should
re-examine both with some care.
A decision tree that is too simple fails to represent
complex realities. Imagine an employment case with
serious dispositive motions, controversy about back
pay, emotional distress, front pay, and punitive damag-
es. A tree with one risk pivot for liability and one round
damages estimate, or even a rough undifferentiated
range, would not fairly map the litigation. This case will
involve multiple risk pivots on liability and damages
components. There is more than one way the plaintiff
could lose or end up with pretty low damages.
One of the strongest reasons to use decision analy-
sis is that the lawyer’s intuitive gut calculator cannot
know the cumulative probabilities for each possible
outcome in a complicated case. We know that, in rare
instances, everything or nothing will break our way.
But reality is more often a dastardly combination of
positive and negative breaks. When the tree fairly
captures an informed analysis of the risk pivots and
yet the cumulative probabilities of desirable and
undesirable outcomes contradict the lawyer’s gut
sense, it’s time for the lawyer and client to carefully
consider arithmetic’s counsel.
On the other hand, experienced lawyers also have
a legitimate gut sense that the more branch clusters
along a given decision-tree path, the lower the
cumulative probability of each possible result and the
lower the discounted value. A highly complex tree
with many layers of branch clusters may also serve to
distort reality. This kind of tree should be “read” with
some caution, with a critical eye for
over-complexity,
for too many risk pivots, and too much interdepen-
dence between their outcomes.
The net matters.
A competent decision analysis should at the very
least account for all quantifiable costs along the path
to any outcome. Estimated attorney’s fees and costs
must be subtracted from the plaintiff’s potential posi-
tive “payoffs” (in non-fee shifting cases) and added to
the defense’s potential negative payoffs.
Estimated verdict amounts should also include
any statutory interest. Particularly in times of higher
general interest rates and when final judgment is far
in the future, it’s best to calculate the time value of
the future award.
Let’s imagine a case with a potentially dispositive
preliminary motion, with a relatively low chance of
success. Assume that if the plaintiff wins on liability,
base damages could be $75,000, $200,000, or
$350,000 — depending. The plaintiff could succeed
on some theory that would entitle her to collect her
16 DISPUTE RESOLUTION MAGAZINE | FALL 2015
attorneys’ fees from the defendant, and a 2X multi-
plier of actual damages. The defense costs will be
approximately $30,000 through the dispositive motion
(including discovery, which is only partially complete),
and an additional $70,000 through trial. Plaintiff’s
counsel’s “reasonable fees and costs” through trial
would also be $100,000.
The tree on page 16 details the discounted value
from the defense perspective without considering
anyone’s costs or fees.
See the bottom of page 17 for the tree after
including costs and fees the defense will or may pay.
Quite a difference in the discounted value.
Having decided to use this method, it would be
misleading to omit these fees and costs. They will be
real when incurred.
Best practice could also include subtracting other
quantifiable costs from net payoffs. For example,
the client might estimate that he will pay $8,000 in
overtime labor to comply with discovery. And what if
five executives will have to testify on deposition and at
trial? Using their high salaries as a base, the lost value
of their time in depositions, prep, and trial may be
in the tens of thousands of dollars. While quantifying
everything would be impossible, we should try to think
through all significant additional costs of the process.
Don’t ignore intangibles.
Intangibles matter when making decisions. Litigants
care and worry about risk.
6
They experience the
emotional value of restoration, vindication, or closure.
Litigants appreciate the value (to sense of self and
to future prospects) of a good recommendation or
endorsement, an enhanced reputation, a trademark’s
cachet, and the importance of goodwill with custom-
ers or upstream vendors. If the decision analyst and
the client can jointly formulate reasonable estimates
of their value, then theoretically these estimates could
be built into the payoffs at the end of the appropriate
path on the tree.
Most important is not to allow these intangibles to
be overshadowed and undervalued by undue focus
on the tree’s numerical inputs and outputs. The deci-
sion analyst is wise to create space in time and on
the page for discussion of intangible consequences
and why they matter. Plaintiffs who cannot afford
to pay the mortgage in the event of $0 recovery
may adjust their sights downward. The possibility
of losing future business or a current friendship if a
certain witness is subpoenaed may weigh heavily.
Intangibles are important, not secondary, because
they reflect the very real contexts within which our
legal disputes occur.
Summing It Up
When done with integrity and competence, deci-
sion analysis can offer considerable insight, improve
communication, and add greater rigor to the decision-
making process. Yet it is also susceptible to error and
manipulation in ways that we hope our readers will
come to recognize and avoid. ■
17 FALL 2015 | DISPUTE RESOLUTION MAGAZINE
Endnotes
1 We have chosen to use the term “risk pivots” as a less
technical way of describing what are called “chance nodes”
on a decision tree, usually represented by small circles.
2 Too much experimental and empirical research exists
confirming the power of bias in human (including lawyers’)
decision-making to attempt its thorough citation here. Thus
this article includes citations only for highly specific refer
-
ences. Those who wish to delve deeper into the impact of
bias and other ways that psychology impacts lawyers’ thinking
are encouraged to read Jennifer Robbennolt’s and Jean
Sternlight’s comprehensive work,
PSYCHOLOGY FOR LAWYERS:
U
NDERSTANDING THE HUMAN FACTORS IN NEGOTIATION, LITIGATION
AND DECISION MAKING (2013). Also, Ch. 5 in Professor Marjorie
Corman Aaron’s book,
CLIENT SCIENCE: ADVICE FOR LAWYERS ON
C
OUNSELING CLIENTS THROUGH BAD NEWS AND OTHER LEGAL REALITIES
(2012) provides a shorter summary on the topics. Important
research specific to lawyers’ decisions regarding settle-
ment and trial can be found in Randall Kiser’s book,
BEYOND
Marjorie Corman Aaron is
a Professor of Practice and
Director of the Center for
Practice at the University of
Cincinnati College of Law.
She regularly teaches deci
-
sion analysis to lawyers and
law students and is the author of many articles on dispute
resolution and decision analysis, as well as
Client Science:
Advice for Lawyers on Counseling Clients on Bad News and
Other Legal Realities (Oxford 2012). She can be reached at
[email protected]. Wayne Brazil is a mediator, arbitra
-
tor, and special master with JAMS. He served as a magistrate
judge in the US District Court for the Northern District of
California between 1984 and 2009. He has been a Professor
at the University
of California’s Hastings College of the Law,
the University of California/Berkeley’s School of Law, and the
University of Missouri School of Law. He can be reached at
RIGHT AND WRONG: THE POWER OF EFFECTIVE DECISION MAKING FOR
A
TTORNEYS AND CLIENTS (2010), drawing upon research reported
in the Randall Kiser, Martin Asher, and Blakeley B. McShane
article,
Let’s Not Make a Deal: An Empirical Study of Decision
Making in Unsuccessful Settlement Negotiations, 5 J.
EMPIRICAL
L
EGAL STUD. 3, 551-91 (2008).
3 Joyce Ehrlinger, Thomas Gilovich & Lee Ross,
Peering
Into the Bias Blind Spot: People’s Assessments of Bias in
Themselves and Others, 31
PERS. SOC. PSYCHOL. BULL. 5, 680-92
(2005).
4 Influential research drawn upon includes: the Elizabeth
F. Loftus and Willem A. Wagenaar article,
Lawyers’ Predictions
of Success, 28
JURIMETRICS 4, 437-53 (1988) and the Jane
Goodman-Delahunty et al. article,
Insightful or Wishful:
Lawyers’ Ability to Predict Case Outcomes, 16
PSYCHOL. PUB.
P
OL’Y. & L. 2, 133-57 (2010).
5 Roselle Wissler et al.,
Decisionmaking about General
Damages: A Comparison of Jurors, Judges, and Lawyers, 98
MICH. L. REV. 3, 751, 805 (1999). Note that, as defined in Kiser’s
study, the mean “decision error cost” — defined as the differ-
ence between the last offer and trial result — was $52,183 in
New York and $73,400 in California for plaintiffs, but $920,874
in New York and $1,403,654 in California for defendants.
See
Kiser et al., supra, 566-70.
6
Wh
ile there are technical ways to include numerical
discounts for risk aversion, these are quite technical (and,
ironically, fraught with risk for the integrity of the process).
18 DISPUTE RESOLUTION MAGAZINE | FALL 2015
