Nice one - Needed this sequel. The observation that you shouldn't focus on hit rate isn't novel, but the facts that past predictions of "hard to predict things" doesn't guarantee future predictions, and quantifying what a "hard prediction" is in the first place is very difficult add some new complexity. Not to mention that we're trying to maximize expected gain, not hit rate per se. Like Soros said, "It's not whether you're right or wrong that's important, but how much money you make when you're right and how much you lose when you're wrong."
Fwiw, I analyzed past predictions of doom by Burry (who definitely got more attention that Roubini because of Bale, I guess :p), and also the track record of Cramer... What I didn't do is compare the hit-rate of just predictions of doom by multiple experts on the same topics - I'm not sure that would be even helpful in future scenarios, considering that true surprises may stump all the existing experts, but it's something to think about...
The comments in the rationality blogs make sense in hindsight as well, but alas, how do we know what we were looking for in the times of crisis?
Something I find interesting is how hard it is to internalise maximise EV strategies. Even VCs, arguably trained in this battleground, lose the facility as soon as they switch from investing to punditry. Even in investing its so rare to hear someone say "we were wrong", as if that's just shameful.
Interestingly this also ends up giving strange advice for a person individually (try to make tail predictions until one's right) vs collectively (try not paying attention to most crazy predictions).
Haha, yes. But I can think of a scenario in which "looking right" has a greater advantage. Especially for VCs, if they have a higher success rate, the companies that want to approach the VC will increase in number. For an actively managed fund, looking good increases the cash inflow. I think it was Nick Maggiuli's first article was about how funds made their income not from profits but from the steady cash inflow - In such situations, if you know that you're not great at picking exponential winners, you'd try to spin the narrative in a way that makes you look good.
Aside: Taking more risks -> Buy everything -> Index fund? But I suppose the real asymmetric opportunities are before things enter the public market, so your point still stands.
I'm not sure the look actually matters that much. Sequoia has a 50% loss ratio, but it still gets all the good companies, because everyone only looks at the winners. The opposite works better for some type of PE companies, though that also is moving with Vista and Insight.
Also the ramp from taking more risks > index fund is a fun one right? In an odd way its about index construction. For instance both YC and Tiger do somewhat similar strategies where they weed out the "bad" investments and take the ranch. Also funnily enough the post IPO bump for many software cos was also pretty fantastic (until very recently :))
"In almost all cases it's better to fight the incentives to maximise hit rate and try to maximise EV instead. Even in cases you're going to sound silly, it seems worthwhile to be a Roubini rather than, say, Jim Cramer."
This seems to assume that in almost all cases there are extraordinarily asymmetric payoffs *in the rigth direction*. For instance, shorting the market has an infinite potential downside. So in this case being wrong more often is bad. If you are a doctor, an engineer, or an airplane pilot, the same applies. I'm not sure if we can say, thus, that we should be wrong more often in almost all cases, but only that we should identify extraordinarily asymmetric *positive* payoffs and then be wrong more often.
Hmmm fair shout. I agree its complicated. I do actually think shorting is fine as well, though if I were running the strategy I'd try and acknowledge that most of my gains might come from identifying 1 Nikola rather than hoping to hit 20 FB dips in a row.
Nice one - Needed this sequel. The observation that you shouldn't focus on hit rate isn't novel, but the facts that past predictions of "hard to predict things" doesn't guarantee future predictions, and quantifying what a "hard prediction" is in the first place is very difficult add some new complexity. Not to mention that we're trying to maximize expected gain, not hit rate per se. Like Soros said, "It's not whether you're right or wrong that's important, but how much money you make when you're right and how much you lose when you're wrong."
Fwiw, I analyzed past predictions of doom by Burry (who definitely got more attention that Roubini because of Bale, I guess :p), and also the track record of Cramer... What I didn't do is compare the hit-rate of just predictions of doom by multiple experts on the same topics - I'm not sure that would be even helpful in future scenarios, considering that true surprises may stump all the existing experts, but it's something to think about...
The comments in the rationality blogs make sense in hindsight as well, but alas, how do we know what we were looking for in the times of crisis?
Something I find interesting is how hard it is to internalise maximise EV strategies. Even VCs, arguably trained in this battleground, lose the facility as soon as they switch from investing to punditry. Even in investing its so rare to hear someone say "we were wrong", as if that's just shameful.
Interestingly this also ends up giving strange advice for a person individually (try to make tail predictions until one's right) vs collectively (try not paying attention to most crazy predictions).
Haha, yes. But I can think of a scenario in which "looking right" has a greater advantage. Especially for VCs, if they have a higher success rate, the companies that want to approach the VC will increase in number. For an actively managed fund, looking good increases the cash inflow. I think it was Nick Maggiuli's first article was about how funds made their income not from profits but from the steady cash inflow - In such situations, if you know that you're not great at picking exponential winners, you'd try to spin the narrative in a way that makes you look good.
Aside: Taking more risks -> Buy everything -> Index fund? But I suppose the real asymmetric opportunities are before things enter the public market, so your point still stands.
I'm not sure the look actually matters that much. Sequoia has a 50% loss ratio, but it still gets all the good companies, because everyone only looks at the winners. The opposite works better for some type of PE companies, though that also is moving with Vista and Insight.
Also the ramp from taking more risks > index fund is a fun one right? In an odd way its about index construction. For instance both YC and Tiger do somewhat similar strategies where they weed out the "bad" investments and take the ranch. Also funnily enough the post IPO bump for many software cos was also pretty fantastic (until very recently :))
"In almost all cases it's better to fight the incentives to maximise hit rate and try to maximise EV instead. Even in cases you're going to sound silly, it seems worthwhile to be a Roubini rather than, say, Jim Cramer."
This seems to assume that in almost all cases there are extraordinarily asymmetric payoffs *in the rigth direction*. For instance, shorting the market has an infinite potential downside. So in this case being wrong more often is bad. If you are a doctor, an engineer, or an airplane pilot, the same applies. I'm not sure if we can say, thus, that we should be wrong more often in almost all cases, but only that we should identify extraordinarily asymmetric *positive* payoffs and then be wrong more often.
Excellent article otherwise.
Hmmm fair shout. I agree its complicated. I do actually think shorting is fine as well, though if I were running the strategy I'd try and acknowledge that most of my gains might come from identifying 1 Nikola rather than hoping to hit 20 FB dips in a row.