Issue: March/May 09


Surveys, surveys and damned surveys

Rankings of managers’ performance have their uses. They also have their flaws. Trustees should be cautious in allowing the former to overshadow the latter.

You know how lots of trustees pick an asset manager. They look at performance surveys, choose one at the top, and on they go. Job done. Nobody can fault a trustee for selecting a manager high in the rankings. Few are particularly bothered that surveys rank the performance of retail funds, not retirement funds, which are a different kettle of fish, so long as they can glean an easy impression.

Performance surveys are influential. Popular too. Amidst the overpowering amount of information available, they offer a short cut for trustees to make decisions. And not only trustees. The surveys are also in-your-face sales tools for any range of investment products, from unit trusts to life policies, because they seem so effortless to understand.

There, neatly tabulated, are all the percentages of comparative performance. What’s not to grasp? The beauty for the salesman is that he can find in the surveys whatever best suits his pitch. Among the hundreds of different retail funds run by dozens of different managers, over different timeframes, it’s only a matter of extracting from surveys the ammunition to clinch a deal. There’s something in them for everybody.

Now comes the hard part. It’s in the illusion that surveys rank managers for skill. They can’t, and they don’t. They’re merely snapshots of performance at a particular moment. They don’t show the thinking behind the numbers, the risks taken or avoided, the astuteness in timing sells to finance buys, or the mandate constraints in a manager’s systemic bias. Worse, they subliminally perpetuate the myth that past performance is a guide to future performance.

Users aren’t left any the wiser whether performance is a function of skill, luck or the market dynamic. Not that it really matters, except in fostering the delusion that the better a manager performs the more skilled he is. This is delusion at its most dangerous when it causes trustees, whose usual term of tenure averages three years against the much-longer time horizons of retirement funds, to boot out managers that appear from the frequent appearance of surveys to be underperforming.

For one thing, it’s commonly accepted that about 90% of a manager’s performance reflects market performance. For another, the JSE is tiny. Its all-share

index (Alsi), commonly used as the performance gauge, has only about 80 stocks with free floats of sufficient size that they are considered by institutions to be “investable” for pension funds.

Cabot-Alletzhauser... don't chop and change
Cabot-Alletzhauser... don’t chop and change

And of these stocks, a handful is core to portfolios. They’re the highest-capitalisation shares, mainly in resources, which dominate the Alsi. When the mega-caps are running, it’s hard for managers to beat the Alsi benchmark because funds’ portfolios require diversification. When they aren’t running, diversification causes the benchmark to be beaten. It simply reflects market structure, not skill.

Between managers, relative outperformance and underperformance happen by fiddling at the fringes of these 80 shares; by downweighting a little among the mega-caps to upweight a bunch of others, or vice versa. Over time the tinkering tends to even out. Yet trustees often conclude, despite the transition costs, that they’re doing the best for their funds by switching to managers that rank higher in a particular survey than the managers they have.

What does matter, emphasises Anne Cabot-Alletzhauser of Advantage Asset Managers, is the fund’s strategic asset allocation and its appropriateness to a time horizon: “The greatest destruction of value is caused by the pension funds themselves. When they chop and change managers, they lock in losses.”

Surveys, she suggests, offer nothing concrete to evaluate. “Active performance relative to the Alsi benchmark seems fuelled more by what the managers didn’t buy than by what they did,” she’s found. “In the face of a highly concentrated and illiquid market, good portfolio construction is the key investment skill for adding value to market performance.”

Or, as Dave Bradfield of Cadiz Securities elaborated last year to a conference on “alpha” (manager outperformance): “Because pension-fund regulations restrict investments to long-only holdings, this regulatory restraint alone can limit a manager’s ability to transfer stock-selection skill into performance by as much as a third. Add to that constraints in the form of mandates that demand excessive levels of risk (high tracking errors above 5%), high levels of conviction (limiting the number of shares to below 40, for example), and require the manager to manage a large amount of assets (over R15bn), then the ability to transfer skill reduces even more.”

So size does matter. With a small fund of, say, R100m, the manager can gear the portfolio to a single macro projection; for instance, a boom in construction. If the bet pays off, the manager can show exceptional outperformance; if it doesn’t come off, the reverse applies. As funds increase in size, it becomes that much more difficult to show outperformance through concentration on a sector.


In her presentations, Anne Cabot-Alletzhauser is fond of reproducing this slide.
The moral is self-evident:

But we want so hard to believe

And this slide too. Take a stab at which percentage applies to which manager
and then look at the slide below for the answer:

Match the Asset Manager with the Performance

Surprise, surprise!

Suprise, suprise!

This illustrates other weaknesses highlighted by Pranay Chagan of Coronation Fund Managers. He points out that the surveys don’t show the sources of returns or compare funds of similar sizes. Neither do they indicate whether a manager’s existing investment process can be replicated as the assets under management increase.

Moreover, surveys rely on data supplied by the managers themselves. Since there’s no independent audit of the data collated, there’s no check on cherry-picking. Not that any managers do, of course. But because so much selling turns on surveys, there’s an obvious temptation for managers to select from the range of ever-proliferating retail products those that show their records in the most marketable light.

When surveys don’t necessarily compare like with like, and where there’s no composite collation of all similarly sized funds with the same mandate (such as equities benchmarked against the Alsi), what can performance comparison by percentages possibly mean to the average trustee? Are they an adequate basis for him to make decisions on manager selection?

Those who do know better are the consultants. They have processes better to monitor managers’ performance. Unfortunately, there are no surveys to help trustees monitor consultants’ performance. Which means the buck stops in the ether.

So read the surveys, such as they are, for what they are. And for no more.