Issues

Issues in Performance Evaluation

Moral Hazard

In the discussion of quantitative measures of performance evaluation, any moral hazard problems that may arise from the agent-principal relationship - between fund

managers and investors - are implicitly ignored.

Most fund managers receive compensation based on the size of the fund. Managers thus have a strong incentive to attract new capital thus increasing the size of the fund, principally by reporting above-average returns. One simple way for a fund manager to obtain superior performance in the short-term is to write out-of-the-money call options. Since any unrealised liability accrued by writing the options does not appear in reported returns, this is a simple way for a manager to increase reported returns in the short-term. Writing the options thus eliminates some of the upside potential of the fund, changing the distribution of the fund's returns, and weakening any justification for mean-variance analysis. 

Most moral hazard problems can be ameliorated by incentive compatible compensation contracts for fund managers. However, gaming strategies - such as writing call options - are virtually impossible to detect using traditional performance valuation techniques. Instead, they may appear as superior performers in the short-term. Indeed, the experiences of Barings and Daiwa indicate that this may hold for the medium-term as well. 

Length of Estimation Period

As mentioned in section 4.3.1, long time periods for estimation theoretically provide lower sampling errors. However, over long time periods 'regime changes', i.e. - fundamental shifts in economic structure - may occur (Merton [1987, p. 486]). 

Asset classes have lower variability than individual assets, thus their means, variances and covariances can be estimated more accurately than those of individual assets for the same estimation period. Asset classes therefore allow longer estimation periods than individual equities due to a 'diversification effect' in structural changes. 

Borrowing/Lending Rates

For all investors risk-free or near risk-free investments are available. The purchase of a government bond of the appropriate maturity is risk-free except for the rate to be earned on reinvested interest and/or the effect of unexpected inflation. Similarly. with the aid of government guarantees, savings accounts are essentially risk-free to individuals. 

However, borrowing rates vary widely amongst investors, dependent on their status. Although the CAPM has been generalised to accommodate this - see section 5.2.4 - the performance evaluation techniques outlined previously in this chapter implicitly assume lending and borrowing at the same risk-free rate. 

This is an assumption which may be acceptable within the confines of a stock market;

but is however, difficult to accept when considering performance across asset classes.

Changing Risk Levels 

Another problem is caused by changing risk levels. Fund managers will often try to earn excess returns by trying to anticipate market cycles, adjusting their portfolios accordingly. The standard scenarios are selling equities in anticipation of a market decline, or buying in anticipation of an increase. An alternative method utilised by  managers is to alter the average β on the portfolio in anticipation of market movements by changing the type of security held. If the market is expected to decrease, the fund manager lowers the average β on the portfolio in order to reduce its sensitivity to the market. If the market is expected to increase, the manager increases the β on the portfolio in order to make it more sensitive to market movements. 

Most fund managers consciously adopt one or more of these strategies in order to adjust its risk. Also, risk often changes as an effect of adjustments in composition of the fund, due to expectations concerning individual equity performance. As a result, the risk level of a fund changes through time, which causes problems in the traditional evaluation process. Most unit trust evaluation studies, and many portfolio evaluation services, calculate the risk of a fund by examining historic returns. If the risk level of a portfolio has changed through time, using historic returns can lead to a current estimate of risk very different from that at any particular point in the past. This issue is exaggerated when traditional performance evaluation techniques are applied to portfolios consisting of more than one asset class. 

Sample Selection Bias 

Survivorship bias deals with the 'survival of the fittest', and the resultant change in sample which generates bias. There has been some evidence to suggest that past unit trust performance predicts that in the future. However, Brown, Goetzmann, Ibbotson & Ross [1992] study the relationship between volatility and returns in a sample truncated by survivorship, and show that this relationship gives rise to the appearance of predictability. They show that truncation by survivorship gives rise to an apparent persistence in performance when there is dispersion of risk among money

managers, and demonstrate that standard risk-adjustments i.e. β, may not correct for

this. Where inclusion in a sample depends in part on the rate-of-return, survivorship bias will lead to bias in the moments and cross-moments of return, including β. Less obviously, survivorship bias will lead to a spurious relationship between volatility and return. This has implications for empirical tests of performance evaluation and for so called anomalies; see section 5.3.8. The magnitude of survivorship bias will depend  on the nature of survivorship, the distribution of returns across investments, and the evolution of portfolio policies over time;8 see section Changing Risk Levels above. 

A number of other selection biases can be made when presenting performance figures. One involves choosing the time interval under which performances are to be evaluated and compared. In addition, the accounting treatment of cash receipts and outflows may be affected by e.g. off - balance-sheet financing; see sections Moral Hazard and C.9. From period to period, a certain amount of turnover and change is to be expected. 

Choice of Market Proxy and Performance Benchmark

Although, many fund managers are often judged by total return performance relative to a prespecified benchmark, such criteria has been shown to result in suboptimal performance. Roll [1992, p. 14] has shown that a fund manager successfully minimising tracking error variance

"... will intentionally not produce a mean-variance Markowitz efficient portfolio under most circumstances. Even with perfect expected return information, the manager will select a portfolio dominated by other portfolios with higher average returns and lower volatilities (although not lower tracking error volatilities). "

As table 1.2 illustrates, this is compounded by the fact that by selecting a commonly used market proxy - the FT All-Share, or the S&P 500 - an investor is ignoring over half the world's wealth held in real estate. DeLisle [1993], when considering world commercial real estate, comments: 

'Estimates of relative levels of investable wealth suggest the value of commercial institutional grade real estate is in the range of US$1.8 - US$3 trillion; in terms of market share, it appears commercial real estate comprises some 20% of total wealth, ...'

First highlighted by Roll [1977],9 Roll & Ross [1994] illustrate that β is not an unambiguous risk measure. They show that the cross-sectional relationship is very sensitive to the choice of market index10 and that this can be quite close to the 'true' market index and the mean-variance frontier, yet produce significantly different cross-sectional11 slopes. To paraphrase, changing the choice of the definition of the market portfolio can alter both the β and the composition and/or ranking of portfolios. For example, the market portfolio may be defined as the FT All-Share or the FT-SE 650. 

These conclusions are, however, based on the analysis of 'efficient' capital markets. When considering β in its application to real estate, in either an international - see section 5.3.6 - or domestic context, the lack of any composite index of all assets makes the results obtained from its calculation, using any proxy, almost meaningless. 

This leads to the choice of a performance benchmark. Benchmark evaluation of a fund's performance is essentially concerned with comparing the return earned on that fund with the return earned on one or more benchmark portfolios; see section 4.3.1. It is therefore important that the portfolios chosen are truly comparable. Benchmarks must not only have similar risk, but also be bounded by similar constraints. The return earned by a fund is either compared to the return earned by a fund of similar risk, or an explicit risk-return trade-off is developed so that comparisons can be made across funds with markedly different risk levels. The benchmark portfolio must also meet the investor's objectives, an issue that may be possible to satisfy at a sector level, but one that becomes increasingly difficult to meet as the field of investments widens. 

The choice of benchmark has a marked effect on a portfolio's relative measured performance. By definition, performance is a zero-sum game,12 Therefore, the more accurately a benchmark reflects an investor's objectives, the more accurately that benchmark will reflect a fund manager's performance relative to it. 

Hypothetically constructed portfolios, or specialised benchmarks,13 are often used to evaluate a fund manager's performance. The use of specialised benchmarks has three distinct advantages. They:

The objectives of a fund manager should therefore be reflected in the benchmark chosen. This reduces the active risk associated with a fund, thereby reducing the time required to provide a statistically significant measure of performance. However, their construction and/or application is severely restricted within the context of real estate investment. The question posed is whether any real estate fund manager's performance can reasonably be assessed against a market index, such as the IPD Annual Index. This is further discussed in sections 2.4.2 and 2.4.3.

 ____________________________

8For a complete discussion on the role of bias in performance studies, see Brown, Goetzrnann k Ross [1995] and Brown &. Goetzmann [1995].

9See section 5.3.8. 

10ln addition to this, Appendix A  illustrates the difficulties involved in calculating one of the most important and widely used of market proxies, the S&P 500. 

11positive, negative or even zero. 

12Except for costs, which are rarely taken into account; see section 4.3. The application of benchmarks to real estate is further discussed in section 2.4.8.

13Also called 'normal portfolios'.