Backtesting Value-at-Risk Daniel Herlemont

16 mars 2010

Table des mati` eres Main source : Elements of Financial Risk Management by Peter Christoffersen see [?]

1

Introduction

Compared to the actual P&L Xt , the V aRt forecast is supposed to be worse only p ∗ 100% of the time. We can define the ”exception sequence” of VaR exception as It = IXt ≤V aRt

(1)

The exception variable I, returns 1 if on day t if the loss on that day is larger than the VaR predicted for that day. If the VaR was not violated, then the exception sequence returns 0. When backtesting the risk model, we construct a sequence of It across T days. If we are using a perfect VaR model, the exception sequence should be independently distributed over time as a Bernoulli variable.

2

The Unconditional Coverage Testing

If we want to test the fraction of exceptions for a particular risk model, call it π. To test it, we write the likelihood of an iid Bernoulli sequence : T Y L(π) = (1 − π)1−It π It = (1 − π)T0 π T1 i=1

1

(2)

3 INDEPENDENCE TESTING where T0 and T1 are the number of 0s and 1s in the sample (T0 + T1 = T ). We can easily estimate π from π ˆ = T1 /T (3) L(ˆ π ) = (T0 /T )T0 (T1 /T )T1 Under the unconditional coverage null hypothesis that π = p, where p is the known VaR coverage rate, we have the likelihood L(p) = (1 − p)T0 pT1

(4)

We can check the unconditional coverage hypothesis using the likelihood ratio LRuc = −2 log[L(p)/L(π)]

(5)

Asymptotically, the test will be a χ21 with one degree of freedom. Choosing a significance level of say 10% for the test, we will have a critical value of 2.7 from the χ21 distribution. If the LRuc test value is large than 2.7 the we reject the VaR model at 10% level. Alternatively, we can calculate the p-value associated with our test statistic. The p-value is p − value = 1 − Fχ21 (LRuc ) (6) Where Fχ21 denotes the cumulative function of a χ21 variable with on degree of freedom. If the p-value is below to the desired significance level, then we can reject the null hypothesis.

3

Independence Testing

Imagine all of the VaR exceptions in a sample happening around the same time. Would you then be happy with a VaR with correct average or unconditional coverage ? The answer is clearly no. For example, if the 5% gave exactly 5% exceptions but all of these exceptions came during a 3-week period, then the risk of bankrupcy would be much higher than if the exception came scattered randomly through time. We therefore would very like to reject the VaR models which imply vilations that are clustured in time. Such clusturing can easily happen in a VaR constructed from the historical simulation method. If the VaR exceptions are clustered, then the risk manager can easily predict that if today is an exception, then tomorrow is more than p likely ti be an exception as well. This is clearly not satisfactory. In such situation, the risk manager should increase the VaR in order to lower the conditional probability of an exception to the promised p. Our task is to establish a test that will be able to reject a VaR with clustered exceptions. To this end, assume the exception sequence is dependent over time and that it can be

Daniel Herlemont

2

3 INDEPENDENCE TESTING described as a first order Markov sequence with transition probability matrix   π00 π01 Π1 = π10 π11

(7)

with πij the transition probability πij = P [It = i and It+1 = j]

(8)

Note that π00 = 1 − π01 and π10 = 1 − π11 For example, π01 is the probability of an exception after a ”non exception”, π11 is the probability of two consecutive exceptions. If we observe a sample of T observations, then we can write the likelihood function of the first-order Markov process as T T T T L(Π1 ) = π00 00π01 01π10 10π11 11

(9)

with Tij is the number of observations with a j following an i. Taking first derivatives with respect to π01 and π11 and setting these derivatives to zero, one can solve for the maximum likelihood estimates T01 (10) πˆ01 = T00 + T01 T11 (11) πˆ11 = T10 + T11 Note that, Ti0 + Ti1 = Ti , so that the estimated matrix is   T00 /T0 T01 /T0 ˆ Π1 = T10 /T1 T11 /T1

(12)

Allowing for dependence in the exception sequence corresponds to allowing π01 to be different from π11 . We are typically worried about positive dependence, which amounts to the probability of an exception following an exception (π11 ) being larger than the probability of an exception following a non exception (π01 ). If, on the other hand, the exceptions are independent over time, then the probability of an exception tomorrow does not depend on today being an exception or not, and we write π01 = π11 = π Under independence, the transition matrix is thus   1−π ˆ π ˆ ˆ Π= 1−π ˆ π ˆ Daniel Herlemont

(13)

(14) 3

5 WEIGHTED HISTORICAL SIMULATION with π ˆ = T1 /T We can test the independence hypothesis that π01 = π11 using a likelihood ratio test LRind = −2 log[L(ˆ π )/L(Πˆ1 )] ∼ χ21

(15)

where L(ˆ π ) is the likelihood under the alternative hypothesis from the LRuc test.

4

Conditional Coverage Testing

Ultimately, we care about simultaneously testing if the VaR violations are independent and the average number of violations is correct. We can test jointly for independence and correct coverage using the conditional coverage test LRcc = −2 log[L(p)/L(Πˆ1 )] χ22

(16)

which corresponds to testing that π01 = π11 = p Notice that LRcc = LRuc + LRind

5

Weighted Historical Simulation

Compared to the normal method, historical simulation (HS) has better unconditional but worse conditional coverages test results. In other words, HS looks to be a good a good VaR model on average, except that exceptions have a tendency to be highly clustered. On the other hand, normal models (like RiskMetrics, ou GARCH) under estimate the VaR (more exceptions that expected), however exceptions are more evenly distributed over time. One can try to combine to best of the two methods using the Weighted Historical Simulation (WHS). In the Historical Simulation (HS) one of the main problem is the choice of the window size m. If m is too small, then we do not have enough observations in the left tail to calculate a precise VaR measure, and if m is too large, then the VaR will not be sufficiently responsive to the most recent returns, which presumably have the most information about tomorrow’s distribution. We now consider a modification of the HS technique, which is designed to relieve the tension in the choice of m by assigning relatively more weight to the most recent observations and relatively less weight to the returns further in the past. This technique is referred to as weighted historical simulation (WHS). WHS is implemented as follows :

Daniel Herlemont

4

7 REFERENCES ˆ The sample of the m past returns Rt+1−i for i = 1, m is assigned a weight ηi declining exponentially through the past as follows :

ηi =

η i−1 (1 − η) 1 − ηm

(17)

with 0 < η < 1. ˆ The observations, along with their assigned weights, are sorted in ascending order. ˆ The 100p% VaR is calculated by accumulating the weights of the ascending returns until 100p% is reached.

Notice that once ¸c is chosen, the WHS technique still does not require estimation and thus retains the ease of implementation.

6

To Do ˆ Perform the unconditional and conditional tests for the standard risk models (RiskMetrics or GARCH VaR, Historical VaR, ...) seen in the previous work. ˆ is Weighted Historical Simulation better ? find the optimal η

7

References

Daniel Herlemont

5