Standard Deviation is the second moment statistic. The first being mean. When we say that we can expect a return of 10% from S&P 500, we are talking about the first moment. That is an average over a specified period of time. However, you can rarely draw a meaningful conclusion from first moment statistics alone. To describe a return distribution and to ascertain the return in the future with reasonable probability, you require a second moment statistic. You need to specify the expected return as well as the expected volatility of the return. In other words, you need mean and standard deviation. This is what we are going to discuss today. In this blog post, we will discuss the S&P 500 standard deviation. We will also look at its long term average and even describe the distribution of the S&P 500 standard deviation.
S&P 500 Standard Deviation – Long Term
We are using S&P 500 price data and python for data manipulation and analysis. We downloaded the daily adjusted price data of the S&P 500 from December 30, 1927, until August 12, 2021. There are 19,660 observations in our data set. Also, please note that all of the figures are on an annualized basis. Another important point is the frequency of the data. We are working with daily price data. You can do a similar analysis with weekly or monthly returns. However, it will not change the analysis significantly. To start, the S&P 500 index has an average return of 9.27%. This is a long term average. Over the same time horizon, the S&P 500 index has a standard deviation of 20.81%; a standard deviation is a measure of volatility. If the standard deviation is zero, it would mean that the return of S&P 500 never varies from 9.27%.
S&P 500 Rolling Standard Deviation
We know that standard deviation is not a constant over time. We should always look at the rolling measure to understand more about the variable. Please refer to the image below. We plot the rolling 252-days standard deviation of the S&P 500. As you can see below, the image gives us more information about the standard deviation and how it changes over time. There are periods of volatility spikes and periods of low volatility.
Let’s look at the distribution and the descriptive statistics of the rolling 252-days S&P 500 standard deviation over the same time frame:
- Minimum – 5.18%
- Maximum – 45.58%
- Mean – 14.46%
- SD – 6.51%
Medium Term Volatility Statistics
Those were the long term statistics. As there are regime changes such as government policies, taxes, rules, and regulations, all of these events affect the financial market. To better understand the volatility of the S&P 500, we should look at the recent data. Let’s analyze the last 20 years of data first and then we will look at the latest 10 years. In our opinion, the latest 10 years of volatility data should capture the normal and abnormal events and should give us a better representation of the current state of the volatility of the financial market.
Following are the two same graphs as above but for the recent 20 years of data.
And here are the statistics for the last 20 years of rolling 252-days volatility of the S&P 500:
- Minimum – 6.67%
- Maximum – 45.58%
- Mean – 17.59%
- SD – 8.57%
What you find after comparing the above data is surprising. One would assume that the general volatility in the financial market should reduce because of more robust rules and regulations and many other factors such as technological advancement in the financial market. However, we find that the average standard deviation of the S&P 500 is higher in the last 20 years at 17.59%. Not just that, but the variability of the standard deviation is 8.57% in comparison to 6.51% over the long term.
Recent Volatility Statistics
What about recent volatility? Well, let’s find out. We can look at the same data for the last 10 years. The graph will look like the end part of the above graph. Following are the statistics for the last 10 years of rolling 252-days standard deviation of S&P 500:
- Minimum – 6.67%
- Maximum – 34.69%
- Mean – 16.03%
- SD – 6.94%
Now we have somewhat expected those results. As you can see, the average rolling 1-year standard deviation of the S&P 500 in the last 10 years is 16.03%. We think this figure is more representative of the current state of the volatility of the S&P 500. When we say the current state of volatility, what we mean is that we expect the volatility of the S&P 500 to be around 16% for quite some time. However, you can still expect periods of low and high volatility. Now the question is where do we stand today? As of August 2021, the S&P 500 has a standard deviation of 14.93% over the last 1 year.
What is your opinion of the S&P 500 Standard Deviation over the next few years? Do you think we will be in a different regime than we are in right now? We are going with 16% because that is equal to the average of the 252-days rolling standard deviation for the last 10 years. And that time period includes two full business cycles from peak to trough. And there is no significant regime change like we had when we shifted the gold standard. This time period has seen good days, bad days, and ugly days. Let us know your thoughts in the comments below.
That’s it for now folks! If you have any questions, please do let us know in the comments below. Please do let us know if you want us to cover something specific in quantitative finance. If you are looking for negative beta stocks in 2021, please do check out the blog post article here. You can find the list of negative beta stocks as of August 2021 in that article.