Let's take a look at the statistics for this new mystery strategy:

With this strategy, I have essentially doubled the Sharpe ratio over buy-and-hold, lowered the volatility substantially, increased the max win, and reduced the max loss by a significant level.

And how is it that I devised this market-trouncing strategy? Wait for it... I did it by generating 5,000 random overnight signals and picked the best one.

This is obviously not the way to beat the market. So why then did I do it? To demonstrate that, if you test enough strategies, just by random chance you will come across a number that appears to be amazing. This is the called the data mining fallacy, and it is a real risk in trading strategy development. That is why it is so important to find a strategy that is anchored to real-world investor biases and behaviors. If you want an edge in trading, you don't trade the markets; you the trade people who trade markets.

An edge then comes from thoughtfully understanding how people might react incorrectly to certain situations.

Let's now extend our analysis. First, we'll pull data for the index beginning with the year 2000:

start_date = pd.to_datetime('2000-01-01') 
stop_date = pd.to_datetime('2018-12-01') 
 
sp = pdr.data.get_data_yahoo('SPY', start_date, stop_date) 

Let's now take a look at our chart:

fig, ax = plt.subplots(figsize=(15,10)) 
sp['Close'].plot(color='k') 
plt.title("SPY", fontsize=20) 

This generates the following output:

Here, we see the price action for the SPY from the start of 2000 until December 1, 2018. There has certainly been a lot of movement during that period as the market has experienced both highly positive and highly negative regimes.

Let's get our baseline for our new expanded period for our three base strategies.

First, let's set up our variables for each:

long_day_rtn = ((sp['Close'] - sp['Close'].shift(1))/sp['Close'].shift(1))*100 
 
long_id_rtn = ((sp['Close'] - sp['Open'])/sp['Open'])*100 
 
long_on_rtn = ((sp['Open'] - sp['Close'].shift(1))/sp['Close'].shift(1))*100 

Now, let's see what the point totals are for each:

(sp['Close'] - sp['Close'].shift(1)).sum() 

This generates the following output:

Now, let's see what the point totals are for open to close:

(sp['Close'] - sp['Open']).sum() 

This generates the following output:

Now, let's see what the point totals are for close to open:

(sp['Open'] - sp['Close'].shift(1)).sum() 

This generates the following output:

And now let's look at the statistics for each:

get_stats(long_day_rtn) 

This generates the following output:

Now, let's look at the statistics for intraday returns:

get_stats(long_id_rtn) 

This generates the following output:

Now, let's look at the statistics for overnight returns:

get_stats(long_on_rtn) 

This generates the following output:

We can see that the differences between the three are even more pronounced over the longer period. Had you only held during the day over the past 18 years, you would have lost money in this S&P ETF. Had you held only overnight, you would have improved your total point returns by over 18%! Obviously, this presumes no trading costs and no taxes along with perfect fills but, regardless, this is a remarkable finding.