SmartQuant Discussion
http://www.smartquant.com/forums/

suspect backtest - how to improve realism?
http://www.smartquant.com/forums/viewtopic.php?f=43&t=9678
Page 1 of 1

Author:  aburnettthompson [ Fri Sep 02, 2011 11:39 pm ]
Post subject:  suspect backtest - how to improve realism?

Hi all,

I've developed a strategy using imported EURUSD Daily OHLC bars. It's taking trades too close together to give a decent return IMO and the equity curve looks suspiciously good.

Here's an image of the chart with buy/sells:

Image

And the equity curve:

Image

This test is done without slippage/commissions. With these enabled the equity curve is more volatile but still viable overall. I'm wondering though how the backtester actually works - how is it simulating the transition from Open to High, Close to Low? If I'm getting buys and sells within a 1 or 2 bar timeframe of each other it suggests to me the system performance in the backtester is unrealistic.

Is there a way to improve this? Can I backtest on daily chart but use a lower granularity of data to get a finer resolution result?

Is there anything else I can do to ensure the robustness of my strategy?

Thanks,

Author:  Dr. Anton Fokin [ Sat Sep 03, 2011 9:43 am ]
Post subject:  Re: suspect backtest - how to improve realism?

Hi,

I believe we need to have more info about your strategy to suggest something useful. For example if you have a pure end of day strategy that only sends market orders on day close, then your results are more or less realistic. If you try to simulate intraday strategy that uses limit orders and you simulate this strategy with (daily) bars only, then the best way is to use quotes (or at least intraday bars) to simulate execution.

PS. Note that you should set BarFilter in the execution simulator to use specific bar series for execution simulation if you have two or more bar series in your strategy. Also don't forget to set FillOnBar to false if you use quotes.


Regards,
Anton

Author:  aburnettthompson [ Sat Sep 03, 2011 3:23 pm ]
Post subject:  Re: suspect backtest - how to improve realism?

Hi Anton,

Thanks for your response. Ok a little more info about the strategy. It basically seeks to identify a trend then take short term trades in the direction of the trend.

The entry order is a Market order executed on bar close (in the OnBar method). At the same time it places an OCA stop/limit order.
The Stop for 100% of the position is typically below a recent low point.
The Limit for 50% of the position is a fixed distance (profit target) away from the entry.

When either stop/limit triggers the other is cancelled. If the limit (profit) was triggered it then places a trailing stop to capture as much of the resulting trend as possible for the other 50% of the position. This final logic for trailing isn't quite working as you can see by the screens, its not capturing much. However it is typically a higher exit than the first profit target.

So - entry Market order, exit stops/limits. I have a series of daily OHLC for the EURUSD which I am testing against. I also have hourly bars going back to 2000 and 15 minute bars going back to 2007. I don't have quote data for EURUSD but I am sure I could get finer granularities of data if I need to.

On the hourly curve this produces an extremely smooth equity curve however when adding slippage of 3-4 pips it causes the system to take a heavy loss. On the daily chart it can survive slippage of 5 pips + $5 roundtrip commissions with ease.

What I'm looking for really is this - is it possible using the method you specified to test the strategy on the daily chart, but the internals of the backtesting engine use a lower granularity of data (e.g. 1 hour, or 15 min bars) to give the Limit orders a more thorough road test?

Thanks :)

Author:  Dr. Anton Fokin [ Sat Sep 03, 2011 4:49 pm ]
Post subject:  Re: suspect backtest - how to improve realism?

aburnettthompson wrote:
What I'm looking for really is this - is it possible using the method you specified to test the strategy on the daily chart, but the internals of the backtesting engine use a lower granularity of data (e.g. 1 hour, or 15 min bars) to give the Limit orders a more thorough road test?


Sure, take a look at this FAQ viewtopic.php?f=64&t=6358 and also don't forget what I said about using BarFilter in the ExecutionSimulator.

Regards,
Anton

Author:  aburnettthompson [ Sat Sep 03, 2011 9:18 pm ]
Post subject:  Re: suspect backtest - how to improve realism?

Hi Anton,

That's not really what I'm after - although it is very helpful to know how to do multi-timeframe strategies (Thats next on my list!) :)

What I meant was, the backtester, Im sure it does some interpolation. Say the backtester only has OHLC for a day. If you set a limit for the day which is filled intraday what is the assumption made by the backtester? I would imagine it would be a sinusoidal-like function which starts at O goes to L, then H then C (for an up-day) or O->H->L->C for a down day.

This may result in an overly optimistic backtest. Ok fine its the best you can do with the limited data. What I meant was, can you somehow instruct the backtester that for your strategy that uses daily OnBarClose only, to simulate the intraday behaviour if your instrument is defined with say 5 minute bars as well as daily?

By the way, I tried the BarFilter, enabling it and adding an entry for 1-hour bars (which I have 10 years of data for) as well as the daily data. My backtest executed and took zero trades? What exactly is the BarFilter doing?

Best regards,
ABT

Author:  Dr. Anton Fokin [ Sat Sep 03, 2011 9:24 pm ]
Post subject:  Re: suspect backtest - how to improve realism?

Have you added both 1hour and 1day data to your solution market data?

Author:  PBJ [ Thu Nov 03, 2011 4:07 am ]
Post subject:  Re: suspect backtest - how to improve realism?

I guess aburnettthompson's question is:
Does adding a second, lower granularity market data type to a solution force OQ's backtesting engine to use the lowest granularity when evaluating trade execution?

An example would be: if I'm using hourly bars as main bar type to compute indicators and run the trading logic but also add tick data (bid/ask) to my solution, does OQ simulate execution using tick data, taking the real bid-ask spread into account or does it just simulate execution using hourly bars?

Author:  Dr. Anton Fokin [ Thu Nov 03, 2011 8:01 am ]
Post subject:  Re: suspect backtest - how to improve realism?

Indeed it would take ticks into account. You can also simulate with market depth (order book) if you have such data in the database...

Author:  PBJ [ Thu Nov 03, 2011 10:52 am ]
Post subject:  Re: suspect backtest - how to improve realism?

Thank you Anton.

If anyone on the forum has experience to share in this area (backtest on 2 different granularities in order to achieve the most realistic backtest execution, not with the goal to manage a multiple timeframes logic), it is welcome.

Author:  Dr. Anton Fokin [ Thu Nov 03, 2011 9:26 pm ]
Post subject:  Re: suspect backtest - how to improve realism?

Do you understand the meaning of BarFilter by the way? If you have two or more barseries in your strategy (and no ticks, i.e. you simulate execution using bars), you should use the BarFilter to select bar size that you want to use for execution simulation. Otherwise you can get pretty messy results. It was discussed several times on this forum, so that you can try to search for BarFilter to get more info.

Regards,
Anton

Author:  PBJ [ Fri Nov 04, 2011 2:16 am ]
Post subject:  Re: suspect backtest - how to improve realism?

Thank you Anton.

I had never dug into this topic before because this is not a concern for my current strategies and goals.
It's specifically because I'm reading lots of posts on this forum that I came accross that thread and thought I'd improve my general culture about OQ's logic and approach.

Kind Regards,
PBJ

Page 1 of 1 All times are UTC + 3 hours
Powered by phpBB® Forum Software © phpBB Group
https://www.phpbb.com/