SmartQuant Discussion

Automated Quantitative Strategy Development, SmartQuant Product Discussion and Technical Support Forums
It is currently Sun Apr 05, 2020 9:31 pm

All times are UTC + 3 hours




Post new topic Reply to topic  [ 8 posts ] 
Author Message
PostPosted: Wed Jan 13, 2010 12:32 pm 
Offline

Joined: Wed Sep 23, 2009 10:05 am
Posts: 66
Location: The Netherlands
Hi,

I'm running several (Dev/Test/Prod) OQ instances on Virtual Machines. The root (host) is a Windows Server 2008 R2 (64 bits) with an E5530 processor (quad core), 12 Gb Ram and 1 Tb RAID 10 Disk. Until now I ran the Virtual Machines with Windows Server 2008 (32 bits), configured each with 3 Gb Ram and 2 processors.

Because I have some instability problems with the (virtual) clock synchronization I upgraded one of the VM's to Windows Server 2008 R2 (64 bits). And than I was amazed about the performance boost of my backtests:

- GBP future, +/- 3 million trades/ticks in OQ db. The bars are built from trades. The intraday strategy generates about 3800 transactions.

with Windows Server 2008 (32 bits): +/- 113 seconds
with Windows Server 2008 R2 (64 bits): +/- 44 seconds

I didn't believed in on first hand so I double checked it on other VM's and on the root. Performance boost stays the same. Lucky me!

I know that Windows Server 2008 R2 shares most of the code with Windows 7 (64 bits) so probably these OS users gain benefit too.


With regards,
Rob.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Jan 13, 2010 2:12 pm 
Offline

Joined: Wed Sep 23, 2009 10:05 am
Posts: 66
Location: The Netherlands
Too good to be true...

I'm sorry: after upgrading my development VM to Windows Server 2008 R2 and copying all my solutions/projects/scripts the performance gain was gone...

I did forgot in the first tests to copy the solution with 6 Market Data time bars instead of only 1 bar in the test. For me it's good to explain why the Bar Factory needs more time to generate 6 bars from +/- 3 mio trades.

In fact the R2 is a little (10%) bit slower than the 32 bits OS. Probably because of the 32 bits support on the 64 bit OS.

I suggest to use this thread to start a discussion about OQ performance (experience/issues etc.).

Thanks,
Rob.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Jan 13, 2010 11:12 pm 
Offline

Joined: Thu Jun 08, 2006 3:56 pm
Posts: 537
Location: BC Canada
Too bad the performance gain didn't last. I wish my simulations only took 113 seconds.

I've been running some small backtests on forex quotes, and notice that are many more quotes than trades, so the simulations take longer. To suggest an idea for this thread, one day I hope to try to run tests while skipping every second quote (ie, don't let it get into the historical database). I figure I certainly don't need every quote, and probably could backtest fine with every 3rd quote or less (the results would show where the tradeoff point is, shortest backtest time vs most accurate results).

I imagine the same could be said for using trades on futures--you could probably skip every second trade without impacting the results very much.

If it helps any, I already know that once in a while my simulation vs paper runs sometimes miss by a single tick, which eventually leads to one winning and the other losing the day. So a difference of one single tick can make a difference. But it doesn't happen often (once every 10 or 20 days). So I think skipping every second trade during backtests might speed up the back test a lot without hurting results.

If I every get around to trying these ideas, I'll try to hunt down this thread and post the results here.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Jan 13, 2010 11:36 pm 
Offline

Joined: Wed Sep 23, 2009 10:05 am
Posts: 66
Location: The Netherlands
Hi Kevin,

Your suggestion to skip quotes or trades is more or less what I do:

The major currency pairs has lots of trades every second. I compress these trades with the same price and same timestamp (depending on your choice this can be the same second, minute, ...) by writing one trade to the OQ database with the cumulated volume.

This way I'm possible to 'compress' the trades db with 50-90% and the backtests goes a lot faster.

What I loose is a kind of depth or duration (I compare it with an orderbook) in the trades: many trades during a long period with the same price are aggregated to one trade. But like you said, in real time there are also hick up's. My opinion is that strategies must be robust enough to overcome these kind of differences.

RT will never be the same as in backtest :cry:

Regards,

Rob.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Jan 21, 2010 1:34 am 
Offline

Joined: Thu Jun 08, 2006 3:56 pm
Posts: 537
Location: BC Canada
Just an update on this performance thread. I thought I would try to replace my normal 7200 rpm harddrive with one of the new SSD (solid state disk) drives, since their I/O rates are much faster.

I thought that maybe the I/O load on pulling all those trades and quotes from the historical database might influence my simulation speeds. So I reinstalled XP on an SSD, along with OQ and several months of
trade data.

Sure enough, the SSD disk are WAY faster (at least 4x, 5x by my disk speed tests) than normal hard drives. So the usual Windows boot time(which is I/O intensive) got cut by 2x or 3x, and the OQ boot time got cut way down too (important if you're in a dev cycle working on a library DLL, because you have to quit OQ each time you recompile the library).

But the simulations were not sped up at all (3min20sec vs 3min10sec or so). My simulations are definitely CPU bound, no question about that. I have a dual core machine, and one of the cores is just flattened. Pulling all those trades from the historical database is not worth talking about in terms of overall simulation speed.

So I'll be sure to use SSD drives on my next machine for IO/boot speed, heat, and reliability.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Jan 21, 2010 11:59 am 
Offline

Joined: Wed Sep 23, 2009 10:05 am
Posts: 66
Location: The Netherlands
I had the same experience that only one core/processor is used during optimization. And for long backtests (more than a few hours) it's a waiste of resources.

Therefore I use several Virtual Machines (created with Virtual PC or Virtual Server/Hyper-V, as many as you have cores/processors), split the backtest in several parts and run the different backtest parts simultaniously on the different VM's. The turnaround time decreases almost with the same factor as the number of VM's.

Regards,
Rob.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Feb 03, 2010 8:05 pm 
Offline

Joined: Thu Jun 08, 2006 3:56 pm
Posts: 537
Location: BC Canada
Just an update on another way to get more performance -- upgrade your computer. I just finished building a new machine with a new i7 860 2.8GHz quad-core processor (4GB DDR3-1333 memory), ASUS P7P55D-E PRO motherboard, with a little 120 GB SSD disk (233Mb/sec max transfer speed). The SSD disk helps OS boot times, and program loading times, but obviously not simulation (CPU bound) times.

The new machine runs simulations about 2x faster than my old machine did (AMD 5200 core duo). Numbers were about 230 seconds (old machine) versus 110 seconds (new machine). My little backtests are nothing like Rob's hours-long backtests, but still, it's nice to see that a 5-year jump forward into newer technology can have such a big effect.

Here is a screenshot of what the CPUs are doing during the simulation. There are 4 cores, each running 2 threads. While most of the OQ work is being done in the first thread, there are 3 other threads doing some real work (20% loaded) too. So multi-core, multi-threaded machines are useful, even if the OQ simulation engine is single threaded.


Attachments:
File comment: Here is what my 8 threads on 4 cores are doing. OQ uses mostly the first core, but three other threads/cores are doing something too, definitely.
Backtest Threads.PNG
Backtest Threads.PNG [ 51.69 KiB | Viewed 4912 times ]
Top
 Profile  
 
 Post subject:
PostPosted: Tue Mar 02, 2010 7:21 pm 
Offline

Joined: Thu Jun 08, 2006 3:56 pm
Posts: 537
Location: BC Canada
Here are some numbers for a little overclocking experiment that I did. The bottom line is that overclocking seems highly overrated for simulations, because I really had to push my machine to the limits in order to see any significant simulation speedup. Here are the results for my machine (i7 860 2.8Mhz cpu, 4GB available memory, solid state hard disks, Win7 x64 Pro, 8 threads available):

Code:
Overclock results
Passmark.com score

OClevel  CpuMHz PassmarkScore  %Gain    SimTime  Sim%Gain CpuTemp
0         2.79     1969         0        185 sec     0%    ~ 40 C
1         2.94     2083        +5.7%     185 sec     0%    ~ 42 C
2         3.15     2324        +18%      185 sec     0%    ~ 42 C
3         3.80     2449        +24%      155 sec    18%    ~ 50 C


As you can see, the overclocking made consistent improvements in the benchmark scores that can adequately test such things. But there was essentially no difference in simulation times until I reached the very highest overclock rates (turning my 2.8Mhz cpu into a 3.8MHz cpu).

I was not happy adding 10C to the cpu temps just to save 30 seconds on my simulations, so I'm running default settings. Maybe overclocking is noticeable on some other non-OQ-simulation computational mix.

BTW, OQ seems to work fine on both Windows 7 x32 and Windows 7 x64. There are a couple of small GUI window alignment issues, but for the most part, you can just install and run OQ as usual. The most noticeable thing for me was that under the default Win7 color schemes, OQ looks washed out because the XP-blue within the interface is mostly replaced by white/grey colors instead. But OQ still works, and that is the main thing.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 8 posts ] 

All times are UTC + 3 hours


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group