Wednesday, March 13, 2019

Clojure on RISC-V? [not related to trading]

....by the way, this is very cool: https://github.com/eclipse/openj9/issues/5058

RISC-V is basically a serious attempt at Open Source hardware:  https://www.google.com/search?q=risc-v

Latency matters

This is simple and perhaps even obvious, but if you're simulating a fast strategy on high-resolution historical data – you at least need to simulate some average latency or the results are probably misleading.

For some exchanges, you also need to look out for weird(?) looking bulks of orders and/or order book updates that all have the exact same timestamp(!). Figuring out what actually happened here is tricky or even impossible and everything but limit orders will, I think, just be guesswork. It seems reasonable to assume that none of your order adds or updates went through during a period like this – and latency is probably higher than average just before and afterwards.

This result is only possible with 0 latency:


..the moment I add a little latency to this simulation, the diagonal blue line (PnL) is flipped!

Friday, March 8, 2019

Clojure quick performance tip: fastest(?) vector concatenation

I don't like CONCAT or MAPCAT; they create intermediate lazy seqs and have some potential problems if you're not careful. I use this instead:
(defn catvec
  ([s] (catvec [] s))

  ([init s]
   (persistent! (reduce #(reduce conj! %1 %2)
                        (transient init)
                        s))))


> (catvec [[1 2] nil [3 4 5] [6] [] [7 8]])
[1 2 3 4 5 6 7 8]


> (catvec [0] [[1 2] nil [3 4 5] [6] [] [7 8]])
[0 1 2 3 4 5 6 7 8]

..for me this yields about a 50% performance improvement compared to the perhaps more typical (vec (mapcat ..)) pattern. The key here is the use of a single transient for the entire process and no creation or use of intermediate values.

NOTE: The simplest case i.e. concatenating two vectors together is of course just   (into [0 1] [2 3]) => [0 1 2 3]  ..which uses transients internally.

I was testing rrb-vector for a while, but it seems to have some serious bugs..