Friday, November 2, 2018

Notes on Clojure: memory leaks, performance and more

work in progress; i'll add more to this later

Memory leaks

  1. https://www.eclipse.org/mat/ is great and works with Clojure.
  2. When debugging, setting -Dclojure.compiler.disable-locals-clearing=true can be nice, but make sure you set this back to false (the default) because it can (will?) create memory leaks in non-trivial code.
  3. In some cases you should use eduction instead of sequence when dealing with transducers and huge amounts of data.
  4. Not related to Clojure, but Emacs actually has some leak problems lately: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=26952 so make sure you run a recent Emacs. This is not a problem that bit me often though.
Spent the last 2 days debugging memory. Turns out 90% of the problem was a wrong JVM flag (#1 above) -- and the remaining 10% of the problem was an issue with clojure.core.async which I'll try to document later. 

Performance

  1. -Dclojure.compiler.direct-linking=true leads to massive performance improvements in some cases. You'll need to change how you do interactive development after setting this flag tho. I also find return value type hints work better when this is true -- which can help with debugging.
  2. Make sure you set *warn-on-reflection* to true and *unchecked-math* to :warn-on-boxed. Dealing with this correctly will probably give you the biggest performance of everything else mentioned here.
  3. It is very important to not use Fns like FIRST on ArrayLists, ArrayDeques and similar. As it will actually generate a temporary object via an internal call to SEQ on every invocation(!). This might be obvious, but it's easy to forget because a simple call to FIRST seems like something that "should" be cheap to do. Stick with the JVM specific methods and interfaces for these classes; get, peekFirst, peekLast etc..
  4. Also, Vector implements IFn which means you can do (my-vector 0) instead of (first my-vector) for a 4-5x win.
  5. Always use PEEK instead of LAST when you can.
  6. This looks useful: https://github.com/clojure-goes-fast
 ..I'll add more tips about performance in future posts, here: https://quantoga.blogspot.com/search/label/performance

No comments:

Post a Comment