clj-async-profiler Rocks

While developing the live reload feature for Clay, a minimalistic Clojure tool for data visualization and literate programming, I found that it constantly takes ~1 minute for beholder to watch a directory. After some code inspect, I was still having no idea why it happened.

So I decided to use a profiler to find out what's going one under the hood, and Google immediately took me to clj-async-profiler. It's easy to set up following its basic usage docs:

Configure your system to allow profiling, and here's how on Linux:

sudo sysctl -w kernel.perf_event_paranoid=1
sudo sysctl -w kernel.kptr_restrict=0

Add deps and jvm-opts:

;; add below to deps.edn
{:deps {com.clojure-goes-fast/clj-async-profiler {:mvn/version "1.4.0"}}
 :aliases
 {:attach {:jvm-opts ["-Djdk.attach.allowAttachSelf"]}}}

Start the REPL with the additional settings, if you're using Cider, type C-u C-c C-j and then add -A:attach to the clojure CLI command after the clojure binary, that is, clojure -A:attach -Sdeps \{\:deps\ \{nrepl/nrepl\ \{\:mvn/version\ \"1.3.0\"\}\ cider/cider-nrepl\ \{\:mvn/version\ \"0.50.1\"\}\} ...

Now, we can profile and show the result in a web browser running two steps in clojure, see this demo code below:

(ns beholder-profile
  (:require [nextjournal.beholder :as beholder]
            [clj-async-profiler.core :as prof]))

(defn- watch-and-stop [dir times]
  (dotimes [_ times]
    (let [beholder (beholder/watch prn dir)]
      (beholder/stop beholder))))

(comment
  ;; step 1
  (prof/profile (watch-and-stop "/home/whatacold/tmp/" 2))
  ;; step 2
  (prof/serve-ui 8010)
  )

/img/2024-11-28-beholder-watch-flamegraph.png — The flamegraph for beholder

From the flamegraph, I immediately realized that the cause was that my directory has a lot of files and it took beholder a long time to traverse them!

clj-async-profiler is really amazing, using it to troubleshoot performance problems is like a breeze! Thank you to Oleksandr Yakushev and all the contributors for this great tool!

Notes:

Use built-in time function to get a basic idea of how long of time a form takes, e.g. (time (+ 1 2))
If you want to save the flamegraph file to your computer, right-click on the link on its home page and save it from there. It doesn't work to directly save it from the flamegraph page, somehow, the resulting html file is different, it looks like it has been "tempered" by the browser aftering rendering.
Beholder's current implementation needs to hash a files under a directory in order to watch changes, see issue #11

Clojure Clay

See also