More Macros and DSLs

Clojure

2026-08-20 11:00

Where we are

The same move, twice

Last unit: closures pushed until they became objects — then we chose not to use them.

This unit: macros pushed until they become languages.

From constructs to languages

The earlier macro unit added unless, randomly, assert-true.

This one:

Anaphoric macros
Compile-time computation
Macro-generating macros
Domain-specific languages

Where Lisp has always pointed

When a problem is awkward in your language —

build a better one for it.

A quick review

Compressed

a macro runs at macroexpansion time
arguments arrive as unevaluated forms
the return value is a form
templates use `, ~, ~@
macroexpand shows what was produced
name# prevents capture

This unit builds on all of it and re-derives none of it.

One rule we are about to break

The earlier unit taught that an un-gensymed binding is a bug.

Variable capture.

Hold onto that. The next section does it on purpose.

Anaphoric macros

An anaphor

A word referring back to something earlier.

take the report and file it

The problem

(defn some-computation [x]
  (if (even? x) false (inc x)))

(if (some-computation 11)
  (* 2 (some-computation 11)))
;=> 24

It runs twice.

The usual fixes

(let [computation (some-computation 11)]
  (if computation
    (* 2 computation)))

(if-let [computation (some-computation 11)]
  (* 2 computation))

Better — but you invented a name and wrote it three times.

What we want

(anaphoric-if (some-computation 11)
  (* 2 it))

Implementing it

(defmacro anaphoric-if [test-form then-form]
  `(if-let [~'it ~test-form]
     ~then-form))

~'it inserts the unqualified symbol.

A bare it would become user/it and be useless.

Deliberately unhygienic.

It works

(macroexpand-1 '(anaphoric-if (some-computation 11) (* 2 it)))
;=> (clojure.core/if-let [it (some-computation 11)] (* 2 it))

(anaphoric-if (some-computation 12) (* 2 it))  ;=> nil
(anaphoric-if (some-computation 11) (* 2 it))  ;=> 24

Generalise over the operator

(defmacro with-it [operator test-form & exprs]
  `(let [~'it ~test-form]
     (~operator ~'it ~@exprs)))

(with-it if   (some-computation 11) (* 2 it))            ;=> 24
(with-it and  (some-computation 11) (> it 10) (* 2 it))  ;=> 24
(with-it when (some-computation 11)
  (println "Got it:" it)
  (* 2 it))

The cost

The binding is invisible at the call site.

A reader who does not know the macro cannot tell where it came from.

And any it the caller already had is shadowed.

Threading in any position

Fixed positions

(defn surface-area-cylinder [r h]
  (-> r
      (+ h)
      (* 2 Math/PI r)))

(defn some-calculation [a-collection]
  (->> (seq a-collection)
       (filter some-pred?)
       (map a-transform)
       (reduce another-function)))

Where they fail

(defn another-calculation [a-collection]
  (->> (seq a-collection)
       (filter some-pred?)
       (map a-transform)
       (#(compute-averages-from % another-pred?))))

That #() exists only to move the value into the right slot.

Clojure’s answer: as->

(defn another-calculation [a-collection]
  (as-> (seq a-collection) result
    (filter some-pred? result)
    (map a-transform result)
    (compute-averages-from result another-pred?)))

Each step binds to result. Put it where it belongs.

The anaphoric version

(defn yet-another-calculation [a-collection]
  (thread-it (seq a-collection)
    (filter some-pred? it)
    (map a-transform it)
    (compute-averages-from it another-pred?)))

(->> )
ArityException Wrong number of args (0)

(thread-it)
;=> nil

Recursive expansion

(defmacro thread-it [& [first-expr & rest-expr]]
  (if (empty? rest-expr)
    first-expr
    `(let [~'it ~first-expr]
       (thread-it ~@rest-expr))))

Each step binds it and recurses — nested let forms.

Shifting computation to compile time

The cipher, without macros

(def ALPHABETS [\a \b \c ... \z])
(def NUM-ALPHABETS (count ALPHABETS))
(def INDICES (range 1 (inc NUM-ALPHABETS)))
(def lookup (zipmap INDICES ALPHABETS))

(defn shift [shift-by index]
  (let [shifted (+ (mod shift-by NUM-ALPHABETS) index)]
    (cond
      (<= shifted 0) (+ shifted NUM-ALPHABETS)
      (> shifted NUM-ALPHABETS) (- shifted NUM-ALPHABETS)
      :else shifted)))

The tableau

(defn shifted-tableau [shift-by]
  (->> (map #(shift shift-by %) INDICES)
       (map lookup)
       (zipmap ALPHABETS)))

(shifted-tableau 13)
;=> {\a \n, \b \o, \c \p, \d \q, \e \r, ...}

(defn encrypt [shift-by plaintext]
  (let [shifted (shifted-tableau shift-by)]
    (apply str (map shifted plaintext))))

(encrypt 13 "abracadabra")
;=> "noenpnqnoen"

It is reciprocal

(encrypt 13 "noenpnqnoen")
;=> "abracadabra"

(def encrypt-with-rot13 (partial encrypt 13))
(def decrypt-with-rot13 (partial decrypt 13))

What is wasteful

shifted-tableau runs on every call —

though it depends only on a constant.

memoize helps, but it still runs at least once.

What we actually want

(defn encrypt-with-rot13 [plaintext]
  (apply str (map {\a \n \b \o \c \p} plaintext)))

The table written into the code.

The macro

(defmacro def-rot-encrypter [name shift-by]
  (let [tableau (shifted-tableau shift-by)]
    `(defn ~name [~'message]
       (apply str (map ~tableau ~'message)))))

Computes the tableau during expansion and embeds the result.

The expansion

(macroexpand-1 '(def-rot-encrypter encrypt13 13))
;=> (clojure.core/defn encrypt13 [message]
      (clojure.core/apply clojure.core/str
        (clojure.core/map {\a \n, \b \o, \c \p, \d \q, \e \r, \f \s,
                           ... \z \m} message)))

An inline literal map.

The flow

Macros expand during reading; the generated tableau lands in the source as a literal.

What that buys

the new encrypt13 function at runtime doesn’t do any tableau computation at all

Ship it as a library and users would never know shifted-tableau was called.

When it is worth doing

Work that is expensive, repeated, and determined by compile-time constants.

All three. And no function can do it, at any level of cleverness.

Macros that write macros

The premise

A macro returns code.

A defmacro is code.

So a macro can return a defmacro.

The goal

(make-synonym b binding)
;=> #'user/b

(b [x 10 y 20] [x y])
;=> [10 20]

The template

Written by hand, b would be:

(defmacro b [& stuff]
  `(binding ~@stuff))

So make-synonym must produce exactly that.

The naive attempt

(defmacro make-synonym [new-name old-name]
  `(defmacro ~new-name [& stuff]
     `(~old-name ~@stuff)))

(macroexpand-1 '(make-synonym b binding))
;=> (clojure.core/defmacro b [& user/stuff]
      (clojure.core/seq (clojure.core/concat
        (clojure.core/list user/old-name) stuff)))

user/stuff and user/old-name. Both wrong.

What a nested backquote does

(defmacro back-quote-test []
  `(something))
(macroexpand '(back-quote-test))
;=> (user/something)

One backquote produces the form.

Two produce code that produces the form — hence all the concat and list.

The working version

(defmacro make-synonym [new-name old-name]
  `(defmacro ~new-name [& ~'stuff]
     `(~'~old-name ~@~'stuff)))

(macroexpand-1 '(make-synonym b binding))
;=> (clojure.core/defmacro b [& stuff]
      (clojure.core/seq (clojure.core/concat
        (clojure.core/list (quote binding)) stuff)))

That odd `~'~old-name`

first, ~old-name is expanded, leaving ~'binding … Then the outer backquote is expanded, leaving 'binding, which finally becomes (quote binding)

Needed so the value is not resolved until the generated macro expands.

How to write these

Not by reasoning about quote levels in your head.

Macroexpand one level at a time and compare against the hand-written template.

When it is warranted

Not for one macro. Write that macro.

It pays for a family of near-identical macros —

which is exactly what a DSL is.

Domain-specific languages

Two design considerations

Decomposition — top-down breaks the problem down; bottom-up builds vocabulary up.

Combinability — vocabulary you can combine covers cases you did not anticipate.

The problem

Segment users of a website by what they did.

Session persistence

(ns clj-in-act.ch11.session
  (:require redis))

(def redis-key-for :consumer-id)
(def ^:dynamic *session*)

(defn save-session [session]
  (redis/set (redis-key-for session) (pr-str session)))

(defn find-session [consumer-id]
  (read-string (redis/get consumer-id)))

(defmacro in-session [consumer-id & body]
  `(binding [*session* (find-session ~consumer-id)]
     (do ~@body)))

The language we want

(defsegment googling-clojurians
  (and
    (> (count $search-terms) 0)
    (matches? $url-referrer "google")))

(defsegment loyal-safari
  (and
    (empty? $url-referrer)
    (= :safari $user-agent)))

Read those as a domain expert

They say what a segment is.

Not how to compute it.

No Redis. No sessions. No lookups.

How it is built

(defmacro defsegment [segment-name & body])

The body is walked with clojure.walk/postwalk, and every $attribute is rewritten into a lookup against *session*:

(defn transform-lookups [dollar-attribute]
  ...)

What had to be a macro

Only two things.

defsegment — it defines a name and must receive the rule unevaluated.

in-session — it establishes a binding around a body.

Everything else is ordinary functions and data.

The lesson

A DSL built mostly from macros is usually a DSL built wrong.

Metalinguistic abstraction

When a problem is awkward, build a better language for it.

Most languages do not offer this option.

Which makes the discipline of not doing it part of the skill.

What you can now build

Every technique has a cost

an anaphoric macro introduces a binding from nowhere
compile-time computation means the code that runs is not the code you wrote
a macro-generating macro is hard to read even when correct
a DSL must be learned before any program in it can be read

Against that

A good DSL makes the problem visible instead of the plumbing.

And lets people who understand the domain read and check the rules.

The test

Would a competent Clojure programmer, new to this codebase, be able to read it?

Easier because the domain vocabulary is clearer? Build it.

Harder because they must learn your language first? Write the functions.

Summary

The six things to carry away

Anaphoric macros invert the hygiene rule on purpose.
thread-it threads into any position, not only first or last.
Only a macro can move work to compile time.
A macro can write a macro; the difficulty is entirely in the quoting.
A good DSL is mostly functions and data.
Metalinguistic abstraction is a real option and a real cost.

Where the module ends

From reading your first parenthesis —

to building languages.

The unifying fact

True since unit 1:

Your program is data, and you can compute with it.

More Macros and DSLs

Where we are

The same move, twice

From constructs to languages

Where Lisp has always pointed

A quick review

Compressed

One rule we are about to break

Anaphoric macros

An anaphor

The problem

The usual fixes

What we want

Implementing it

It works

Generalise over the operator

The cost

Threading in any position

Fixed positions

Where they fail

Clojure’s answer: as->

The anaphoric version

Recursive expansion

Shifting computation to compile time

The cipher, without macros

The tableau

It is reciprocal

What is wasteful

What we actually want

The macro

The expansion

The flow

What that buys

When it is worth doing

Macros that write macros

The premise

The goal

The template

The naive attempt

What a nested backquote does

The working version

That odd ~'~old-name

How to write these

When it is warranted

Domain-specific languages

Two design considerations

The problem

Session persistence

The language we want

Read those as a domain expert

How it is built

What had to be a macro

The lesson

Metalinguistic abstraction

What you can now build

Every technique has a cost

Against that

The test

Summary

The six things to carry away

Where the module ends

The unifying fact

That odd `~'~old-name`