Lecture notes — Evolving Clojure Through Macros

Published

2026-08-20 00:00

Keywords

ver. 1.0.0

Where we are

In State and the Concurrent World you saw how immutability dissolves most concurrency problems. That unit finished a thread running since unit 1 — the separation of values from identities.

This unit picks up a different one. Introducing Clojure claimed:

your entire Clojure program is a series of lists: the very source code of your program is interpreted by the Clojure compiler as lists that contain function names and arguments.

and observed that because the same language features exist at the compiler level and in ordinary code, Lisp enables uniquely powerful metaprogramming. Unit 2 added a second hint: cond is a nested list of pairs, and generating such a list is easy.

This unit collects on both. Everything so far has been about using a language. This is about changing it.

What you will be able to do

explain-the-runtime-phases — Explain Clojure’s read, macroexpand, compile and run phases, and where macros act.
explain-homoiconicity — Explain why macros are possible in Lisp and awkward everywhere else.
say-why-a-function-cannot-do-it — Explain why some constructs cannot be functions, using the unless example.
write-a-basic-macro — Write a macro with defmacro and inspect what it produces.
use-syntax-quote-and-unquote — Use syntax quote, unquote and unquote-splicing to build a macro template.
avoid-variable-capture — Avoid variable capture with auto-gensym.
read-clojures-own-macros — Read the definitions of macros in Clojure’s own core library.
write-your-own-macros — Write macros that add genuinely new constructs to the language.
judge-when-to-use-a-macro — Judge when a macro is warranted and when a function would do.

What we will cover

Homoiconicity — code as data, and why that makes macros ordinary.
Macros — functions that run at macroexpansion time on unevaluated forms.
macroexpand — inspecting what a macro produced.
Syntax quote, unquote, unquote-splicing — writing a macro as a template.
Auto-gensym — generating unique names.
Variable capture — the bug auto-gensym prevents.

The promise from unit one

The route this unit takes:

The phases — read, macroexpand, compile, run, and where a macro acts
Why a function cannot do it — the unless example, which fails instructively
Templates — syntax quote, unquote, unquote-splicing, and auto-gensym
Clojure’s own macros — comment, declare, defonce, and, time
Your own — five macros, each demonstrating a different technique
Judgement — when a macro is warranted, and what it costs

Learning outcomes

explain-homoiconicity: Explain why macros are possible in Lisp and awkward everywhere else.
explain-the-runtime-phases: Explain Clojure’s read, macroexpand, compile and run phases, and where macros act.

The phases a form passes through

Nearly every macro confusion is a phase confusion, so start here.

The phases of the Clojure runtime. This separation is what makes the macro system possible.

Read — text becomes Clojure data structures: lists, vectors, symbols, keywords. Reader macros act here — the quote, #(), tagged literals. Nothing has been evaluated; there is no function call yet, only a list whose first element happens to be a symbol.
Macroexpand — the compiler walks the forms. Where the first element of a list names a macro, it calls that macro as an ordinary function, passing the remaining forms unevaluated, as data. The return value replaces the original, and the process repeats until no macros remain.
Compile — the expanded forms become JVM bytecode.
Run — the bytecode executes.

Textual substitution

The simplest way in. Suppose you have a ref:

(def a-ref (ref 0))

and want to set it, which requires a transaction:

(dosync
  (ref-set a-ref 1))

That is verbose if you do it often. You would rather write:

(sync-set a-ref 1)

A macro can do exactly that — build the longer form from the shorter one:

(defmacro sync-set [r v]
  (list 'dosync
        (list 'ref-set r v)))

Look at what the body does. It is an ordinary function that builds a list. The first element is the symbol dosync, quoted so it is not evaluated. That list is the code that will run.

Two consequences to hold onto

A macro’s arguments are forms, not values. (sync-set a-ref 1) hands the macro the symbol a-ref, not the ref it names. That is the entire source of a macro’s power.

Macros have no runtime existence. By the time your program runs, every macro call has been replaced by its expansion. A macro cannot be passed to map, stored in a collection or called dynamically — it is not a value. The final section returns to that cost.

Learning outcomes

explain-the-runtime-phases: Explain Clojure’s read, macroexpand, compile and run phases, and where macros act.
explain-homoiconicity: Explain why macros are possible in Lisp and awkward everywhere else.

Concepts

homoiconicity: explains why code being data is what makes the macro system possible
macros: introduces macros as functions running at macroexpansion time on unevaluated forms

Why unless cannot be a function

The pivotal section, and the failure is more instructive than the fix.

Clojure has if, and sometimes you want its inverse:

(defn exhibits-oddity? [x]
  (if (odd? x)
    (println "Very odd!")))

reads slightly better as:

(defn exhibits-oddity? [x]
  (unless (even? x)
    (println "Very odd, indeed!")))

The attempt

(defn unless [test then]
  (if (not test)
    then))

Test it with an odd number:

(exhibits-oddity? 11)
Very odd, indeed!
;=> nil

Correct. Now an even number:

(exhibits-oddity? 10)
Very odd, indeed!
;=> nil

Wrong. It printed anyway.

Why

Clojure evaluates arguments before calling a function. By the time unless receives then, (println "Very odd, indeed!") has already run and produced nil. The function cannot un-print it — it only ever sees the nil that was left behind.

Worse, this failure is quiet when the body is pure. It becomes visible only when the body has a side effect or is expensive, which is exactly when you care.

The workaround, and why it is not good enough

You can delay evaluation by wrapping the body in a function:

(defn unless [test then-thunk]
  (if (not test)
    (then-thunk)))

(defn exhibits-oddity? [x]
  (unless (even? x)
    #(println "Rather odd!")))

This works:

(exhibits-oddity? 11)
Rather odd!
;=> nil
(exhibits-oddity? 10)
;=> nil

But it forces every caller to remember the #(). Forget it once and the bug is back, silently.

The macro

(defmacro unless [test then]
  (list 'if (list 'not test)
        then))

Now callers write ordinary code, and you can see exactly what happens to it:

(macroexpand '(unless (even? x) (println "Very odd, indeed!")))
;=> (if (not (even? x)) (println "Very odd, indeed!"))

The body was never evaluated — it was moved into an if.

macroexpand, macroexpand-1 and macroexpand-all

macroexpand-1 performs one expansion step. macroexpand repeats until the outermost form is no longer a macro call. macroexpand-all (from clojure.walk) expands everything, including nested forms.

None of them run the result. Reading the expansion is how you debug a macro — a macro producing wrong code usually produces a confusing runtime error, and the expansion shows the actual problem.

The general rule

A function receives values. A macro receives forms. So:

must it decide whether to evaluate something? Not a function.
must it decide when, or how many times? Not a function.
does it only compute with the values it is given? Use a function.

Learning outcomes

say-why-a-function-cannot-do-it: Explain why some constructs cannot be functions, using the unless example.
write-a-basic-macro: Write a macro with defmacro and inspect what it produces.

Concepts

macros: shows a construct that cannot be a function, and the macro that replaces it
macroexpand: introduces the expansion tools as the way to inspect and debug a macro

Templates and hygiene

Building lists by hand with list and quoted symbols works and does not scale. Compare the unless macro above with what we want to write.

Syntax quote

The backquote quotes a form like ' does, with two differences:

(defmacro unless [test then]
  `(if (not ~test)
     ~then))

symbols are resolved to their namespaces, so not becomes clojure.core/not and cannot be shadowed by whatever the caller has defined
the form becomes a template, into which values can be inserted

Unquote

~form evaluates form and inserts the result. Forget it and the symbol is inserted literally:

(defmacro unless [test then]
  `(if (not ~test)
     then))

(macroexpand '(unless (even? x) (println "Very odd, indeed!")))
;=> (if (clojure.core/not (even? x)) user/then)

user/then — the macro’s own parameter name, namespace-qualified, instead of the caller’s code.

Splicing

Now let the macro take several body expressions:

(defn exhibits-oddity? [x]
  (unless (even? x)
    (println "Odd!")
    (println "Very odd!")))

Try it with ~:

(defmacro unless [test & exprs]
  `(if (not ~test)
     (do ~exprs)))

(exhibits-oddity? 11)
Odd!
Very odd!
NullPointerException  user/exhibits-oddity? (NO_SOURCE_FILE:4)

It printed both lines and then threw. The expansion says why:

(macroexpand-1 '(unless (even? x)
                  (println "Odd!")
                  (println "Very odd!")))
;=> (if (clojure.core/not (even? x))
      (do ((println "Odd!") (println "Very odd!"))))

Look at the double parentheses. ~exprs inserted the list of expressions as a single form, so after both printlns evaluate to nil, the result is (nil nil) — an attempt to call nil as a function.

~@ splices the elements instead:

(defmacro unless [test & exprs]
  `(if (not ~test)
     (do ~@exprs)))

~ inserts the sequence; ~@ inserts its elements. This confuses everyone once. The way to settle it is to macroexpand both and look.

Variable capture and auto-gensym

Consider a macro that defines a function logging its own calls:

(defmacro def-logged-fn [fn-name args & body]
  `(defn ~fn-name ~args
     (let [now (System/currentTimeMillis)]
       (println "[" now "] Call to" (str (var ~fn-name)))
       ~@body)))

Clojure lets you define it, and using it throws:

(def-logged-fn printname [name]
  (println "hi" name))

CompilerException java.lang.RuntimeException: Can't let qualified name: user/now

The expansion shows the problem:

;=> (clojure.core/defn printname [name]
      (clojure.core/let [user/now (java.lang.System/currentTimeMillis)]
        (clojure.core/println "[" user/now "] Call to"
          (clojure.core/str (var printname)))
        (println "hi" name)))

let cannot bind a namespace-qualified name like user/now.

But the error is protecting you from something worse. Suppose Clojure did not qualify it:

(def-logged-fn daily-report [the-day]
  ;; code to generate a report here
  )

(let [now "2009-10-22"]
  (daily-report now))

daily-report would see now as a number like 1259828075387, not "2009-10-22" — because the let generated by the macro captured the caller’s now.

This behavior is known as variable capture, and it can happen in most Lisps.

The fix is the # suffix:

(defmacro def-logged-fn [fn-name args & body]
  `(defn ~fn-name ~args
     (let [now# (System/currentTimeMillis)]
       (println "[" now# "] Call to" (str (var ~fn-name)))
       ~@body)))

This is auto-gensym. now# might expand to now_14187_auto_, and Clojure replaces every occurrence of now# in the template with the same generated symbol.

Note the division of labour: syntax quote’s namespace resolution prevents capture of function names; auto-gensym prevents capture of local bindings.

Learning outcomes

use-syntax-quote-and-unquote: Use syntax quote, unquote and unquote-splicing to build a macro template.
avoid-variable-capture: Avoid variable capture with auto-gensym.
write-a-basic-macro: Write a macro with defmacro and inspect what it produces.

Concepts

syntax-quote: introduces the template form with namespace resolution
unquote: inserts an evaluated value into a template
unquote-splicing: inserts a sequence’s elements rather than the sequence
auto-gensym: generates unique names to prevent capture
variable-capture: shows the bug auto-gensym exists to prevent

Reading Clojure’s own macros

The best examples are the ones you have been using since unit 2. Each is a handful of lines.

comment

(defmacro comment [& body])

That is the whole implementation — it takes any forms and returns nil. It demonstrates that a macro’s arguments really are unevaluated: the body can be nonsense and nothing happens.

declare

The macro unit 3 used for mutual recursion:

(defmacro declare [& names]
  `(do
     ~@(map #(list 'def %) names)))

(macroexpand '(declare add multiply subtract divide))
;=> (do
      (def add)
      (def multiply)
      (def subtract)
      (def divide))

Note the technique: it produces several forms from a variable number of arguments, which is what ~@ is for.

defonce

(defmacro defonce [name expr]
  `(let [v# (def ~name)]
     (when-not (.hasRoot v#)
       (def ~name ~expr))))

Cannot be a function — it must not evaluate expr when the name already has a root binding. Note the v# auto-gensym.

and

The one that most repays reading:

(defmacro and
  ([] true)
  ([x] x)
  ([x & next]
   `(let [and# ~x]
      (if and# (and ~@next) and#))))

(macroexpand '(and (even? x) (> x 50) (< x 500)))
;=> (let* [and_4357_auto_ (even? x)]
      (if and_4357_auto_
        (clojure.core/and (> x 50) (< x 500))
        and_4357_auto_))

Three things at once:

it must be a macro, because it short-circuits — it stops at the first falsey value and does not evaluate the rest
it is recursive, expanding into nested let/if forms
the and# auto-gensym means x is evaluated once, not twice

And it explains something from unit 2: and returns the deciding value rather than a boolean, because the expansion literally returns and#.

time

(time (* 1331 13531))
;=> 18009761

(defmacro time [expr]
  `(let [start# (. System (nanoTime))
         ret# ~expr]
     (prn
       (str "Elapsed time: "
            (/ (double (- (. System (nanoTime)) start#)) 1000000.0)
            " msecs"))
     ret#))

Must be a macro: it needs the form unevaluated in order to time evaluating it. By the time a function received the value, the work would be over.

What to take from the set. Every technique from the previous section appears here in production code, and each macro exists for a reason expressible in one sentence: it needs to control evaluation, or to define a name. None of them is a macro for style.

Learning outcomes

read-clojures-own-macros: Read the definitions of macros in Clojure’s own core library.
use-syntax-quote-and-unquote: Use syntax quote, unquote and unquote-splicing to build a macro template.
say-why-a-function-cannot-do-it: Explain why some constructs cannot be functions, using the unless example.

Concepts

macros: reads five core macros as worked examples
unquote-splicing: shows splicing producing several forms in declare and and
auto-gensym: shows gensyms preventing double evaluation in and, defonce and time

Writing your own

Five macros, ordered by technique rather than difficulty.

infix

(defmacro infix [expr]
  (let [[left op right] expr]
    (list op left right)))

The minimal macro: no evaluation control, no bindings, purely rearranging a form. Note the destructuring — the argument is a list, and we take it apart like any other data. This is here to show that a macro is, at bottom, a function from a list to a list.

randomly

Evaluate exactly one of several bodies, chosen at random:

(defmacro randomly [& exprs]
  (let [len (count exprs)
        conditions (map #(list '= % '(rand-int len)) (range len))]
    `(cond ~@(interleave conditions exprs))))

(randomly (println "amit") (println "deepthi") (println "adi"))

Run it repeatedly and you get a different name each time. This one must be a macro — a function would evaluate all three printlns before choosing. It also needs ~@, since it receives a variable number of forms.

There is a simpler version:

(defmacro randomly-2 [& exprs]
  (nth exprs (rand-int (count exprs))))

though note this chooses at expansion time rather than run time — a real difference, and a good exercise in seeing which phase you are in.

defwebmethod

Web handlers often start by pulling values out of a request map:

(defn login-user [request]
  (let [username (:username request)
        password (:password request)]
    (if (check-credentials username password)
      (str "Welcome back, " username ", " password " is correct!")
      (str "Login failed!"))))

That destructuring preamble repeats in every handler. A macro removes it:

(defmacro defwebmethod [name args & exprs]
  `(defn ~name [{:keys ~args}]
     ~@exprs))

(defwebmethod login-user [username password]
  (if (check-credentials username password)
    (str "Welcome, " username ", " password " is still correct!")
    (str "Login failed!")))

(login-user request)
;=> "Welcome, amit, 123456 is still correct!"

The shape behind every def-something in every Clojure library you will use: a macro whose output is itself a macro call, with expansion continuing into it.

defnn

Push further — generate a function taking keyword arguments in any order:

(defnn print-details [name salary start-date]
  (println "Name:" name)
  (println "Salary:" salary)
  (println "Started on:" start-date))

(print-details :start-date "10/22/2009" :name "Rob" :salary 1000000)
Name: Rob
Salary: 1000000
Started on: 10/22/2009

(defmacro defnn [fname [& names] & body]
  (let [ks {:keys (vec names)}]
    `(defn ~fname [& {:as arg-map#}]
       (let [~ks arg-map#]
         ~@body))))

Here the macro is computing part of the code — building the :keys destructuring map from the parameter names — rather than filling blanks in a template. That is the difference between a template and generating code.

assert-true

The one showing a capability functions simply do not have:

(assert-true (= (* 2 4) (/ 16 2)))
;=> true
(assert-true (< (* 2 4) (/ 18 2)))
;=> true

and on failure:

(assert-true (>= (* 2 4) (/ 18 2)))
;=> RuntimeException (* 2 4) is not >= 9

The message quotes the source you wrote. The macro receives the assertion as a form, so it can take it apart:

(defmacro assert-true [test-expr]
  (let [[operator lhs rhs] test-expr]
    ...))

A function would receive only false and could say nothing about where it came from. Every testing library you have used depends on this trick.

Key ideas

infix — a macro is a function from a list to a list.
randomly — must be a macro, because a function would evaluate every branch.
defwebmethod — macros that expand into defn are the shape behind every def-something.
defnn — a macro can compute the code, not merely template it.
assert-true — only a macro can see the source form and report it.

Learning outcomes

write-your-own-macros: Write macros that add genuinely new constructs to the language.
use-syntax-quote-and-unquote: Use syntax quote, unquote and unquote-splicing to build a macro template.
avoid-variable-capture: Avoid variable capture with auto-gensym.

Concepts

macros: builds five macros demonstrating rearranging, choosing, defining and quoting
unquote-splicing: splices variable-length bodies in randomly, defwebmethod and defnn
macroexpand: verifies each macro by inspecting its expansion

When not to write a macro

You can now extend the language. The remaining skill is knowing when not to.

The rule: if a function can do it, use a function. Macros are warranted when:

you must control whether or when something is evaluated — unless, and, time, randomly
you must introduce a binding the caller’s code can see
you must define a name — anything shaped like def...
you need the source form itself, not its value — assert-true

What a macro costs:

it is not a value — it cannot be passed to map, stored in a collection, or composed with comp
it runs at a different time from the code around it, so errors surface as confusing expansions
it is harder for a reader, who must know your macro before they can read code that uses it

Macros let you grow the language toward your problem, which is a real and unusual power. Used well, a macro removes a repeated shape that no function could abstract. Used badly, it makes ordinary code unreadable for no gain.

Learning outcomes

judge-when-to-use-a-macro: Judge when a macro is warranted and when a function would do.
explain-the-runtime-phases: Explain Clojure’s read, macroexpand, compile and run phases, and where macros act.
explain-homoiconicity: Explain why macros are possible in Lisp and awkward everywhere else.
say-why-a-function-cannot-do-it: Explain why some constructs cannot be functions, using the unless example.
write-a-basic-macro: Write a macro with defmacro and inspect what it produces.
use-syntax-quote-and-unquote: Use syntax quote, unquote and unquote-splicing to build a macro template.
avoid-variable-capture: Avoid variable capture with auto-gensym.
read-clojures-own-macros: Read the definitions of macros in Clojure’s own core library.
write-your-own-macros: Write macros that add genuinely new constructs to the language.

Concepts

macros: collects when a macro is warranted and what it costs
homoiconicity: collects code-as-data as the property the whole unit rests on

Conclusion

A macro runs between reading and compiling, on forms rather than values.

Read, macroexpand, compile, run. By the time your program runs, every macro call has been replaced by its expansion and the macro itself no longer exists.
Homoiconicity is what makes macros ordinary rather than exotic.

A macro is a function whose arguments happen to be program forms and whose return value happens to be a program form. There is no separate macro language, because the language is already its own data format.
Some constructs cannot be functions, and unless shows exactly why.

Arguments are evaluated before the call. A function receives values; a macro receives forms. If a construct must control whether, when, or how many times something is evaluated, it cannot be a function.
Templates make macros writable; expansion makes them debuggable.

Syntax quote resolves namespaces and builds a template; ~ inserts a value and ~@ splices a sequence’s elements. When a macro misbehaves, macroexpand it and read what it actually produced.
Variable capture is silent, and auto-gensym prevents it.

A macro’s let binding can shadow the caller’s name and change what their code means. now# generates a unique symbol; namespace resolution handles the same problem for function names.
The judgement matters more than the mechanism.

If a function can do it, use a function. A macro is not a value, runs at a different time, and must be learned before code using it can be read.

Where next

The next unit, More on Functional Programming, sets macros aside and returns to functions — pushing higher-order functions and closures far enough to build an object system out of them. It also revisits delayed evaluation from the other side, showing what a closure can do that a macro cannot.

← Evolving Clojure Through Macros More on Functional Programming →

--- title: Lecture notes — Evolving Clojure Through Macros keywords: - ver. 1.0.0 date: '2026-08-20' --- [← Evolving Clojure Through Macros](index.qmd){.hg-back} ## Where we are In **State and the Concurrent World** you saw how immutability dissolves most concurrency problems. That unit finished a thread running since unit 1 — the separation of values from identities. This unit picks up a different one. **Introducing Clojure** claimed: > your entire Clojure program is a series of lists: the very source code of your program is interpreted by the Clojure compiler as lists that contain function names and arguments. and observed that because the same language features exist at the compiler level and in ordinary code, Lisp enables uniquely powerful metaprogramming. Unit 2 added a second hint: `cond` is a nested list of pairs, and generating such a list is easy. This unit collects on both. Everything so far has been about *using* a language. This is about **changing** it. ## What you will be able to do 1. `explain-the-runtime-phases` — Explain Clojure's read, macroexpand, compile and run phases, and where macros act. 2. `explain-homoiconicity` — Explain why macros are possible in Lisp and awkward everywhere else. 3. `say-why-a-function-cannot-do-it` — Explain why some constructs cannot be functions, using the unless example. 4. `write-a-basic-macro` — Write a macro with defmacro and inspect what it produces. 5. `use-syntax-quote-and-unquote` — Use syntax quote, unquote and unquote-splicing to build a macro template. 6. `avoid-variable-capture` — Avoid variable capture with auto-gensym. 7. `read-clojures-own-macros` — Read the definitions of macros in Clojure's own core library. 8. `write-your-own-macros` — Write macros that add genuinely new constructs to the language. 9. `judge-when-to-use-a-macro` — Judge when a macro is warranted and when a function would do. ## What we will cover - **Homoiconicity** — code as data, and why that makes macros ordinary. - **Macros** — functions that run at macroexpansion time on unevaluated forms. - **macroexpand** — inspecting what a macro produced. - **Syntax quote, unquote, unquote-splicing** — writing a macro as a template. - **Auto-gensym** — generating unique names. - **Variable capture** — the bug auto-gensym prevents. ## The promise from unit one The route this unit takes: 1. **The phases** — read, macroexpand, compile, run, and where a macro acts 2. **Why a function cannot do it** — the `unless` example, which fails instructively 3. **Templates** — syntax quote, unquote, unquote-splicing, and auto-gensym 4. **Clojure's own macros** — `comment`, `declare`, `defonce`, `and`, `time` 5. **Your own** — five macros, each demonstrating a different technique 6. **Judgement** — when a macro is warranted, and what it costs ::: {.hypergen-refs .outcomes} <h1>Learning outcomes</h1> - {{< badge outcome "explain-homoiconicity" >}}: Explain why macros are possible in Lisp and awkward everywhere else. - {{< badge outcome "explain-the-runtime-phases" >}}: Explain Clojure's read, macroexpand, compile and run phases, and where macros act. ::: ## The phases a form passes through Nearly every macro confusion is a phase confusion, so start here. ![The phases of the Clojure runtime. This separation is what makes the macro system possible.](7_evolving_clojure_through_macros/img-0.jpeg) - **Read** — text becomes Clojure data structures: lists, vectors, symbols, keywords. Reader macros act here — the quote, `#()`, tagged literals. Nothing has been evaluated; there is no function call yet, only a list whose first element happens to be a symbol. - **Macroexpand** — the compiler walks the forms. Where the first element of a list names a macro, it **calls that macro** as an ordinary function, passing the remaining forms **unevaluated, as data**. The return value replaces the original, and the process repeats until no macros remain. - **Compile** — the expanded forms become JVM bytecode. - **Run** — the bytecode executes. ### Textual substitution The simplest way in. Suppose you have a ref: ```clojure (def a-ref (ref 0)) ``` and want to set it, which requires a transaction: ```clojure (dosync (ref-set a-ref 1)) ``` That is verbose if you do it often. You would rather write: ```clojure (sync-set a-ref 1) ``` A macro can do exactly that — build the longer form from the shorter one: ```clojure (defmacro sync-set [r v] (list 'dosync (list 'ref-set r v))) ``` Look at what the body does. It is an ordinary function that **builds a list**. The first element is the symbol `dosync`, quoted so it is not evaluated. That list *is* the code that will run. ::: {.callout-important title="Two consequences to hold onto"} **A macro's arguments are forms, not values.** `(sync-set a-ref 1)` hands the macro the *symbol* `a-ref`, not the ref it names. That is the entire source of a macro's power. **Macros have no runtime existence.** By the time your program runs, every macro call has been replaced by its expansion. A macro cannot be passed to `map`, stored in a collection or called dynamically — it is not a value. The final section returns to that cost. ::: ::: {.hypergen-refs .outcomes} <h1>Learning outcomes</h1> - {{< badge outcome "explain-the-runtime-phases" >}}: Explain Clojure's read, macroexpand, compile and run phases, and where macros act. - {{< badge outcome "explain-homoiconicity" >}}: Explain why macros are possible in Lisp and awkward everywhere else. ::: ::: {.hypergen-refs .concepts} <h1>Concepts</h1> - {{< badge concept "homoiconicity" >}}: explains why code being data is what makes the macro system possible - {{< badge concept "macros" >}}: introduces macros as functions running at macroexpansion time on unevaluated forms ::: ## Why unless cannot be a function The pivotal section, and the failure is more instructive than the fix. Clojure has `if`, and sometimes you want its inverse: ```clojure (defn exhibits-oddity? [x] (if (odd? x) (println "Very odd!"))) ``` reads slightly better as: ```clojure (defn exhibits-oddity? [x] (unless (even? x) (println "Very odd, indeed!"))) ``` ### The attempt ```clojure (defn unless [test then] (if (not test) then)) ``` Test it with an odd number: ```clojure (exhibits-oddity? 11) Very odd, indeed! ;=> nil ``` Correct. Now an even number: ```clojure (exhibits-oddity? 10) Very odd, indeed! ;=> nil ``` **Wrong.** It printed anyway. ### Why Clojure evaluates arguments **before** calling a function. By the time `unless` receives `then`, `(println "Very odd, indeed!")` has already run and produced `nil`. The function cannot un-print it — it only ever sees the `nil` that was left behind. Worse, this failure is **quiet** when the body is pure. It becomes visible only when the body has a side effect or is expensive, which is exactly when you care. ### The workaround, and why it is not good enough You can delay evaluation by wrapping the body in a function: ```clojure (defn unless [test then-thunk] (if (not test) (then-thunk))) (defn exhibits-oddity? [x] (unless (even? x) #(println "Rather odd!"))) ``` This works: ```clojure (exhibits-oddity? 11) Rather odd! ;=> nil (exhibits-oddity? 10) ;=> nil ``` But it forces every caller to remember the `#()`. Forget it once and the bug is back, silently. ### The macro ```clojure (defmacro unless [test then] (list 'if (list 'not test) then)) ``` Now callers write ordinary code, and you can see exactly what happens to it: ```clojure (macroexpand '(unless (even? x) (println "Very odd, indeed!"))) ;=> (if (not (even? x)) (println "Very odd, indeed!")) ``` The body was never evaluated — it was **moved** into an `if`. ::: {.callout-tip title="macroexpand, macroexpand-1 and macroexpand-all"} `macroexpand-1` performs **one** expansion step. `macroexpand` repeats until the outermost form is no longer a macro call. `macroexpand-all` (from `clojure.walk`) expands everything, including nested forms. None of them **run** the result. Reading the expansion is how you debug a macro — a macro producing wrong code usually produces a confusing *runtime* error, and the expansion shows the actual problem. ::: ### The general rule A function receives **values**. A macro receives **forms**. So: - must it decide **whether** to evaluate something? Not a function. - must it decide **when**, or **how many times**? Not a function. - does it only compute with the values it is given? **Use a function.** ::: {.hypergen-refs .outcomes} <h1>Learning outcomes</h1> - {{< badge outcome "say-why-a-function-cannot-do-it" >}}: Explain why some constructs cannot be functions, using the unless example. - {{< badge outcome "write-a-basic-macro" >}}: Write a macro with defmacro and inspect what it produces. ::: ::: {.hypergen-refs .concepts} <h1>Concepts</h1> - {{< badge concept "macros" >}}: shows a construct that cannot be a function, and the macro that replaces it - {{< badge concept "macroexpand" >}}: introduces the expansion tools as the way to inspect and debug a macro ::: ## Templates and hygiene Building lists by hand with `list` and quoted symbols works and does not scale. Compare the `unless` macro above with what we want to write. ### Syntax quote The backquote quotes a form like `'` does, with two differences: ```clojure (defmacro unless [test then] `(if (not ~test) ~then)) ``` - symbols are **resolved to their namespaces**, so `not` becomes `clojure.core/not` and cannot be shadowed by whatever the caller has defined - the form becomes a **template**, into which values can be inserted ### Unquote `~form` evaluates `form` and inserts the result. Forget it and the symbol is inserted literally: ```clojure (defmacro unless [test then] `(if (not ~test) then)) (macroexpand '(unless (even? x) (println "Very odd, indeed!"))) ;=> (if (clojure.core/not (even? x)) user/then) ``` `user/then` — the macro's own parameter name, namespace-qualified, instead of the caller's code. ### Splicing Now let the macro take several body expressions: ```clojure (defn exhibits-oddity? [x] (unless (even? x) (println "Odd!") (println "Very odd!"))) ``` Try it with `~`: ```clojure (defmacro unless [test & exprs] `(if (not ~test) (do ~exprs))) ``` ```clojure (exhibits-oddity? 11) Odd! Very odd! NullPointerException user/exhibits-oddity? (NO_SOURCE_FILE:4) ``` It printed both lines and then threw. The expansion says why: ```clojure (macroexpand-1 '(unless (even? x) (println "Odd!") (println "Very odd!"))) ;=> (if (clojure.core/not (even? x)) (do ((println "Odd!") (println "Very odd!")))) ``` Look at the double parentheses. `~exprs` inserted the **list of expressions** as a single form, so after both `println`s evaluate to `nil`, the result is `(nil nil)` — an attempt to call `nil` as a function. `~@` splices the elements instead: ```clojure (defmacro unless [test & exprs] `(if (not ~test) (do ~@exprs))) ``` [`~` inserts the sequence; `~@` inserts its elements.]{.hg-key} This confuses everyone once. The way to settle it is to macroexpand both and look. ### Variable capture and auto-gensym Consider a macro that defines a function logging its own calls: ```clojure (defmacro def-logged-fn [fn-name args & body] `(defn ~fn-name ~args (let [now (System/currentTimeMillis)] (println "[" now "] Call to" (str (var ~fn-name))) ~@body))) ``` Clojure lets you define it, and using it throws: ```clojure (def-logged-fn printname [name] (println "hi" name)) CompilerException java.lang.RuntimeException: Can't let qualified name: user/now ``` The expansion shows the problem: ```clojure ;=> (clojure.core/defn printname [name] (clojure.core/let [user/now (java.lang.System/currentTimeMillis)] (clojure.core/println "[" user/now "] Call to" (clojure.core/str (var printname))) (println "hi" name))) ``` `let` cannot bind a namespace-qualified name like `user/now`. **But the error is protecting you from something worse.** Suppose Clojure did not qualify it: ```clojure (def-logged-fn daily-report [the-day] ;; code to generate a report here ) (let [now "2009-10-22"] (daily-report now)) ``` `daily-report` would see `now` as a number like `1259828075387`, **not** `"2009-10-22"` — because the `let` generated by the macro captured the caller's `now`. > This behavior is known as **variable capture**, and it can happen in most Lisps. The fix is the `#` suffix: ```clojure (defmacro def-logged-fn [fn-name args & body] `(defn ~fn-name ~args (let [now# (System/currentTimeMillis)] (println "[" now# "] Call to" (str (var ~fn-name))) ~@body))) ``` This is **auto-gensym**. `now#` might expand to `now_14187_auto_`, and Clojure replaces every occurrence of `now#` in the template with the same generated symbol. Note the division of labour: syntax quote's namespace resolution prevents capture of **function names**; auto-gensym prevents capture of **local bindings**. ::: {.hypergen-refs .outcomes} <h1>Learning outcomes</h1> - {{< badge outcome "use-syntax-quote-and-unquote" >}}: Use syntax quote, unquote and unquote-splicing to build a macro template. - {{< badge outcome "avoid-variable-capture" >}}: Avoid variable capture with auto-gensym. - {{< badge outcome "write-a-basic-macro" >}}: Write a macro with defmacro and inspect what it produces. ::: ::: {.hypergen-refs .concepts} <h1>Concepts</h1> - {{< badge concept "syntax-quote" >}}: introduces the template form with namespace resolution - {{< badge concept "unquote" >}}: inserts an evaluated value into a template - {{< badge concept "unquote-splicing" >}}: inserts a sequence's elements rather than the sequence - {{< badge concept "auto-gensym" >}}: generates unique names to prevent capture - {{< badge concept "variable-capture" >}}: shows the bug auto-gensym exists to prevent ::: ## Reading Clojure's own macros The best examples are the ones you have been using since unit 2. Each is a handful of lines. ### comment ```clojure (defmacro comment [& body]) ``` That is the whole implementation — it takes any forms and returns `nil`. It demonstrates that a macro's arguments really are unevaluated: the body can be nonsense and nothing happens. ### declare The macro unit 3 used for mutual recursion: ```clojure (defmacro declare [& names] `(do ~@(map #(list 'def %) names))) ``` ```clojure (macroexpand '(declare add multiply subtract divide)) ;=> (do (def add) (def multiply) (def subtract) (def divide)) ``` Note the technique: it produces **several** forms from a variable number of arguments, which is what `~@` is for. ### defonce ```clojure (defmacro defonce [name expr] `(let [v# (def ~name)] (when-not (.hasRoot v#) (def ~name ~expr)))) ``` Cannot be a function — it must **not evaluate** `expr` when the name already has a root binding. Note the `v#` auto-gensym. ### and The one that most repays reading: ```clojure (defmacro and ([] true) ([x] x) ([x & next] `(let [and# ~x] (if and# (and ~@next) and#)))) ``` ```clojure (macroexpand '(and (even? x) (> x 50) (< x 500))) ;=> (let* [and_4357_auto_ (even? x)] (if and_4357_auto_ (clojure.core/and (> x 50) (< x 500)) and_4357_auto_)) ``` Three things at once: - it **must** be a macro, because it short-circuits — it stops at the first falsey value and does not evaluate the rest - it is **recursive**, expanding into nested `let`/`if` forms - the `and#` auto-gensym means `x` is evaluated **once**, not twice And it explains something from unit 2: `and` returns the deciding value rather than a boolean, because the expansion literally returns `and#`. ### time ```clojure (time (* 1331 13531)) ;=> 18009761 ``` ```clojure (defmacro time [expr] `(let [start# (. System (nanoTime)) ret# ~expr] (prn (str "Elapsed time: " (/ (double (- (. System (nanoTime)) start#)) 1000000.0) " msecs")) ret#)) ``` Must be a macro: it needs the form **unevaluated** in order to time evaluating it. By the time a function received the value, the work would be over. **What to take from the set.** Every technique from the previous section appears here in production code, and each macro exists for a reason expressible in one sentence: it needs to control evaluation, or to define a name. None of them is a macro for style. ::: {.hypergen-refs .outcomes} <h1>Learning outcomes</h1> - {{< badge outcome "read-clojures-own-macros" >}}: Read the definitions of macros in Clojure's own core library. - {{< badge outcome "use-syntax-quote-and-unquote" >}}: Use syntax quote, unquote and unquote-splicing to build a macro template. - {{< badge outcome "say-why-a-function-cannot-do-it" >}}: Explain why some constructs cannot be functions, using the unless example. ::: ::: {.hypergen-refs .concepts} <h1>Concepts</h1> - {{< badge concept "macros" >}}: reads five core macros as worked examples - {{< badge concept "unquote-splicing" >}}: shows splicing producing several forms in declare and and - {{< badge concept "auto-gensym" >}}: shows gensyms preventing double evaluation in and, defonce and time ::: ## Writing your own Five macros, ordered by technique rather than difficulty. ### infix ```clojure (defmacro infix [expr] (let [[left op right] expr] (list op left right))) ``` The minimal macro: no evaluation control, no bindings, purely rearranging a form. Note the destructuring — the argument is a list, and we take it apart like any other data. This is here to show that a macro is, at bottom, **a function from a list to a list**. ### randomly Evaluate exactly one of several bodies, chosen at random: ```clojure (defmacro randomly [& exprs] (let [len (count exprs) conditions (map #(list '= % '(rand-int len)) (range len))] `(cond ~@(interleave conditions exprs)))) ``` ```clojure (randomly (println "amit") (println "deepthi") (println "adi")) ``` Run it repeatedly and you get a different name each time. This one **must** be a macro — a function would evaluate all three `println`s before choosing. It also needs `~@`, since it receives a variable number of forms. There is a simpler version: ```clojure (defmacro randomly-2 [& exprs] (nth exprs (rand-int (count exprs)))) ``` though note this chooses at **expansion** time rather than run time — a real difference, and a good exercise in seeing which phase you are in. ### defwebmethod Web handlers often start by pulling values out of a request map: ```clojure (defn login-user [request] (let [username (:username request) password (:password request)] (if (check-credentials username password) (str "Welcome back, " username ", " password " is correct!") (str "Login failed!")))) ``` That destructuring preamble repeats in every handler. A macro removes it: ```clojure (defmacro defwebmethod [name args & exprs] `(defn ~name [{:keys ~args}] ~@exprs)) ``` ```clojure (defwebmethod login-user [username password] (if (check-credentials username password) (str "Welcome, " username ", " password " is still correct!") (str "Login failed!"))) (login-user request) ;=> "Welcome, amit, 123456 is still correct!" ``` The shape behind every `def`-something in every Clojure library you will use: a macro whose output is itself a macro call, with expansion continuing into it. ### defnn Push further — generate a function taking **keyword arguments in any order**: ```clojure (defnn print-details [name salary start-date] (println "Name:" name) (println "Salary:" salary) (println "Started on:" start-date)) (print-details :start-date "10/22/2009" :name "Rob" :salary 1000000) Name: Rob Salary: 1000000 Started on: 10/22/2009 ``` ```clojure (defmacro defnn [fname [& names] & body] (let [ks {:keys (vec names)}] `(defn ~fname [& {:as arg-map#}] (let [~ks arg-map#] ~@body)))) ``` Here the macro is **computing** part of the code — building the `:keys` destructuring map from the parameter names — rather than filling blanks in a template. That is the difference between a template and generating code. ### assert-true The one showing a capability functions simply do not have: ```clojure (assert-true (= (* 2 4) (/ 16 2))) ;=> true (assert-true (< (* 2 4) (/ 18 2))) ;=> true ``` and on failure: ```clojure (assert-true (>= (* 2 4) (/ 18 2))) ;=> RuntimeException (* 2 4) is not >= 9 ``` The message quotes **the source you wrote**. The macro receives the assertion as a form, so it can take it apart: ```clojure (defmacro assert-true [test-expr] (let [[operator lhs rhs] test-expr] ...)) ``` A function would receive only `false` and could say nothing about where it came from. **Every testing library you have used depends on this trick.** ::: {.highlights} <h1>Key ideas</h1> - `infix` — a macro is a function from a list to a list. - `randomly` — must be a macro, because a function would evaluate every branch. - `defwebmethod` — macros that expand into `defn` are the shape behind every `def`-something. - `defnn` — a macro can *compute* the code, not merely template it. - `assert-true` — only a macro can see the source form and report it. ::: ::: {.hypergen-refs .outcomes} <h1>Learning outcomes</h1> - {{< badge outcome "write-your-own-macros" >}}: Write macros that add genuinely new constructs to the language. - {{< badge outcome "use-syntax-quote-and-unquote" >}}: Use syntax quote, unquote and unquote-splicing to build a macro template. - {{< badge outcome "avoid-variable-capture" >}}: Avoid variable capture with auto-gensym. ::: ::: {.hypergen-refs .concepts} <h1>Concepts</h1> - {{< badge concept "macros" >}}: builds five macros demonstrating rearranging, choosing, defining and quoting - {{< badge concept "unquote-splicing" >}}: splices variable-length bodies in randomly, defwebmethod and defnn - {{< badge concept "macroexpand" >}}: verifies each macro by inspecting its expansion ::: ## When not to write a macro You can now extend the language. The remaining skill is knowing when not to. **The rule: if a function can do it, use a function.** Macros are warranted when: - you must control **whether** or **when** something is evaluated — `unless`, `and`, `time`, `randomly` - you must introduce a **binding** the caller's code can see - you must **define** a name — anything shaped like `def...` - you need the **source form** itself, not its value — `assert-true` **What a macro costs:** - **it is not a value** — it cannot be passed to `map`, stored in a collection, or composed with `comp` - **it runs at a different time** from the code around it, so errors surface as confusing expansions - **it is harder for a reader**, who must know your macro before they can read code that uses it Macros let you grow the language toward your problem, which is a real and unusual power. Used well, a macro removes a repeated shape that no function could abstract. Used badly, it makes ordinary code unreadable for no gain. ::: {.hypergen-refs .outcomes} <h1>Learning outcomes</h1> - {{< badge outcome "judge-when-to-use-a-macro" >}}: Judge when a macro is warranted and when a function would do. - {{< badge outcome "explain-the-runtime-phases" >}}: Explain Clojure's read, macroexpand, compile and run phases, and where macros act. - {{< badge outcome "explain-homoiconicity" >}}: Explain why macros are possible in Lisp and awkward everywhere else. - {{< badge outcome "say-why-a-function-cannot-do-it" >}}: Explain why some constructs cannot be functions, using the unless example. - {{< badge outcome "write-a-basic-macro" >}}: Write a macro with defmacro and inspect what it produces. - {{< badge outcome "use-syntax-quote-and-unquote" >}}: Use syntax quote, unquote and unquote-splicing to build a macro template. - {{< badge outcome "avoid-variable-capture" >}}: Avoid variable capture with auto-gensym. - {{< badge outcome "read-clojures-own-macros" >}}: Read the definitions of macros in Clojure's own core library. - {{< badge outcome "write-your-own-macros" >}}: Write macros that add genuinely new constructs to the language. ::: ::: {.hypergen-refs .concepts} <h1>Concepts</h1> - {{< badge concept "macros" >}}: collects when a macro is warranted and what it costs - {{< badge concept "homoiconicity" >}}: collects code-as-data as the property the whole unit rests on ::: ## Conclusion - A macro runs between reading and compiling, on forms rather than values. > Read, macroexpand, compile, run. By the time your program runs, every macro call has been replaced by its expansion and the macro itself no longer exists. - Homoiconicity is what makes macros ordinary rather than exotic. > A macro is a function whose arguments happen to be program forms and whose return value happens to be a program form. There is no separate macro language, because the language is already its own data format. - Some constructs cannot be functions, and `unless` shows exactly why. > Arguments are evaluated before the call. A function receives values; a macro receives forms. If a construct must control whether, when, or how many times something is evaluated, it cannot be a function. - Templates make macros writable; expansion makes them debuggable. > Syntax quote resolves namespaces and builds a template; `~` inserts a value and `~@` splices a sequence's elements. When a macro misbehaves, macroexpand it and read what it actually produced. - Variable capture is silent, and auto-gensym prevents it. > A macro's `let` binding can shadow the caller's name and change what their code means. `now#` generates a unique symbol; namespace resolution handles the same problem for function names. - The judgement matters more than the mechanism. > If a function can do it, use a function. A macro is not a value, runs at a different time, and must be learned before code using it can be read. ### Where next The next unit, **More on Functional Programming**, sets macros aside and returns to functions — pushing higher-order functions and closures far enough to build an object system out of them. It also revisits delayed evaluation from the other side, showing what a closure can do that a macro cannot. ::: {.hg-nav} [← Evolving Clojure Through Macros](index.qmd){.hg-prev} [More on Functional Programming →](../8_more_on_functional_programming/index.qmd){.hg-next} :::