Lecture notes — More on Functional Programming

Published

2026-08-20 00:00

Keywords

ver. 1.0.0

Where we are

In Evolving Clojure Through Macros you learned to change the language. This unit puts that aside and goes back to using it — taking one idea, the first-class function, much further than the earlier units did.

Unit 3 introduced higher-order functions and defined closures. Here both get pushed until something unexpected happens: by the end you will have built an object system in Clojure, with private state and message dispatch, from nothing but closures.

Then we ask whether you should, and the answer will be mostly no. The exercise is what makes the answer meaningful.

What you will be able to do

define-a-higher-order-function — Define what makes a function higher-order and say why it matters.
collect-results-with-map — Collect the results of applying a function across a collection.
reduce-and-filter-collections — Reduce a collection to a single value and filter it down to the elements you want.
use-partial-application — Fix some arguments now and supply the rest later.
adapt-functions-to-a-context — Adapt a general function to a specific context by fixing its varying parts.
identify-free-variables — Identify a function’s free variables and explain what a closure captures.
write-closures-that-carry-state — Write a function that returns a function carrying captured state.
use-closures-to-delay-computation — Use a closure to delay computation and build your own control structure.
build-objects-from-closures — Build message-passing objects out of closures, and say what that reveals.
contrast-data-abstraction-with-objects — Contrast Clojure’s data-oriented approach with the object system you just built.

What we will cover

Higher-order functions — deriving map, reduce and filter rather than being handed them.
Lazy sequences — why do-to-all needs lazy-seq to survive a large input.
Partial application — fixing arguments now and supplying the rest later.
Lexical closures — free variables, captured state, and information hiding.
Message-passing objects — closures that dispatch on a keyword.
Data abstraction versus objects — why Clojure prefers plain data.

Functions all the way down

Two halves, and the second is where it gets interesting.

Higher-order functions — collecting, reducing, filtering, partial application, adapting.

Closures — free variables, delayed computation, and closures that carry state.

Learning outcomes

define-a-higher-order-function: Define what makes a function higher-order and say why it matters.
identify-free-variables: Identify a function’s free variables and explain what a closure captures.

Higher-order functions in practice

The core functions were handed to you in unit 2. Here we derive them, which is a better way to understand what they are for.

Collecting results

Start with squaring every element:

(defn square [x]
  (* x x))

(defn square-all [numbers]
  (if (empty? numbers)
    ()
    (cons (square (first numbers))
          (square-all (rest numbers)))))

(square-all [1 2 3 4 5 6])
;=> (1 4 9 16 25 36)

Now cubing:

(defn cube [x]
  (* x x x))

(defn cube-all [numbers]
  (if (empty? numbers)
    ()
    (cons (cube (first numbers))
          (cube-all (rest numbers)))))

(cube-all [1 2 3 4 5 6])
;=> (1 8 27 64 125 216)

Put them side by side. They are identical except for one function call. Pull that out as a parameter:

(defn do-to-all [f numbers]
  (if (empty? numbers)
    ()
    (cons (f (first numbers))
          (do-to-all f (rest numbers)))))

(do-to-all square [1 2 3 4 5 6])
;=> (1 4 9 16 25 36)
(do-to-all cube [1 2 3 4 5 6])
;=> (1 8 27 64 125 216)

You have written map.

Where it breaks, and laziness

(do-to-all square (range 11000))
StackOverflowError  clojure.lang.Numbers$LongOps.multiply (Numbers.java:459)

The recursion is not in tail position — cons wraps it — so recur cannot help. The fix is laziness:

(defn do-to-all [f numbers]
  (lazy-seq
    (if (empty? numbers)
      ()
      (cons (f (first numbers))
            (do-to-all f (rest numbers))))))

(take 10 (drop 10000 (do-to-all square (range 11000))))
;=> (100000000 100020001 100040004 100060009 100080016 ...)

lazy-seq means the recursive call is not made until the next element is asked for, so the stack never deepens. Clojure’s real map is lazy for exactly this reason.

What laziness costs you

The work has not happened when map returns. That is usually invisible and occasionally decisive — a lazy sequence over side-effecting work is a trap, which is why doseq and doall exist. Unit 3’s dynamic-var example was the same problem wearing a different hat.

Reducing

Summing:

(defn total-of [numbers]
  (loop [nums numbers sum 0]
    (if (empty? nums)
      sum
      (recur (rest nums) (+ sum (first nums))))))

(total-of [5 7 9 3 4 1 2 8])
;=> 39

Finding the largest:

(defn larger-of [x y]
  (if (> x y) x y))

(defn largest-of [numbers]
  (loop [l numbers candidate (first numbers)]
    (if (empty? l)
      candidate
      (recur (rest l) (larger-of candidate (first l))))))

(largest-of [5 7 9 3 4 1 2 8])
;=> 9
(largest-of [])
;=> nil

Same shape again — walk the collection carrying an accumulator, differing only in how they combine. Abstract it:

(defn compute-across [func elements value]
  (if (empty? elements)
    value
    (recur func (rest elements) (func value (first elements)))))

(defn total-of [numbers]
  (compute-across + numbers 0))

(defn largest-of [numbers]
  (compute-across larger-of numbers (first numbers)))

Both originals are now one-liners. You have written reduce.

And it does more than arithmetic:

(defn all-greater-than [threshold numbers]
  (compute-across #(if (> %2 threshold) (conj %1 %2) %1) numbers []))

(all-greater-than 5 [5 7 9 3 4 1 2 8])
;=> [7 9 8]

The accumulator is a vector rather than a number. With Clojure’s reduce — note the argument order differs:

(defn all-greater-than [threshold numbers]
  (reduce #(if (> %2 threshold) (conj %1 %2) %1) [] numbers))

Filtering

(defn all-lesser-than [threshold numbers]
  (compute-across #(if (< %2 threshold) (conj %1 %2) %1) numbers []))

(all-lesser-than 5 [5 7 9 3 4 1 2 8])
;=> [3 4 1 2]

Compare with all-greater-than: identical but for the comparison. Extract it:

(defn select-if [pred elements]
  (compute-across #(if (pred %2) (conj %1 %2) %1) elements []))

(select-if odd? [5 7 9 3 4 1 2 8])
;=> [5 7 9 3 1]

(defn all-lesser-than [threshold numbers]
  (select-if #(< % threshold) numbers))

You have written filter:

(filter odd? [5 7 9 3 4 1 2 8])
;=> (5 7 9 3 1)

Every one of these came from noticing that two functions differed in one place, and making that place a parameter. That move is the whole unit in a sentence.

Learning outcomes

define-a-higher-order-function: Define what makes a function higher-order and say why it matters.
collect-results-with-map: Collect the results of applying a function across a collection.
reduce-and-filter-collections: Reduce a collection to a single value and filter it down to the elements you want.

Concepts

higher-order-functions: derives map, reduce and filter by abstracting the difference between near-identical functions
lazy-sequences: shows lazy-seq rescuing a non-tail recursion over a large input

Partial application

Adapting a function to a context

A price calculation with tax:

(defn price-with-tax [tax-rate amount]
  (-> (/ tax-rate 100)
      (+ 1)
      (* amount)))

(price-with-tax 9.5M 100)
;=> 109.500M

Now you need California prices:

(defn with-california-taxes [prices]
  (map #(price-with-tax 9.25M %) prices))

(def prices [100 200 300 400 500])
(with-california-taxes prices)
;=> (109.2500M 218.5000M 327.7500M 437.0000M 546.2500M)

Then New York, then everywhere else. The naive route is one function per state:

(defn price-with-ca-tax [price]
  (price-with-tax 9.25M price))

(defn price-with-ny-tax [price]
  (price-with-tax 8.0M price))

Duplication, and a new function for every jurisdiction. Write a factory instead:

(defn price-calculator-for-tax [state-tax]
  (fn [price]
    (price-with-tax state-tax price)))

(def price-with-ca-tax (price-calculator-for-tax 9.25M))
(def price-with-ny-tax (price-calculator-for-tax 8.0M))

Each result is an ordinary one-argument function. Callers never know about tax rates.

Adapting a general function into specialised ones by fixing the varying argument.

Note what is really happening: the returned function is a closure over state-tax. This section looks like it is about partial application, and it is the first place captured state does something useful.

Generalising it

Take a function of five arguments:

(defn of-n-args [a b c d e]
  (str a b c d e))

Fixing the first three by hand:

(defn of-k-args [d e]
  (of-n-args 1 2 3 d e))

(of-k-args \a \b)
;=> "123ab"

Generalise it, and the mechanism is again a closure:

(defn partially-applied [of-n-args & n-minus-k-args]
  (fn [& k-args]
    (apply of-n-args (concat n-minus-k-args k-args))))

(def of-2-args (partially-applied of-n-args \a \b \c))
(def of-3-args (partially-applied of-n-args \a \b))

(of-2-args 4 5)
;=> "abc45"
(of-3-args 3 4 5)
;=> "ab345"

Clojure has this built in as partial, used identically:

(def of-2-args (partial of-n-args \a \b \c))
(def of-3-args (partial of-n-args \a \b))

Having written it yourself, partial reads as a convenience rather than magic.

Argument order is a design decision

(defn select-into-if [container pred elements]
  (compute-across #(if (pred %2) (conj %1 %2) %1) elements container))

(def numbers [4 9 5 7 6 3 8])

(select-into-if [] #(< % 7) numbers)
;=> [4 5 6 3]
(select-into-if () #(< % 7) numbers)
;=> (3 6 5 4)

Note the second: conjoining onto a list adds at the front, so the result is reversed. With the container first, both are partially applicable:

(def select-up   (partial select-into-if []))
(def select-down (partial select-into-if ()))

You can only fix arguments from the left. So a function taking its configuration first and its data last partially-applies well, and one taking them the other way round does not. This is why (map f coll) puts the function first, and it is a real consideration when designing your own.

Learning outcomes

use-partial-application: Fix some arguments now and supply the rest later.
adapt-functions-to-a-context: Adapt a general function to a specific context by fixing its varying parts.

Concepts

partial-application: builds partial application by hand and then with partial
lexical-closures: shows the closure underneath every adapted function

Closures and free variables

(defn adder [num1 num2]
  (let [x (+ num1 num2)]
    (fn [y]
      (+ x y))))

(def add-5 (adder 2 3))
(add-5 10)
;=> 15

The inner function uses x, which is neither its parameter nor bound in its body. x is a free variable, and the inner function closes over it — which is where the name comes from. A function with free variables is “open”: you cannot say what it computes without knowing what those names refer to. Capturing them closes it.

Three things to notice:

adder returned long ago, and its x is still there

The captured binding outlives the form that created it.
(adder 2 3) and (adder 10 20) produce independent functions

Each closure gets its own capture, so they cannot interfere.
add-5 takes one argument, not three

It was configured at creation rather than parameterised at every call.

Learning outcomes

identify-free-variables: Identify a function’s free variables and explain what a closure captures.
write-closures-that-carry-state: Write a function that returns a function carrying captured state.

Concepts

lexical-closures: defines closures through free variables and shows captured state persisting

Delayed computation

Start with something that fails:

(let [x 1
      y 0]
  (/ x y))
ArithmeticException Divide by zero  clojure.lang.Numbers.divide (Numbers.java:156)

Wrap it:

(let [x 1
      y 0]
  (try
    (/ x y)
    (catch Exception e (println (.getMessage e)))))
Divide by zero
;=> nil

That pattern repeats often enough to be worth extracting — into a higher-order control structure:

(defn try-catch [the-try the-catch]
  (try
    (the-try)
    (catch Exception e (the-catch e))))

and used:

(let [x 1
      y 0]
  (try-catch #(/ x y)
             #(println (.getMessage %))))

Wrapping a computation in a function of no arguments postpones it. Nothing happens until someone calls it. That single move lets a function receive unevaluated work — otherwise the exclusive privilege of macros. Exception handling has become a value you can pass around and reuse.

Closure or macro?

Both delay evaluation, differently:

a closure delays it by wrapping in a function. The caller writes the wrapper, so it is visible at the call site, and the result is an ordinary value.
a macro delays it by rewriting code. The caller writes ordinary-looking code, but the construct is not a value and cannot be composed.

The trade is syntax versus composability. Unit 7’s unless had to be a macro, because requiring callers to wrap bodies in #() would have been intolerable — and easy to forget. Here, where the wrapper is a deliberate reusable abstraction, the extra #() is a fair price for getting a value back.

Knowing both, and knowing which trade you are making, is the skill.

Learning outcomes

use-closures-to-delay-computation: Use a closure to delay computation and build your own control structure.
write-closures-that-carry-state: Write a function that returns a function carrying captured state.

Concepts

lexical-closures: uses a closure to postpone evaluation and build a control structure

Closures as objects

Now the surprise.

(defn new-user [login password email]
  (fn [a]
    (case a
      :login    login
      :password password
      :email    email
      nil)))

(def arjun (new-user "arjun" "secret" "arjun@zololabs.com"))

(arjun :login)     ;=> "arjun"
(arjun :password)  ;=> "secret"
(arjun :email)     ;=> "arjun@zololabs.com"
(arjun :name)      ;=> nil

A closure capturing three values, dispatching on a message. Nothing new mechanically — and it behaves exactly like an object.

Information hiding

Change one line and the password stops being readable:

(defn new-user [login password email]
  (fn [a]
    (case a
      :login login
      :email email
      :password-hash (hash password)
      nil)))

(def arjun (new-user "arjun" "secret" "arjun@zololabs.com"))

(arjun :password)
;=> nil
(arjun :password-hash)
;=> 1614358358

The inner function can see password; callers cannot. Genuinely private state, from a closure.

Adding behaviour

(defn new-user [login password email]
  (fn [a & args]
    (case a
      :login login
      :email email
      :authenticate (= password (first args)))))

(def adi (new-user "adi" "secret" "adi@currylogic.com"))

(adi :authenticate "blah")
;=> false
(adi :authenticate "secret")
;=> true

Which gives the general form:

(object message-name & arguments)

Objects in OOP are usually defined as entities that have state, behavior, and equality.

We now have state (login, password, email) and behaviour (:authenticate).

Is it data or a function?

Although arjun is a function, semantically it looks and behaves like data.

It behaves like a hash map you can query — except that you decide which keys are public. Because it is a closure, the free variables passed to new-user are captured and stay alive as long as the closure does.

The object system

The chapter generalises this into a small class system with defclass, inheritance, this, and method definitions:

(defn new-class [class-name]
  (fn [command & args]
    (case command
      :name (name class-name))))

(defmacro defclass [class-name]
  `(def ~class-name (new-class '~class-name)))

(defclass Person)
(Person :name)
;=> "Person"

Built up with methods, state and inheritance, the whole thing runs to a little over 50 lines.

The key to this implementation is the lexical closure. It’s up to you to take a stand on the old debate: Are objects a poor man’s closures, or is it the other way around?

Note how the 50 lines divide

Fewer than half the lines manipulate functions. The rest — defclass and its supporting functions — exist to make the syntax look a certain way.

The semantics would not change under a different syntax. The syntax is not what makes the code useful; the underlying mechanisms are. Though a nice syntax does help: it is convenient, and because the code is data you can write checkers that analyse it and give meaningful errors.

Learning outcomes

build-objects-from-closures: Build message-passing objects out of closures, and say what that reveals.
write-closures-that-carry-state: Write a function that returns a function carrying captured state.

Concepts

message-passing-objects: builds objects with private state and message dispatch from closures
lexical-closures: shows capture providing genuine information hiding

What you can now abstract

You built an object system. Should you use it?

in most cases, such artificial constructs are unnecessary in languages such as Clojure.

Two reasons.

Abstraction. You do not need objects for data abstraction. Clojure’s core data structures are the real alternative — each implementation of the sequence abstraction is a candidate for representing data. They are immutable and therefore thread safe, so there is no need for procedural abstractions wrapping their mutation. And when you need a new abstraction that existing types should participate in, that is what protocols are for.

Alan Perlis’s epigram, which is the more memorable argument:

it’s better to have 100 functions that operate on a single data structure instead of 10 functions that operate on 10 data structures.

A common data structure allows more code reuse, because code that works on sequences works no matter what specific data it contains — as the entire sequence library demonstrates.

There is also a practical cost:

creating an object system like the one created in the second half raises a barrier of inoperability with other libraries that don’t know about it.

A map is transparent, printable, comparable and serialisable, and any function can consume it. A closure’s captured state is opaque, and only answers what it chooses to.

So why build one? Because the exercise establishes the equivalence, and the equivalence is what makes the choice informed. Deciding to use maps because objects are unavailable is not a decision. Deciding to use maps having built objects and seen what they cost is.

Learning outcomes

contrast-data-abstraction-with-objects: Contrast Clojure’s data-oriented approach with the object system you just built.
define-a-higher-order-function: Define what makes a function higher-order and say why it matters.
collect-results-with-map: Collect the results of applying a function across a collection.
reduce-and-filter-collections: Reduce a collection to a single value and filter it down to the elements you want.
use-partial-application: Fix some arguments now and supply the rest later.
adapt-functions-to-a-context: Adapt a general function to a specific context by fixing its varying parts.
identify-free-variables: Identify a function’s free variables and explain what a closure captures.
write-closures-that-carry-state: Write a function that returns a function carrying captured state.
use-closures-to-delay-computation: Use a closure to delay computation and build your own control structure.
build-objects-from-closures: Build message-passing objects out of closures, and say what that reveals.

Concepts

data-abstraction-vs-objects: argues for plain data over closure-based objects, via Perlis’s epigram
higher-order-functions: collects the derived core functions
lazy-sequences: collects laziness as the property that makes them work at scale

Conclusion

Every core sequence function comes from abstracting one difference.

square-all and cube-all differ in a function — that is map. total-of and largest-of differ in a combining function — that is reduce. all-greater-than and all-lesser-than differ in a predicate — that is filter.
Laziness is what lets those functions survive a large input.

do-to-all overflows the stack at 11,000 elements because cons puts the recursion out of tail position. lazy-seq defers each step until it is asked for.
Partial application is a closure, and argument order decides whether it is possible.

You can only fix arguments from the left. Put the parts that vary least first, which is why (map f coll) is ordered as it is.
A closure outlives the scope that created it, and each gets its own capture.

adder returns and its x persists. Two closures from one factory never interfere, which is what makes them usable as independent things.
Closures give private state and message dispatch — which is to say, objects.

new-user can see password and callers can only get its hash. Add :authenticate and you have behaviour. The object system that follows is a little over 50 lines, and half of it is syntax.
Knowing you can build objects is what lets you choose not to.

Clojure prefers plain data: transparent, printable, comparable, and consumable by any function. A hundred functions on one data structure beats ten on ten.

Where next

The next unit, More Macros and DSLs, returns to the macro thread with the same shape of ambition: not extending the language a construct at a time, but building a small language of your own on top of it — anaphoric macros, compile-time computation, macros that write macros, and a domain-specific language.

← More on Functional Programming More Macros and DSLs →

--- title: Lecture notes — More on Functional Programming keywords: - ver. 1.0.0 date: '2026-08-20' --- [← More on Functional Programming](index.qmd){.hg-back} ## Where we are In **Evolving Clojure Through Macros** you learned to change the language. This unit puts that aside and goes back to using it — taking one idea, the first-class function, much further than the earlier units did. Unit 3 introduced higher-order functions and defined closures. Here both get pushed until something unexpected happens: by the end you will have built an **object system** in Clojure, with private state and message dispatch, from nothing but closures. Then we ask whether you should, and the answer will be mostly no. The exercise is what makes the answer meaningful. ## What you will be able to do 1. `define-a-higher-order-function` — Define what makes a function higher-order and say why it matters. 2. `collect-results-with-map` — Collect the results of applying a function across a collection. 3. `reduce-and-filter-collections` — Reduce a collection to a single value and filter it down to the elements you want. 4. `use-partial-application` — Fix some arguments now and supply the rest later. 5. `adapt-functions-to-a-context` — Adapt a general function to a specific context by fixing its varying parts. 6. `identify-free-variables` — Identify a function's free variables and explain what a closure captures. 7. `write-closures-that-carry-state` — Write a function that returns a function carrying captured state. 8. `use-closures-to-delay-computation` — Use a closure to delay computation and build your own control structure. 9. `build-objects-from-closures` — Build message-passing objects out of closures, and say what that reveals. 10. `contrast-data-abstraction-with-objects` — Contrast Clojure's data-oriented approach with the object system you just built. ## What we will cover - **Higher-order functions** — deriving `map`, `reduce` and `filter` rather than being handed them. - **Lazy sequences** — why `do-to-all` needs `lazy-seq` to survive a large input. - **Partial application** — fixing arguments now and supplying the rest later. - **Lexical closures** — free variables, captured state, and information hiding. - **Message-passing objects** — closures that dispatch on a keyword. - **Data abstraction versus objects** — why Clojure prefers plain data. ## Functions all the way down Two halves, and the second is where it gets interesting. **Higher-order functions** — collecting, reducing, filtering, partial application, adapting. **Closures** — free variables, delayed computation, and closures that carry state. ::: {.hypergen-refs .outcomes} <h1>Learning outcomes</h1> - {{< badge outcome "define-a-higher-order-function" >}}: Define what makes a function higher-order and say why it matters. - {{< badge outcome "identify-free-variables" >}}: Identify a function's free variables and explain what a closure captures. ::: ## Higher-order functions in practice The core functions were handed to you in unit 2. Here we derive them, which is a better way to understand what they are for. ### Collecting results Start with squaring every element: ```clojure (defn square [x] (* x x)) (defn square-all [numbers] (if (empty? numbers) () (cons (square (first numbers)) (square-all (rest numbers))))) (square-all [1 2 3 4 5 6]) ;=> (1 4 9 16 25 36) ``` Now cubing: ```clojure (defn cube [x] (* x x x)) (defn cube-all [numbers] (if (empty? numbers) () (cons (cube (first numbers)) (cube-all (rest numbers))))) (cube-all [1 2 3 4 5 6]) ;=> (1 8 27 64 125 216) ``` Put them side by side. They are **identical** except for one function call. Pull that out as a parameter: ```clojure (defn do-to-all [f numbers] (if (empty? numbers) () (cons (f (first numbers)) (do-to-all f (rest numbers))))) (do-to-all square [1 2 3 4 5 6]) ;=> (1 4 9 16 25 36) (do-to-all cube [1 2 3 4 5 6]) ;=> (1 8 27 64 125 216) ``` You have written `map`. ### Where it breaks, and laziness ```clojure (do-to-all square (range 11000)) StackOverflowError clojure.lang.Numbers$LongOps.multiply (Numbers.java:459) ``` The recursion is not in tail position — `cons` wraps it — so `recur` cannot help. The fix is **laziness**: ```clojure (defn do-to-all [f numbers] (lazy-seq (if (empty? numbers) () (cons (f (first numbers)) (do-to-all f (rest numbers)))))) (take 10 (drop 10000 (do-to-all square (range 11000)))) ;=> (100000000 100020001 100040004 100060009 100080016 ...) ``` `lazy-seq` means the recursive call is not made until the next element is **asked for**, so the stack never deepens. Clojure's real `map` is lazy for exactly this reason. ::: {.callout-note title="What laziness costs you"} The work has not happened when `map` returns. That is usually invisible and occasionally decisive — a lazy sequence over side-effecting work is a trap, which is why `doseq` and `doall` exist. Unit 3's dynamic-var example was the same problem wearing a different hat. ::: ### Reducing Summing: ```clojure (defn total-of [numbers] (loop [nums numbers sum 0] (if (empty? nums) sum (recur (rest nums) (+ sum (first nums)))))) (total-of [5 7 9 3 4 1 2 8]) ;=> 39 ``` Finding the largest: ```clojure (defn larger-of [x y] (if (> x y) x y)) (defn largest-of [numbers] (loop [l numbers candidate (first numbers)] (if (empty? l) candidate (recur (rest l) (larger-of candidate (first l)))))) (largest-of [5 7 9 3 4 1 2 8]) ;=> 9 (largest-of []) ;=> nil ``` Same shape again — walk the collection carrying an accumulator, differing only in **how they combine**. Abstract it: ```clojure (defn compute-across [func elements value] (if (empty? elements) value (recur func (rest elements) (func value (first elements))))) (defn total-of [numbers] (compute-across + numbers 0)) (defn largest-of [numbers] (compute-across larger-of numbers (first numbers))) ``` Both originals are now one-liners. You have written `reduce`. And it does more than arithmetic: ```clojure (defn all-greater-than [threshold numbers] (compute-across #(if (> %2 threshold) (conj %1 %2) %1) numbers [])) (all-greater-than 5 [5 7 9 3 4 1 2 8]) ;=> [7 9 8] ``` The accumulator is a **vector** rather than a number. With Clojure's `reduce` — note the argument order differs: ```clojure (defn all-greater-than [threshold numbers] (reduce #(if (> %2 threshold) (conj %1 %2) %1) [] numbers)) ``` ### Filtering ```clojure (defn all-lesser-than [threshold numbers] (compute-across #(if (< %2 threshold) (conj %1 %2) %1) numbers [])) (all-lesser-than 5 [5 7 9 3 4 1 2 8]) ;=> [3 4 1 2] ``` Compare with `all-greater-than`: identical but for the comparison. Extract it: ```clojure (defn select-if [pred elements] (compute-across #(if (pred %2) (conj %1 %2) %1) elements [])) (select-if odd? [5 7 9 3 4 1 2 8]) ;=> [5 7 9 3 1] (defn all-lesser-than [threshold numbers] (select-if #(< % threshold) numbers)) ``` You have written `filter`: ```clojure (filter odd? [5 7 9 3 4 1 2 8]) ;=> (5 7 9 3 1) ``` [Every one of these came from noticing that two functions differed in one place, and making that place a parameter.]{.hg-key} That move is the whole unit in a sentence. ::: {.hypergen-refs .outcomes} <h1>Learning outcomes</h1> - {{< badge outcome "define-a-higher-order-function" >}}: Define what makes a function higher-order and say why it matters. - {{< badge outcome "collect-results-with-map" >}}: Collect the results of applying a function across a collection. - {{< badge outcome "reduce-and-filter-collections" >}}: Reduce a collection to a single value and filter it down to the elements you want. ::: ::: {.hypergen-refs .concepts} <h1>Concepts</h1> - {{< badge concept "higher-order-functions" >}}: derives map, reduce and filter by abstracting the difference between near-identical functions - {{< badge concept "lazy-sequences" >}}: shows lazy-seq rescuing a non-tail recursion over a large input ::: ## Partial application ### Adapting a function to a context A price calculation with tax: ```clojure (defn price-with-tax [tax-rate amount] (-> (/ tax-rate 100) (+ 1) (* amount))) (price-with-tax 9.5M 100) ;=> 109.500M ``` Now you need California prices: ```clojure (defn with-california-taxes [prices] (map #(price-with-tax 9.25M %) prices)) (def prices [100 200 300 400 500]) (with-california-taxes prices) ;=> (109.2500M 218.5000M 327.7500M 437.0000M 546.2500M) ``` Then New York, then everywhere else. The naive route is one function per state: ```clojure (defn price-with-ca-tax [price] (price-with-tax 9.25M price)) (defn price-with-ny-tax [price] (price-with-tax 8.0M price)) ``` Duplication, and a new function for every jurisdiction. Write a **factory** instead: ```clojure (defn price-calculator-for-tax [state-tax] (fn [price] (price-with-tax state-tax price))) (def price-with-ca-tax (price-calculator-for-tax 9.25M)) (def price-with-ny-tax (price-calculator-for-tax 8.0M)) ``` Each result is an ordinary one-argument function. Callers never know about tax rates. ![Adapting a general function into specialised ones by fixing the varying argument.](8_more_on_functional_programming/img-0.jpeg) Note what is really happening: the returned function is a **closure** over `state-tax`. This section looks like it is about partial application, and it is the first place captured state does something useful. ### Generalising it Take a function of five arguments: ```clojure (defn of-n-args [a b c d e] (str a b c d e)) ``` Fixing the first three by hand: ```clojure (defn of-k-args [d e] (of-n-args 1 2 3 d e)) (of-k-args \a \b) ;=> "123ab" ``` Generalise it, and the mechanism is again a closure: ```clojure (defn partially-applied [of-n-args & n-minus-k-args] (fn [& k-args] (apply of-n-args (concat n-minus-k-args k-args)))) (def of-2-args (partially-applied of-n-args \a \b \c)) (def of-3-args (partially-applied of-n-args \a \b)) (of-2-args 4 5) ;=> "abc45" (of-3-args 3 4 5) ;=> "ab345" ``` Clojure has this built in as `partial`, used identically: ```clojure (def of-2-args (partial of-n-args \a \b \c)) (def of-3-args (partial of-n-args \a \b)) ``` Having written it yourself, `partial` reads as a convenience rather than magic. ### Argument order is a design decision ```clojure (defn select-into-if [container pred elements] (compute-across #(if (pred %2) (conj %1 %2) %1) elements container)) (def numbers [4 9 5 7 6 3 8]) (select-into-if [] #(< % 7) numbers) ;=> [4 5 6 3] (select-into-if () #(< % 7) numbers) ;=> (3 6 5 4) ``` Note the second: conjoining onto a list adds at the front, so the result is reversed. With the container **first**, both are partially applicable: ```clojure (def select-up (partial select-into-if [])) (def select-down (partial select-into-if ())) ``` [You can only fix arguments from the left.]{.hg-key} So a function taking its configuration first and its data last partially-applies well, and one taking them the other way round does not. This is why `(map f coll)` puts the function first, and it is a real consideration when designing your own. ::: {.hypergen-refs .outcomes} <h1>Learning outcomes</h1> - {{< badge outcome "use-partial-application" >}}: Fix some arguments now and supply the rest later. - {{< badge outcome "adapt-functions-to-a-context" >}}: Adapt a general function to a specific context by fixing its varying parts. ::: ::: {.hypergen-refs .concepts} <h1>Concepts</h1> - {{< badge concept "partial-application" >}}: builds partial application by hand and then with partial - {{< badge concept "lexical-closures" >}}: shows the closure underneath every adapted function ::: ## Closures and free variables ```clojure (defn adder [num1 num2] (let [x (+ num1 num2)] (fn [y] (+ x y)))) (def add-5 (adder 2 3)) (add-5 10) ;=> 15 ``` The inner function uses `x`, which is neither its parameter nor bound in its body. `x` is a **free variable**, and the inner function **closes over** it — which is where the name comes from. A function with free variables is "open": you cannot say what it computes without knowing what those names refer to. Capturing them **closes** it. Three things to notice: - `adder` returned long ago, and its `x` is **still there** > The captured binding outlives the form that created it. - `(adder 2 3)` and `(adder 10 20)` produce **independent** functions > Each closure gets its own capture, so they cannot interfere. - `add-5` takes **one** argument, not three > It was configured at creation rather than parameterised at every call. ::: {.hypergen-refs .outcomes} <h1>Learning outcomes</h1> - {{< badge outcome "identify-free-variables" >}}: Identify a function's free variables and explain what a closure captures. - {{< badge outcome "write-closures-that-carry-state" >}}: Write a function that returns a function carrying captured state. ::: ::: {.hypergen-refs .concepts} <h1>Concepts</h1> - {{< badge concept "lexical-closures" >}}: defines closures through free variables and shows captured state persisting ::: ## Delayed computation Start with something that fails: ```clojure (let [x 1 y 0] (/ x y)) ArithmeticException Divide by zero clojure.lang.Numbers.divide (Numbers.java:156) ``` Wrap it: ```clojure (let [x 1 y 0] (try (/ x y) (catch Exception e (println (.getMessage e))))) Divide by zero ;=> nil ``` That pattern repeats often enough to be worth extracting — into a **higher-order control structure**: ```clojure (defn try-catch [the-try the-catch] (try (the-try) (catch Exception e (the-catch e)))) ``` and used: ```clojure (let [x 1 y 0] (try-catch #(/ x y) #(println (.getMessage %)))) ``` Wrapping a computation in a function of no arguments **postpones** it. Nothing happens until someone calls it. That single move lets a function receive *unevaluated work* — otherwise the exclusive privilege of macros. Exception handling has become a **value** you can pass around and reuse. ::: {.callout-important title="Closure or macro?"} Both delay evaluation, differently: - a **closure** delays it by wrapping in a function. *The caller* writes the wrapper, so it is visible at the call site, and the result is an **ordinary value**. - a **macro** delays it by rewriting code. The caller writes ordinary-looking code, but the construct is **not a value** and cannot be composed. The trade is **syntax versus composability**. Unit 7's `unless` had to be a macro, because requiring callers to wrap bodies in `#()` would have been intolerable — and easy to forget. Here, where the wrapper is a deliberate reusable abstraction, the extra `#()` is a fair price for getting a value back. Knowing both, and knowing which trade you are making, is the skill. ::: ::: {.hypergen-refs .outcomes} <h1>Learning outcomes</h1> - {{< badge outcome "use-closures-to-delay-computation" >}}: Use a closure to delay computation and build your own control structure. - {{< badge outcome "write-closures-that-carry-state" >}}: Write a function that returns a function carrying captured state. ::: ::: {.hypergen-refs .concepts} <h1>Concepts</h1> - {{< badge concept "lexical-closures" >}}: uses a closure to postpone evaluation and build a control structure ::: ## Closures as objects Now the surprise. ```clojure (defn new-user [login password email] (fn [a] (case a :login login :password password :email email nil))) (def arjun (new-user "arjun" "secret" "arjun@zololabs.com")) (arjun :login) ;=> "arjun" (arjun :password) ;=> "secret" (arjun :email) ;=> "arjun@zololabs.com" (arjun :name) ;=> nil ``` A closure capturing three values, dispatching on a **message**. Nothing new mechanically — and it behaves exactly like an object. ### Information hiding Change one line and the password stops being readable: ```clojure (defn new-user [login password email] (fn [a] (case a :login login :email email :password-hash (hash password) nil))) (def arjun (new-user "arjun" "secret" "arjun@zololabs.com")) (arjun :password) ;=> nil (arjun :password-hash) ;=> 1614358358 ``` The inner function **can** see `password`; callers cannot. Genuinely private state, from a closure. ### Adding behaviour ```clojure (defn new-user [login password email] (fn [a & args] (case a :login login :email email :authenticate (= password (first args))))) (def adi (new-user "adi" "secret" "adi@currylogic.com")) (adi :authenticate "blah") ;=> false (adi :authenticate "secret") ;=> true ``` Which gives the general form: ```text (object message-name & arguments) ``` > Objects in OOP are usually defined as entities that have state, behavior, and equality. We now have state (`login`, `password`, `email`) and behaviour (`:authenticate`). ### Is it data or a function? > Although `arjun` is a function, semantically it looks and behaves like data. It behaves like a hash map you can query — except that you decide which keys are public. Because it is a closure, the free variables passed to `new-user` are captured and stay alive as long as the closure does. ### The object system The chapter generalises this into a small class system with `defclass`, inheritance, `this`, and method definitions: ```clojure (defn new-class [class-name] (fn [command & args] (case command :name (name class-name)))) (defmacro defclass [class-name] `(def ~class-name (new-class '~class-name))) ``` ```clojure (defclass Person) (Person :name) ;=> "Person" ``` Built up with methods, state and inheritance, the whole thing runs to **a little over 50 lines**. > The key to this implementation is the lexical closure. It's up to you to take a stand on the old debate: **Are objects a poor man's closures, or is it the other way around?** ::: {.callout-note title="Note how the 50 lines divide"} Fewer than half the lines manipulate functions. The rest — `defclass` and its supporting functions — exist to make the **syntax** look a certain way. The semantics would not change under a different syntax. The syntax is not what makes the code useful; the underlying mechanisms are. Though a nice syntax does help: it is convenient, and because the code is data you can write checkers that analyse it and give meaningful errors. ::: ::: {.hypergen-refs .outcomes} <h1>Learning outcomes</h1> - {{< badge outcome "build-objects-from-closures" >}}: Build message-passing objects out of closures, and say what that reveals. - {{< badge outcome "write-closures-that-carry-state" >}}: Write a function that returns a function carrying captured state. ::: ::: {.hypergen-refs .concepts} <h1>Concepts</h1> - {{< badge concept "message-passing-objects" >}}: builds objects with private state and message dispatch from closures - {{< badge concept "lexical-closures" >}}: shows capture providing genuine information hiding ::: ## What you can now abstract You built an object system. Should you use it? > in most cases, such artificial constructs are unnecessary in languages such as Clojure. Two reasons. **Abstraction.** You do not need objects for data abstraction. Clojure's core data structures are the real alternative — each implementation of the sequence abstraction is a candidate for representing data. They are immutable and therefore thread safe, so there is no need for procedural abstractions wrapping their mutation. And when you need a new abstraction that existing types should participate in, that is what protocols are for. **Alan Perlis's epigram**, which is the more memorable argument: > it's better to have 100 functions that operate on a single data structure instead of 10 functions that operate on 10 data structures. A common data structure allows more code reuse, because code that works on sequences works no matter what specific data it contains — as the entire sequence library demonstrates. There is also a practical cost: > creating an object system like the one created in the second half raises a barrier of inoperability with other libraries that don't know about it. A map is transparent, printable, comparable and serialisable, and any function can consume it. A closure's captured state is opaque, and only answers what it chooses to. **So why build one?** Because the exercise establishes the equivalence, and the equivalence is what makes the choice informed. Deciding to use maps because objects are unavailable is not a decision. Deciding to use maps **having built objects and seen what they cost** is. ::: {.hypergen-refs .outcomes} <h1>Learning outcomes</h1> - {{< badge outcome "contrast-data-abstraction-with-objects" >}}: Contrast Clojure's data-oriented approach with the object system you just built. - {{< badge outcome "define-a-higher-order-function" >}}: Define what makes a function higher-order and say why it matters. - {{< badge outcome "collect-results-with-map" >}}: Collect the results of applying a function across a collection. - {{< badge outcome "reduce-and-filter-collections" >}}: Reduce a collection to a single value and filter it down to the elements you want. - {{< badge outcome "use-partial-application" >}}: Fix some arguments now and supply the rest later. - {{< badge outcome "adapt-functions-to-a-context" >}}: Adapt a general function to a specific context by fixing its varying parts. - {{< badge outcome "identify-free-variables" >}}: Identify a function's free variables and explain what a closure captures. - {{< badge outcome "write-closures-that-carry-state" >}}: Write a function that returns a function carrying captured state. - {{< badge outcome "use-closures-to-delay-computation" >}}: Use a closure to delay computation and build your own control structure. - {{< badge outcome "build-objects-from-closures" >}}: Build message-passing objects out of closures, and say what that reveals. ::: ::: {.hypergen-refs .concepts} <h1>Concepts</h1> - {{< badge concept "data-abstraction-vs-objects" >}}: argues for plain data over closure-based objects, via Perlis's epigram - {{< badge concept "higher-order-functions" >}}: collects the derived core functions - {{< badge concept "lazy-sequences" >}}: collects laziness as the property that makes them work at scale ::: ## Conclusion - Every core sequence function comes from abstracting one difference. > `square-all` and `cube-all` differ in a function — that is `map`. `total-of` and `largest-of` differ in a combining function — that is `reduce`. `all-greater-than` and `all-lesser-than` differ in a predicate — that is `filter`. - Laziness is what lets those functions survive a large input. > `do-to-all` overflows the stack at 11,000 elements because `cons` puts the recursion out of tail position. `lazy-seq` defers each step until it is asked for. - Partial application is a closure, and argument order decides whether it is possible. > You can only fix arguments from the left. Put the parts that vary least first, which is why `(map f coll)` is ordered as it is. - A closure outlives the scope that created it, and each gets its own capture. > `adder` returns and its `x` persists. Two closures from one factory never interfere, which is what makes them usable as independent things. - Closures give private state and message dispatch — which is to say, objects. > `new-user` can see `password` and callers can only get its hash. Add `:authenticate` and you have behaviour. The object system that follows is a little over 50 lines, and half of it is syntax. - Knowing you can build objects is what lets you choose not to. > Clojure prefers plain data: transparent, printable, comparable, and consumable by any function. A hundred functions on one data structure beats ten on ten. ### Where next The next unit, **More Macros and DSLs**, returns to the macro thread with the same shape of ambition: not extending the language a construct at a time, but building a small language of your own on top of it — anaphoric macros, compile-time computation, macros that write macros, and a domain-specific language. ::: {.hg-nav} [← More on Functional Programming](index.qmd){.hg-prev} [More Macros and DSLs →](../11_more_macros_and_dsls/index.qmd){.hg-next} :::