Lecture notes — State and the Concurrent World

Published

2026-08-20 00:00

Keywords

ver. 1.0.0

Where we are

In Building Blocks of Clojure you met vars and binding, and saw how a dynamic rebinding is thread-local. That was one member of a family, met out of context.

Further back, Introducing Clojure made a large claim and deferred all of it: that Clojure separates values from identities, that immutability makes shared mutable state largely moot, and that vars, atoms, refs and agents each have clearly defined change semantics. It named them and stopped.

This unit pays that off.

What you will be able to do

name-the-problems-with-shared-state — Name the anomalies that arise from shared mutable state, and why locking is not a real fix.
separate-identity-from-value — Separate an identity from the succession of values it takes over time.
explain-managed-references — Explain what a managed reference is and why Clojure provides four of them.
use-refs-and-stm — Use refs inside transactions and explain the STM guarantees behind them.
use-agents — Use agents for asynchronous, independent change, and handle their errors.
use-atoms — Use atoms for synchronous, uncoordinated change to a single identity.
use-vars-for-thread-local-state — Use vars and thread-local binding as the fourth kind of managed reference.
apply-the-unified-access-model — Use the one interface that all four reference types share.
choose-a-reference-type — Choose the right reference type for a given problem.
use-futures-and-promises — Use futures and promises for parallelism that needs no shared state.

What we will cover

Identity versus value — a stable name for a succession of immutable values.
Persistent data structures — immutability made affordable by structural sharing.
Refs — coordinated, synchronous change inside a transaction.
Software transactional memory — the ACI guarantees, and MVCC underneath them.
Agents — asynchronous, independent change, and the only type tolerating side effects.
Atoms — synchronous, uncoordinated change to one identity.
Vars — thread-local isolation.
The unified access model — one interface across four semantics.
Futures and promises — concurrency without shared state.

The hardest problem, and a different answer

The order this unit takes:

The problem — what goes wrong with shared mutable state, and why locking is not the fix it appears to be
The reframing — separating identity from value
Clojure’s way — persistent structures made fast, and the idea of a managed reference
The four types — refs and STM, agents, atoms, vars
The unified model — one interface across all four, and how to choose
Futures and promises — parallelism where there is no shared state at all

Watch for this as we go: almost nothing here is a concurrency primitive in the traditional sense. There are no mutexes to acquire and no critical sections to guard. The work is done by making values immutable and then defining, precisely, the one moment when an identity takes a new one.

Learning outcomes

name-the-problems-with-shared-state: Name the anomalies that arise from shared mutable state, and why locking is not a real fix.
separate-identity-from-value: Separate an identity from the succession of values it takes over time.

What goes wrong with shared state

State is the current set of values associated with things in a program. A payroll program has employee objects; each represents the state of an employee.

There is no problem with state as such, or even with mutating it. The real world is full of perceived changes — people change, plans change, the weather changes, a bank balance changes.

The problem occurs when concurrent (multithreaded) programs share this sort of state among different threads and then attempt to make updates to it.

When the illusion of single-threaded execution breaks down, the code encounters all manner of inconsistent data.

The anomalies

Lost or buried updates — two threads update the same data one after the other, and the first update is overwritten.

Two threads increment a counter at 10. Both read 10, both write 11. The answer should have been 12.
Dirty reads — a thread reads data another thread is in the middle of updating, and sees an inconsistent state.
Unrepeatable reads — a thread reads a data set and can never read the same data again, because others keep updating it.
Phantom reads — a thread reads data that has been deleted, or misses data that has been added.

The traditional solution, and its price

The obvious fix is locks: constructs ensuring only one thread runs a protected section at a time. A thread that cannot obtain the lock blocks until it becomes available.

Reasonable — until more than one piece of mutable data needs a coordinated change. Then each thread must obtain multiple locks, causing contention and further concurrency problems. Correctness becomes hard to ensure, and bugs are hard to find because multithreaded programs are inherently nondeterministic.

The disadvantages:

Less multithreaded than before. While one thread holds a lock, others wait. That is wasteful and reduces throughput.
Excessive. A thread that only wants to read must still lock, blocking not just writers but other readers.
It depends on you remembering. You must lock, lock the right things, in the right order. A forgotten lock produces no compile-time or runtime warning — only a program that behaves unexpectedly. And the knowledge of what to lock and in what order cannot be expressed in the program; it lives in documentation.

“Everyone in the software industry knows how well documentation works.”

And locking creates new problems

Issue	Description
Deadlock	Two or more threads each wait for locks the other holds. The cyclic dependency stops all of them.
Starvation	A thread never gets enough resources to finish.
Livelock	A special case of starvation: two threads keep executing and changing state but make no progress. Two people in a hallway, each stepping aside in the same direction.
Race condition	Interleaving produces an undesired result. Hard to debug because it happens in rare scenarios.

Brian Goetz’s Java Concurrency in Practice (2006) covers these issues thoroughly. Its examples are in Java, so not directly usable here, but it remains the standard reference.

Learning outcomes

name-the-problems-with-shared-state: Name the anomalies that arise from shared mutable state, and why locking is not a real fix.

Concepts

identity-vs-value: sets up the anomalies that motivate separating identity from value
software-transactional-memory: presents locking as the traditional alternative STM replaces

Identities and values

Object-oriented languages offer classes containing state and related operations. A noble goal — and one with a flaw that surfaces the moment a program becomes multithreaded.

The flaw is that these languages conflate the idea of what Rich Hickey calls identity with that of state.

The favourite-movies example

Consider a person’s favourite set of movies. As a child, Disney and Pixar. As a grownup, Tim Burton and Robert Zemeckis. The entity favorite-movies changes over time.

Or does it?

In reality there are two different sets. At one point favorite-movies referred to a set of children’s movies; later it referred to a different set. What changes over time is not the set but which set the entity refers to. And at any given point, a set of movies does not itself change.

So there are two distinct concepts:

an identity — someone’s favourite movies, the subject of the action in the program
the sequence of values that identity assumes over the course of the program

Which gives a definition worth memorising:

State is the value of an identity at a particular point in time.

The separation between an identity and the values it refers to. The identity never changes; it refers to different values over time.

Immutable values

An immutable object cannot change once created. To simulate change you create a whole new object and replace the old one.

Several languages already do this for some types. Consider:

x = 101

Most languages treat 101 as immutable. Nobody expects this to work:

x.setUnitsDigit(3)
x.setTensDigit(2)

as a way to turn 101 into 123. Instead you write:

x = 101 + 22

and x now points at 123 — a completely new value, also immutable. Java strings work the same way. The identity x refers to different immutable numbers over time, exactly as favorite-movies refers to different immutable sets.

The move Clojure makes is to extend this from numbers and strings to everything.

Why this dissolves the problem

If a value cannot change:

a reader can never see a partially updated value, because no value is ever updated
no lock is needed to read, ever
two threads can hold the same value simultaneously with no interaction at all

The only thing left needing coordination is the instant an identity swaps one value for another. That is a far smaller problem than protecting every access — and it is the one the reference types solve.

Learning outcomes

separate-identity-from-value: Separate an identity from the succession of values it takes over time.

Concepts

identity-vs-value: distinguishes the identity from the sequence of immutable values it refers to

Managed references

First, the performance objection

For this model to work, it must be as fast as in-place mutation. The naive approach — copy the object on every update so readers keep valid data — grows linearly with size and is unusable in production.

So the new and updated objects must share data with the old ones. The requirements are precise. Immutable structures must:

leave the old version of itself in a usable state when it mutates
satisfy the same performance characteristics as the mutable versions

Persistent data structures

A persistent data structure is one that preserves the previous version of itself when it’s modified.

Older versions persist after updates, which makes such structures inherently immutable — every update yields a new value.

All of Clojure’s core data structures are persistent: maps, vectors, lists and sets. They perform well because instead of copying they share structure, keeping performance on par with or extremely close to the equivalent Java structures.

This is the mechanism unit 1 showed with the tree diagrams. Here the point is operational: you can build a model on immutable values without paying for it.

The four managed references

With values immutable, we need something that can change. Instead of an identity being a direct reference to a memory location, it is a managed reference pointing at an immutable value. Over the program’s life it can be made to point at other immutable values.

What makes it managed is that the language can then enforce concurrency semantics — check for modified data, enforce validity, require transactions.

Managed reference type	Useful for
ref	Shared, synchronous, coordinated changes
agent	Shared, asynchronous, independent changes
atom	Shared, synchronous, independent changes
var	Isolated changes — thread-local

Three axes separate them: coordinated or independent, synchronous or asynchronous, shared or thread-local.

We take refs first because they are the most elaborate; the others are simplifications.

Learning outcomes

explain-managed-references: Explain what a managed reference is and why Clojure provides four of them.
separate-identity-from-value: Separate an identity from the succession of values it takes over time.

Concepts

persistent-data-structures: explains structure sharing as what makes immutability affordable
refs: introduces the managed reference for coordinated change
agents: introduces the managed reference for asynchronous change
atoms: introduces the managed reference for independent synchronous change
vars: introduces the managed reference for thread-local isolation

Refs and software transactional memory

Creating and reading

(def all-users (ref {}))

(deref all-users)
;=> {}
@all-users
;=> {}

Asking for the ref itself shows the container rather than the value:

all-users
;=> #<Ref@227e9896: {}>

Reading needs no transaction and never blocks — a direct consequence of values being immutable.

Mutating

Every change happens inside a transaction:

(ref-set all-users {})
IllegalStateException No transaction running

(dosync
  (ref-set all-users {}))
;=> {}

alter applies a function to the current value:

(alter ref function & args)

(defn add-new-user [login budget-amount]
  (dosync
    (let [current-number (count @all-users)
          user (new-user (inc current-number) login budget-amount)]
      (alter all-users assoc login user))))

(add-new-user "amit" 1000000)
;=> {"amit" {:id 2, :login "amit", :monthly-budget 1000000, :total-expenses 0}}

alter returns the final state of the ref. And commute:

(commute ref function & args)

is for commutative operations, allowing more concurrency at the cost of the function possibly running against a different value than you last saw.

What STM is

STM is a concurrency control mechanism working like database transactions — but controlling access to shared memory rather than tables and rows. It is a lock-free solution, and it is optimistic where locking is inherently pessimistic.

How a transaction runs. Code that mutates data goes inside dosync. The runtime then lets any number of threads begin the transaction. Changes made to refs are isolated — only the thread that made them can see them.

The first thread to complete the block is allowed to commit. When another thread then attempts to commit, its transaction is aborted and rolled back — and Clojure retries it automatically, up to an internal limit.

Atomic, Consistent, Isolated

The STM has ACI properties. Not durability — it is volatile in-memory data, not a persistent system.

Atomicity — if a transaction mutates several refs, the changes become visible at one instant. Either all happen, or the transaction fails and none do.
Isolation — changed data inside a transaction is called an in-transaction value, visible only to the thread that made it.
Consistency — if any ref changes during a transaction, the whole transaction is retried. Refs, agents and atoms also accept validator functions at creation, checked on change; a failing validator rolls the transaction back.

MVCC

Clojure’s STM implements multiversion concurrency control, the mechanism behind Oracle and PostgreSQL. Each thread gets a snapshot of the mutable world when its transaction starts. Changes to the snapshot are invisible to others until commit.

The consequence is the good one:

readers never block writers (or other readers) … In fact, writers never block readers either.

Contrast the locking model, where both readers and writers block while one thread works.

The rule you must internalise

Because a transaction can be retried, the code inside a dosync must be free of side effects — it may run several times.

This is not small print. It is the main practical rule for using refs, and it is why the next section exists.

Learning outcomes

use-refs-and-stm: Use refs inside transactions and explain the STM guarantees behind them.
explain-managed-references: Explain what a managed reference is and why Clojure provides four of them.

Concepts

refs: covers creating, dereferencing and mutating refs inside dosync
software-transactional-memory: explains the ACI properties and the MVCC snapshot model

Agents

Creating and sending

(def total-cpu-time (agent 0))

(deref total-cpu-time)
;=> 0

You do not set an agent’s value. You send it a function:

(send the-agent the-function & more-args)

(send total-cpu-time + 700)
(deref total-cpu-time)
;=> 700

The + function is sent to the agent, using the agent’s current value as the first argument and 700 as the second. At some point the function executes and its result becomes the agent’s new value. The send call itself returns immediately.

Two senders, and the choice matters:

send — a fixed thread pool sized for CPU-bound actions
send-off — an expanding pool for actions that block, on I/O or a lock

A blocking action sent with send occupies a pool thread that CPU-bound actions need.

Waiting and failing

(await & the-agents)
(await-for timeout-in-millis & the-agents)

await blocks until sent actions complete; await-for gives up after a timeout.

Errors are the substantive part:

(def bad-agent (agent 10))

(send bad-agent / 0)
;=> #<Agent@125b9ec1 FAILED: 10>

(deref bad-agent)
;=> 10

The agent is now in a failed state, and further sends fail too:

(send bad-agent / 2)
ArithmeticException Divide by zero  (Numbers.java:156)

Inspect and clear it:

(agent-error bad-agent)
;=> #<ArithmeticException java.lang.ArithmeticException: Divide by zero>

(clear-agent-errors bad-agent)

An agent that has silently stopped accepting work is a real failure mode. Knowing this is what makes it diagnosable.

Side effects in STM transactions

This is why agents follow refs directly.

A dosync body may retry, so it must not have side effects — but real programs need to send an email or write a log when a transaction commits.

Sends made inside a transaction are held until it commits, and discarded if it retries or aborts. So an agent is the correct place for the side effect: the transaction stays pure, and the effect happens exactly once, only on success.

Learning outcomes

use-agents: Use agents for asynchronous, independent change, and handle their errors.
use-refs-and-stm: Use refs inside transactions and explain the STM guarantees behind them.

Concepts

agents: covers send, send-off, await, error states, and side effects in transactions

Atoms and vars

Atoms

The difference between an atom and an agent is that updates to agents happen asynchronously at some point in the future, whereas atoms are updated synchronously (immediately). Atoms differ from refs in that changes to atoms are independent from each other and can’t be coordinated.

(def total-rows (atom 0))
(deref total-rows)
;=> 0

Two ways to change one:

(reset! atom new-value)
(swap! the-atom the-function & more-args)

(swap! total-rows + 100)

And the primitive underneath:

(compare-and-set! the-atom old-value new-value)

swap! may call your function more than once

swap! is a compare-and-set loop: read the current value, compute the new one, install it only if the current value has not changed meanwhile. If it has, discard and retry.

So the function passed to swap! must be free of side effects.

Notice this is the same rule as the STM’s, arrived at by a different mechanism. It follows from optimistic concurrency generally, not from one implementation.

Atoms give no coordination. Two atoms changed one after another are two separate atomic changes, with a moment in between where one has moved and the other has not. If that matters, you needed refs.

Vars

You met these in Building Blocks of Clojure. Here they take their place in the family.

(def hbase-master "localhost")
(def ^:dynamic *hbase-master* "localhost")

A var can be declared without a value, and you can ask whether it has one:

(def ^:dynamic *rabbitmq-host*)
;=> #'user/*rabbitmq-host*

(bound? #'*rabbitmq-host*)
;=> false

And the thread-local property, demonstrated with parallel map:

(def ^:dynamic *mysql-host*)

(defn db-query [db]
  (binding [*mysql-host* db]
    (count *mysql-host*)))

(def mysql-hosts ["test-mysql" "dev-mysql" "staging-mysql"])

(pmap db-query mysql-hosts)
;=> (10 9 13)

Each thread rebinds *mysql-host* for itself, and no thread sees another’s binding. Nothing to coordinate, no cost to pay — the degenerate case of the family: state that is not actually shared.

That also bounds its usefulness. Vars are for per-thread context, not for communication.

Learning outcomes

use-atoms: Use atoms for synchronous, uncoordinated change to a single identity.
use-vars-for-thread-local-state: Use vars and thread-local binding as the fourth kind of managed reference.

Concepts

atoms: covers reset!, swap! and the compare-and-set retry
vars: places vars as the thread-local member of the reference family

One interface, four semantics

Having met the four separately, look at them together.

Creating

(def a-ref (ref 0))
(def an-agent (agent 0))
(def an-atom (atom 0))

One constructor each, all taking an initial value.

Reading

(deref a-ref)    ; or @a-ref
(deref an-agent) ; or @an-agent
(deref an-atom)  ; or @an-atom

Identical for all four. This is the payoff of immutability: reading is never coordinated, never blocks, and cannot fail — so it needs no per-type variation.

Mutation

Here they differ, because this is precisely where their semantics live:

Refs	Agents	Atoms
`(ref-set ref new-value)`	`(send agent function & args)`	`(reset! atom new-value)`
`(alter ref function & args)`	`(send-off agent function & args)`	`(swap! atom function & args)`
`(commute ref function & args)`		`(compare-and-set! atom old new)`

Different functions, same shape: supply a function of the old value, get a new value installed under defined rules.

Transactions involve only refs. Worth stating explicitly, because it is the sharpest line between refs and everything else.

Watching for mutation

(def adi (atom 0))

(defn on-change [the-key the-ref old-value new-value]
  (println "Hey, seeing change from" old-value "to" new-value))

(add-watch adi :adi-watcher on-change)
;=> 0

(swap! adi inc)
Hey, seeing change from 0 to 1
;=> 1

(remove-watch adi :adi-watcher)

The watch function receives the key, the reference, and the old and new values. The same for all four types. Watches are how you attach logging, metrics or cache invalidation without the mutating code knowing anything about it.

Four genuinely different concurrency semantics behind one interface. Learn deref once and it works everywhere; learn add-watch once and it works everywhere. The variation is confined to the one place it must be — the moment of change.

Learning outcomes

apply-the-unified-access-model: Use the one interface that all four reference types share.
use-refs-and-stm: Use refs inside transactions and explain the STM guarantees behind them.
use-agents: Use agents for asynchronous, independent change, and handle their errors.
use-atoms: Use atoms for synchronous, uncoordinated change to a single identity.

Concepts

unified-access-model: shows one creating, reading, mutating and watching interface across all four types

Choosing a reference type

Four options, and here is the guide.

Vars are the most basic. They are only useful for isolating changes — to a thread or a scope — not coordinating or sharing them. When you have an ordinarily global value like a database connection or configuration map and need it different for one run of code, use a dynamic var with binding. But vars cannot be written to by multiple parts of your code.

Atoms are one step more powerful: the simplest way to manage state written and read by multiple threads.

The vast majority of the time you’ll be using atoms because you don’t need anything more.

Two drawbacks:

multiple atoms cannot be changed together atomically
changes must be free of side effects, because swap! may run more than once

Refs are atoms with coordination. Instead of one giant atom, split state into several refs and read and write them atomically inside dosync. If you have multiple pieces of shared state that must update together but rarely all in one transaction, you reduce contention and increase concurrency — the same tradeoff as one big lock versus several small ones. Changes must still be side-effect free, because the transaction may retry.

Agents are the only reference type that can tolerate side effects, and they cost more to manage. Side-effecting operations cannot be safely retried, so agents have error states that must be checked and cleared. The action runs asynchronously, so you send a function and wait an indefinite time for it.

The combination worth remembering

Agents combine well with refs inside dosync when you have a mostly pure mutation (refs) with a little side effect that must happen only on success (an agent).

A test worth applying. If you find yourself changing two atoms and worrying about the gap between them, you have discovered your change was coordinated after all. That is the signal to switch to refs.

Learning outcomes

choose-a-reference-type: Choose the right reference type for a given problem.
explain-managed-references: Explain what a managed reference is and why Clojure provides four of them.

Concepts

unified-access-model: reduces the choice among four types to coordination and synchrony

Futures and promises

Not every parallel problem involves shared state.

A future is an object that represents the result of a function that will execute on a different thread. A promise is an object that represents a value that will be delivered to it at some point in the future.

They are not really for state management — unlike reference types they can only ever have one value. What distinguishes them from ordinary values is that the value may not yet be known.

Futures

Start with something slow:

(defn long-calculation [num1 num2]
  (Thread/sleep 5000)
  (* num1 num2))

(defn long-run []
  (let [x (long-calculation 11 13)
        y (long-calculation 13 17)
        z (long-calculation 17 19)]
    (* x y z)))

(time (long-run))
;=> 10207769

Three sequential five-second calls: fifteen seconds. The general form of a future:

(future & body)

Wrap each call in one, then dereference them all:

(time (fast-run))
;=> 10207769

On a machine with at least four cores this completes in about five seconds rather than fifteen. Same answer, the longest rather than the sum.

Related functions: future?, future-done?, future-cancel (which does nothing if execution has already started) and future-cancelled?.

Code run in a future or agent sees vars as they were in the context that created it, at the moment future or send was called. So you can create a future inside a binding and rely on the binding’s value inside it.

Promises

(def p (promise))

Reading it blocks until it is filled:

(def value (deref p))

and someone else fills it, once:

(deliver promise value)

The difference between them

a future computes its own value — you decide what work happens
a promise is filled by someone else — you decide when to wait for it

So a future is a computation you have started; a promise is a rendezvous point between threads.

These need no reference type, no transaction and no coordination, because nothing is shared and nothing changes twice. When a problem fits them, they are the simplest thing in the chapter.

Learning outcomes

use-futures-and-promises: Use futures and promises for parallelism that needs no shared state.

Concepts

futures-and-promises: covers futures for parallel computation and promises for cross-thread rendezvous

What you can now coordinate

The idea to carry out of this unit is smaller than the machinery suggests.

Nothing here is a traditional concurrency primitive. There are no mutexes and no critical sections. Making values immutable removes almost all of the problem, and what remains is the single moment an identity takes a new value — which is exactly what the four reference types define.

Learning outcomes

name-the-problems-with-shared-state: Name the anomalies that arise from shared mutable state, and why locking is not a real fix.
separate-identity-from-value: Separate an identity from the succession of values it takes over time.
explain-managed-references: Explain what a managed reference is and why Clojure provides four of them.
use-refs-and-stm: Use refs inside transactions and explain the STM guarantees behind them.
use-agents: Use agents for asynchronous, independent change, and handle their errors.
use-atoms: Use atoms for synchronous, uncoordinated change to a single identity.
use-vars-for-thread-local-state: Use vars and thread-local binding as the fourth kind of managed reference.
apply-the-unified-access-model: Use the one interface that all four reference types share.
choose-a-reference-type: Choose the right reference type for a given problem.
use-futures-and-promises: Use futures and promises for parallelism that needs no shared state.

Concepts

identity-vs-value: collects the reframing the whole unit rests on
unified-access-model: collects the four types under one interface

Conclusion

The problem is not state, and not even mutation — it is shared mutation.

Lost updates, dirty reads, unrepeatable reads, phantom reads. The real world changes; the trouble starts when threads share the changing thing.
Locking works and moves the burden of correctness onto you, permanently.

It reduces throughput, blocks readers unnecessarily, and depends on remembering to lock the right things in the right order — knowledge that cannot be expressed in the program. Then it adds deadlock, starvation, livelock and races.
Separating identity from value dissolves most of the problem.

An identity is a stable name for a succession of immutable values. State is the value of an identity at a point in time. If no value ever changes, reading can never be unsafe.
Persistent data structures make immutability affordable.

They preserve the previous version of themselves and share structure rather than copying, staying close to the performance of their mutable equivalents.
Four reference types, separated by two questions.

Coordinated? Synchronous? Refs coordinate under STM; atoms are the common case; agents are asynchronous and the only ones tolerating side effects; vars are thread-local and share nothing.
Optimistic concurrency means your function may run more than once.

True for dosync and for swap!, by different mechanisms. Which is why side effects belong in an agent, where they run once and only on success.

Where next

The next unit, Evolving Clojure Through Macros, changes subject entirely. It returns to a thread running since unit 1: your program is a list, the compiler reads it as data, and the same language is available at compile time and run time. Macros are what that fact makes possible.

← State and the Concurrent World Evolving Clojure Through Macros →

--- title: Lecture notes — State and the Concurrent World keywords: - ver. 1.0.0 date: '2026-08-20' --- [← State and the Concurrent World](index.qmd){.hg-back} ## Where we are In **Building Blocks of Clojure** you met vars and `binding`, and saw how a dynamic rebinding is thread-local. That was one member of a family, met out of context. Further back, **Introducing Clojure** made a large claim and deferred all of it: that Clojure separates values from identities, that immutability makes shared mutable state largely moot, and that vars, atoms, refs and agents each have clearly defined change semantics. It named them and stopped. This unit pays that off. ## What you will be able to do 1. `name-the-problems-with-shared-state` — Name the anomalies that arise from shared mutable state, and why locking is not a real fix. 2. `separate-identity-from-value` — Separate an identity from the succession of values it takes over time. 3. `explain-managed-references` — Explain what a managed reference is and why Clojure provides four of them. 4. `use-refs-and-stm` — Use refs inside transactions and explain the STM guarantees behind them. 5. `use-agents` — Use agents for asynchronous, independent change, and handle their errors. 6. `use-atoms` — Use atoms for synchronous, uncoordinated change to a single identity. 7. `use-vars-for-thread-local-state` — Use vars and thread-local binding as the fourth kind of managed reference. 8. `apply-the-unified-access-model` — Use the one interface that all four reference types share. 9. `choose-a-reference-type` — Choose the right reference type for a given problem. 10. `use-futures-and-promises` — Use futures and promises for parallelism that needs no shared state. ## What we will cover - **Identity versus value** — a stable name for a succession of immutable values. - **Persistent data structures** — immutability made affordable by structural sharing. - **Refs** — coordinated, synchronous change inside a transaction. - **Software transactional memory** — the ACI guarantees, and MVCC underneath them. - **Agents** — asynchronous, independent change, and the only type tolerating side effects. - **Atoms** — synchronous, uncoordinated change to one identity. - **Vars** — thread-local isolation. - **The unified access model** — one interface across four semantics. - **Futures and promises** — concurrency without shared state. ## The hardest problem, and a different answer The order this unit takes: 1. **The problem** — what goes wrong with shared mutable state, and why locking is not the fix it appears to be 2. **The reframing** — separating identity from value 3. **Clojure's way** — persistent structures made fast, and the idea of a managed reference 4. **The four types** — refs and STM, agents, atoms, vars 5. **The unified model** — one interface across all four, and how to choose 6. **Futures and promises** — parallelism where there is no shared state at all Watch for this as we go: almost nothing here is a concurrency primitive in the traditional sense. There are no mutexes to acquire and no critical sections to guard. The work is done by making values immutable and then defining, precisely, the one moment when an identity takes a new one. ::: {.hypergen-refs .outcomes} <h1>Learning outcomes</h1> - {{< badge outcome "name-the-problems-with-shared-state" >}}: Name the anomalies that arise from shared mutable state, and why locking is not a real fix. - {{< badge outcome "separate-identity-from-value" >}}: Separate an identity from the succession of values it takes over time. ::: ## What goes wrong with shared state **State is the current set of values associated with things in a program.** A payroll program has employee objects; each represents the state of an employee. There is no problem with state as such, or even with mutating it. The real world is full of perceived changes — people change, plans change, the weather changes, a bank balance changes. > The problem occurs when concurrent (multithreaded) programs share this sort of state among different threads and then attempt to make updates to it. When the illusion of single-threaded execution breaks down, the code encounters all manner of inconsistent data. ### The anomalies - **Lost or buried updates** — two threads update the same data one after the other, and the first update is overwritten. > Two threads increment a counter at 10. Both read 10, both write 11. The answer should have been 12. - **Dirty reads** — a thread reads data another thread is in the middle of updating, and sees an inconsistent state. - **Unrepeatable reads** — a thread reads a data set and can never read the same data again, because others keep updating it. - **Phantom reads** — a thread reads data that has been deleted, or misses data that has been added. ### The traditional solution, and its price The obvious fix is **locks**: constructs ensuring only one thread runs a protected section at a time. A thread that cannot obtain the lock blocks until it becomes available. Reasonable — until more than one piece of mutable data needs a coordinated change. Then each thread must obtain **multiple** locks, causing contention and further concurrency problems. Correctness becomes hard to ensure, and bugs are hard to find because multithreaded programs are inherently nondeterministic. **The disadvantages:** - **Less multithreaded than before.** While one thread holds a lock, others wait. That is wasteful and reduces throughput. - **Excessive.** A thread that only wants to *read* must still lock, blocking not just writers but other readers. - **It depends on you remembering.** You must lock, lock the right things, in the right order. A forgotten lock produces no compile-time or runtime warning — only a program that behaves unexpectedly. And the knowledge of what to lock and in what order **cannot be expressed in the program**; it lives in documentation. > "Everyone in the software industry knows how well documentation works." ### And locking creates new problems | Issue | Description | |---|---| | **Deadlock** | Two or more threads each wait for locks the other holds. The cyclic dependency stops all of them. | | **Starvation** | A thread never gets enough resources to finish. | | **Livelock** | A special case of starvation: two threads keep executing and changing state but make no progress. Two people in a hallway, each stepping aside in the same direction. | | **Race condition** | Interleaving produces an undesired result. Hard to debug because it happens in rare scenarios. | ::: {.column-margin} Brian Goetz's *Java Concurrency in Practice* (2006) covers these issues thoroughly. Its examples are in Java, so not directly usable here, but it remains the standard reference. ::: ::: {.hypergen-refs .outcomes} <h1>Learning outcomes</h1> - {{< badge outcome "name-the-problems-with-shared-state" >}}: Name the anomalies that arise from shared mutable state, and why locking is not a real fix. ::: ::: {.hypergen-refs .concepts} <h1>Concepts</h1> - {{< badge concept "identity-vs-value" >}}: sets up the anomalies that motivate separating identity from value - {{< badge concept "software-transactional-memory" >}}: presents locking as the traditional alternative STM replaces ::: ## Identities and values Object-oriented languages offer classes containing state and related operations. A noble goal — and one with a flaw that surfaces the moment a program becomes multithreaded. > The flaw is that these languages conflate the idea of what Rich Hickey calls **identity** with that of **state**. ### The favourite-movies example Consider a person's favourite set of movies. As a child, Disney and Pixar. As a grownup, Tim Burton and Robert Zemeckis. The entity `favorite-movies` changes over time. Or does it? In reality there are **two different sets**. At one point `favorite-movies` referred to a set of children's movies; later it referred to a different set. What changes over time is not the set but **which set the entity refers to**. And at any given point, a set of movies does not itself change. So there are two distinct concepts: - an **identity** — someone's favourite movies, the subject of the action in the program - the **sequence of values** that identity assumes over the course of the program Which gives a definition worth memorising: > **State** is the value of an identity at a particular point in time. ![The separation between an identity and the values it refers to. The identity never changes; it refers to different values over time.](6_state_and_the_concurrent_world/img-0.jpeg) ### Immutable values An immutable object cannot change once created. To simulate change you create a whole new object and replace the old one. Several languages already do this for some types. Consider: ```text x = 101 ``` Most languages treat `101` as immutable. Nobody expects this to work: ```text x.setUnitsDigit(3) x.setTensDigit(2) ``` as a way to turn 101 into 123. Instead you write: ```text x = 101 + 22 ``` and `x` now points at `123` — a completely new value, also immutable. Java strings work the same way. **The identity `x` refers to different immutable numbers over time**, exactly as `favorite-movies` refers to different immutable sets. The move Clojure makes is to extend this from numbers and strings to *everything*. ### Why this dissolves the problem If a value cannot change: - a reader can **never** see a partially updated value, because no value is ever updated - no lock is needed to read, ever - two threads can hold the same value simultaneously with no interaction at all [The only thing left needing coordination is the instant an identity swaps one value for another.]{.hg-key} That is a far smaller problem than protecting every access — and it is the one the reference types solve. ::: {.hypergen-refs .outcomes} <h1>Learning outcomes</h1> - {{< badge outcome "separate-identity-from-value" >}}: Separate an identity from the succession of values it takes over time. ::: ::: {.hypergen-refs .concepts} <h1>Concepts</h1> - {{< badge concept "identity-vs-value" >}}: distinguishes the identity from the sequence of immutable values it refers to ::: ## Managed references ### First, the performance objection For this model to work, it must be as fast as in-place mutation. The naive approach — copy the object on every update so readers keep valid data — grows linearly with size and is unusable in production. So the new and updated objects must **share** data with the old ones. The requirements are precise. Immutable structures must: - **leave the old version of itself in a usable state** when it mutates - **satisfy the same performance characteristics** as the mutable versions ### Persistent data structures > A **persistent data structure** is one that preserves the previous version of itself when it's modified. Older versions persist after updates, which makes such structures inherently immutable — every update yields a new value. **All of Clojure's core data structures are persistent**: maps, vectors, lists and sets. They perform well because instead of copying they **share structure**, keeping performance on par with or extremely close to the equivalent Java structures. This is the mechanism unit 1 showed with the tree diagrams. Here the point is operational: you can build a model on immutable values without paying for it. ### The four managed references With values immutable, we need something that *can* change. Instead of an identity being a direct reference to a memory location, it is a **managed reference** pointing at an immutable value. Over the program's life it can be made to point at other immutable values. What makes it *managed* is that the language can then enforce concurrency semantics — check for modified data, enforce validity, require transactions. | Managed reference type | Useful for | |---|---| | **ref** | Shared, synchronous, **coordinated** changes | | **agent** | Shared, **asynchronous**, independent changes | | **atom** | Shared, **synchronous**, independent changes | | **var** | **Isolated** changes — thread-local | Three axes separate them: coordinated or independent, synchronous or asynchronous, shared or thread-local. We take refs first because they are the most elaborate; the others are simplifications. ::: {.hypergen-refs .outcomes} <h1>Learning outcomes</h1> - {{< badge outcome "explain-managed-references" >}}: Explain what a managed reference is and why Clojure provides four of them. - {{< badge outcome "separate-identity-from-value" >}}: Separate an identity from the succession of values it takes over time. ::: ::: {.hypergen-refs .concepts} <h1>Concepts</h1> - {{< badge concept "persistent-data-structures" >}}: explains structure sharing as what makes immutability affordable - {{< badge concept "refs" >}}: introduces the managed reference for coordinated change - {{< badge concept "agents" >}}: introduces the managed reference for asynchronous change - {{< badge concept "atoms" >}}: introduces the managed reference for independent synchronous change - {{< badge concept "vars" >}}: introduces the managed reference for thread-local isolation ::: ## Refs and software transactional memory ### Creating and reading ```clojure (def all-users (ref {})) (deref all-users) ;=> {} @all-users ;=> {} ``` Asking for the ref itself shows the container rather than the value: ```clojure all-users ;=> #<Ref@227e9896: {}> ``` Reading needs no transaction and never blocks — a direct consequence of values being immutable. ### Mutating Every change happens inside a transaction: ```clojure (ref-set all-users {}) IllegalStateException No transaction running (dosync (ref-set all-users {})) ;=> {} ``` `alter` applies a function to the current value: ```text (alter ref function & args) ``` ```clojure (defn add-new-user [login budget-amount] (dosync (let [current-number (count @all-users) user (new-user (inc current-number) login budget-amount)] (alter all-users assoc login user)))) (add-new-user "amit" 1000000) ;=> {"amit" {:id 2, :login "amit", :monthly-budget 1000000, :total-expenses 0}} ``` `alter` returns the final state of the ref. And `commute`: ```text (commute ref function & args) ``` is for **commutative** operations, allowing more concurrency at the cost of the function possibly running against a different value than you last saw. ### What STM is STM is a concurrency control mechanism working like database transactions — but controlling access to **shared memory** rather than tables and rows. It is a **lock-free** solution, and it is **optimistic** where locking is inherently pessimistic. **How a transaction runs.** Code that mutates data goes inside `dosync`. The runtime then lets **any number of threads** begin the transaction. Changes made to refs are isolated — only the thread that made them can see them. The first thread to complete the block is allowed to **commit**. When another thread then attempts to commit, its transaction is **aborted and rolled back** — and Clojure retries it automatically, up to an internal limit. ### Atomic, Consistent, Isolated The STM has **ACI** properties. Not durability — it is volatile in-memory data, not a persistent system. - **Atomicity** — if a transaction mutates several refs, the changes become visible at one instant. Either all happen, or the transaction fails and none do. - **Isolation** — changed data inside a transaction is called an **in-transaction value**, visible only to the thread that made it. - **Consistency** — if any ref changes during a transaction, the whole transaction is retried. Refs, agents and atoms also accept **validator functions** at creation, checked on change; a failing validator rolls the transaction back. ### MVCC Clojure's STM implements **multiversion concurrency control**, the mechanism behind Oracle and PostgreSQL. Each thread gets a **snapshot** of the mutable world when its transaction starts. Changes to the snapshot are invisible to others until commit. The consequence is the good one: > readers never block writers (or other readers) ... In fact, writers never block readers either. Contrast the locking model, where both readers and writers block while one thread works. ::: {.callout-important title="The rule you must internalise"} Because a transaction **can be retried**, the code inside a `dosync` must be **free of side effects** — it may run several times. This is not small print. It is the main practical rule for using refs, and it is why the next section exists. ::: ::: {.hypergen-refs .outcomes} <h1>Learning outcomes</h1> - {{< badge outcome "use-refs-and-stm" >}}: Use refs inside transactions and explain the STM guarantees behind them. - {{< badge outcome "explain-managed-references" >}}: Explain what a managed reference is and why Clojure provides four of them. ::: ::: {.hypergen-refs .concepts} <h1>Concepts</h1> - {{< badge concept "refs" >}}: covers creating, dereferencing and mutating refs inside dosync - {{< badge concept "software-transactional-memory" >}}: explains the ACI properties and the MVCC snapshot model ::: ## Agents ### Creating and sending ```clojure (def total-cpu-time (agent 0)) (deref total-cpu-time) ;=> 0 ``` You do not set an agent's value. You **send it a function**: ```text (send the-agent the-function & more-args) ``` ```clojure (send total-cpu-time + 700) (deref total-cpu-time) ;=> 700 ``` The `+` function is sent to the agent, using the agent's current value as the first argument and `700` as the second. At some point the function executes and its result becomes the agent's new value. The `send` call itself returns immediately. **Two senders**, and the choice matters: - **`send`** — a fixed thread pool sized for **CPU-bound** actions - **`send-off`** — an expanding pool for actions that **block**, on I/O or a lock A blocking action sent with `send` occupies a pool thread that CPU-bound actions need. ### Waiting and failing ```text (await & the-agents) (await-for timeout-in-millis & the-agents) ``` `await` blocks until sent actions complete; `await-for` gives up after a timeout. Errors are the substantive part: ```clojure (def bad-agent (agent 10)) (send bad-agent / 0) ;=> #<Agent@125b9ec1 FAILED: 10> (deref bad-agent) ;=> 10 ``` The agent is now in a **failed** state, and further sends fail too: ```clojure (send bad-agent / 2) ArithmeticException Divide by zero (Numbers.java:156) ``` Inspect and clear it: ```clojure (agent-error bad-agent) ;=> #<ArithmeticException java.lang.ArithmeticException: Divide by zero> (clear-agent-errors bad-agent) ``` An agent that has silently stopped accepting work is a real failure mode. Knowing this is what makes it diagnosable. ### Side effects in STM transactions This is why agents follow refs directly. A `dosync` body may retry, so it must not have side effects — but real programs need to send an email or write a log when a transaction commits. **Sends made inside a transaction are held until it commits**, and discarded if it retries or aborts. So an agent is the correct place for the side effect: the transaction stays pure, and the effect happens exactly once, only on success. ::: {.hypergen-refs .outcomes} <h1>Learning outcomes</h1> - {{< badge outcome "use-agents" >}}: Use agents for asynchronous, independent change, and handle their errors. - {{< badge outcome "use-refs-and-stm" >}}: Use refs inside transactions and explain the STM guarantees behind them. ::: ::: {.hypergen-refs .concepts} <h1>Concepts</h1> - {{< badge concept "agents" >}}: covers send, send-off, await, error states, and side effects in transactions ::: ## Atoms and vars ### Atoms > The difference between an atom and an agent is that updates to agents happen asynchronously at some point in the future, whereas atoms are updated **synchronously** (immediately). Atoms differ from refs in that changes to atoms are **independent** from each other and can't be coordinated. ```clojure (def total-rows (atom 0)) (deref total-rows) ;=> 0 ``` Two ways to change one: ```text (reset! atom new-value) (swap! the-atom the-function & more-args) ``` ```clojure (swap! total-rows + 100) ``` And the primitive underneath: ```text (compare-and-set! the-atom old-value new-value) ``` ::: {.callout-important title="swap! may call your function more than once"} `swap!` is a compare-and-set loop: read the current value, compute the new one, install it **only if** the current value has not changed meanwhile. If it has, discard and retry. So the function passed to `swap!` must be **free of side effects**. Notice this is the same rule as the STM's, arrived at by a different mechanism. It follows from optimistic concurrency generally, not from one implementation. ::: Atoms give **no coordination**. Two atoms changed one after another are two separate atomic changes, with a moment in between where one has moved and the other has not. If that matters, you needed refs. ### Vars You met these in **Building Blocks of Clojure**. Here they take their place in the family. ```clojure (def hbase-master "localhost") (def ^:dynamic *hbase-master* "localhost") ``` A var can be declared without a value, and you can ask whether it has one: ```clojure (def ^:dynamic *rabbitmq-host*) ;=> #'user/*rabbitmq-host* (bound? #'*rabbitmq-host*) ;=> false ``` And the thread-local property, demonstrated with parallel map: ```clojure (def ^:dynamic *mysql-host*) (defn db-query [db] (binding [*mysql-host* db] (count *mysql-host*))) (def mysql-hosts ["test-mysql" "dev-mysql" "staging-mysql"]) (pmap db-query mysql-hosts) ;=> (10 9 13) ``` Each thread rebinds `*mysql-host*` for itself, and no thread sees another's binding. **Nothing to coordinate, no cost to pay** — the degenerate case of the family: state that is not actually shared. That also bounds its usefulness. Vars are for per-thread context, not for communication. ::: {.hypergen-refs .outcomes} <h1>Learning outcomes</h1> - {{< badge outcome "use-atoms" >}}: Use atoms for synchronous, uncoordinated change to a single identity. - {{< badge outcome "use-vars-for-thread-local-state" >}}: Use vars and thread-local binding as the fourth kind of managed reference. ::: ::: {.hypergen-refs .concepts} <h1>Concepts</h1> - {{< badge concept "atoms" >}}: covers reset!, swap! and the compare-and-set retry - {{< badge concept "vars" >}}: places vars as the thread-local member of the reference family ::: ## One interface, four semantics Having met the four separately, look at them together. ### Creating ```clojure (def a-ref (ref 0)) (def an-agent (agent 0)) (def an-atom (atom 0)) ``` One constructor each, all taking an initial value. ### Reading ```clojure (deref a-ref) ; or @a-ref (deref an-agent) ; or @an-agent (deref an-atom) ; or @an-atom ``` **Identical for all four.** This is the payoff of immutability: reading is never coordinated, never blocks, and cannot fail — so it needs no per-type variation. ### Mutation Here they differ, because this is precisely where their semantics live: | Refs | Agents | Atoms | |---|---|---| | `(ref-set ref new-value)` | `(send agent function & args)` | `(reset! atom new-value)` | | `(alter ref function & args)` | `(send-off agent function & args)` | `(swap! atom function & args)` | | `(commute ref function & args)` | | `(compare-and-set! atom old new)` | Different functions, same shape: supply a function of the old value, get a new value installed under defined rules. **Transactions** involve only refs. Worth stating explicitly, because it is the sharpest line between refs and everything else. ### Watching for mutation ```clojure (def adi (atom 0)) (defn on-change [the-key the-ref old-value new-value] (println "Hey, seeing change from" old-value "to" new-value)) (add-watch adi :adi-watcher on-change) ;=> 0 (swap! adi inc) Hey, seeing change from 0 to 1 ;=> 1 (remove-watch adi :adi-watcher) ``` The watch function receives the key, the reference, and the old and new values. **The same for all four types.** Watches are how you attach logging, metrics or cache invalidation without the mutating code knowing anything about it. [Four genuinely different concurrency semantics behind one interface.]{.hg-key} Learn `deref` once and it works everywhere; learn `add-watch` once and it works everywhere. The variation is confined to the one place it must be — the moment of change. ::: {.hypergen-refs .outcomes} <h1>Learning outcomes</h1> - {{< badge outcome "apply-the-unified-access-model" >}}: Use the one interface that all four reference types share. - {{< badge outcome "use-refs-and-stm" >}}: Use refs inside transactions and explain the STM guarantees behind them. - {{< badge outcome "use-agents" >}}: Use agents for asynchronous, independent change, and handle their errors. - {{< badge outcome "use-atoms" >}}: Use atoms for synchronous, uncoordinated change to a single identity. ::: ::: {.hypergen-refs .concepts} <h1>Concepts</h1> - {{< badge concept "unified-access-model" >}}: shows one creating, reading, mutating and watching interface across all four types ::: ## Choosing a reference type Four options, and here is the guide. **Vars** are the most basic. They are only useful for **isolating** changes — to a thread or a scope — not coordinating or sharing them. When you have an ordinarily global value like a database connection or configuration map and need it different for one run of code, use a dynamic var with `binding`. But **vars cannot be written to by multiple parts of your code**. **Atoms** are one step more powerful: the simplest way to manage state written and read by multiple threads. > The vast majority of the time you'll be using atoms because you don't need anything more. Two drawbacks: - multiple atoms **cannot** be changed together atomically - changes must be **free of side effects**, because `swap!` may run more than once **Refs are atoms with coordination.** Instead of one giant atom, split state into several refs and read and write them atomically inside `dosync`. If you have multiple pieces of shared state that must update together but rarely all in one transaction, you reduce contention and increase concurrency — the same tradeoff as one big lock versus several small ones. Changes must still be side-effect free, because the transaction may retry. **Agents are the only reference type that can tolerate side effects**, and they cost more to manage. Side-effecting operations cannot be safely retried, so agents have **error states** that must be checked and cleared. The action runs **asynchronously**, so you send a function and wait an indefinite time for it. ::: {.callout-tip title="The combination worth remembering"} Agents combine well with refs inside `dosync` when you have a mostly pure mutation (refs) with a little side effect that must happen only on success (an agent). ::: **A test worth applying.** If you find yourself changing two atoms and worrying about the gap between them, you have discovered your change was coordinated after all. That is the signal to switch to refs. ::: {.hypergen-refs .outcomes} <h1>Learning outcomes</h1> - {{< badge outcome "choose-a-reference-type" >}}: Choose the right reference type for a given problem. - {{< badge outcome "explain-managed-references" >}}: Explain what a managed reference is and why Clojure provides four of them. ::: ::: {.hypergen-refs .concepts} <h1>Concepts</h1> - {{< badge concept "unified-access-model" >}}: reduces the choice among four types to coordination and synchrony ::: ## Futures and promises Not every parallel problem involves shared state. > A **future** is an object that represents the result of a function that will execute on a different thread. A **promise** is an object that represents a value that will be delivered to it at some point in the future. They are not really for state management — unlike reference types they can only ever have **one** value. What distinguishes them from ordinary values is that the value may not yet be known. ### Futures Start with something slow: ```clojure (defn long-calculation [num1 num2] (Thread/sleep 5000) (* num1 num2)) (defn long-run [] (let [x (long-calculation 11 13) y (long-calculation 13 17) z (long-calculation 17 19)] (* x y z))) (time (long-run)) ;=> 10207769 ``` Three sequential five-second calls: fifteen seconds. The general form of a future: ```text (future & body) ``` Wrap each call in one, then dereference them all: ```clojure (time (fast-run)) ;=> 10207769 ``` On a machine with at least four cores this completes in about **five** seconds rather than fifteen. Same answer, the longest rather than the sum. Related functions: `future?`, `future-done?`, `future-cancel` (which does nothing if execution has already started) and `future-cancelled?`. ::: {.column-margin} Code run in a future or agent sees vars as they were **in the context that created it**, at the moment `future` or `send` was called. So you can create a future inside a `binding` and rely on the binding's value inside it. ::: ### Promises ```clojure (def p (promise)) ``` Reading it **blocks until it is filled**: ```clojure (def value (deref p)) ``` and someone else fills it, once: ```text (deliver promise value) ``` ### The difference between them - a **future** computes its own value — you decide *what* work happens - a **promise** is filled by someone else — you decide *when* to wait for it So a future is a computation you have started; a promise is a rendezvous point between threads. These need no reference type, no transaction and no coordination, because nothing is shared and nothing changes twice. When a problem fits them, they are the simplest thing in the chapter. ::: {.hypergen-refs .outcomes} <h1>Learning outcomes</h1> - {{< badge outcome "use-futures-and-promises" >}}: Use futures and promises for parallelism that needs no shared state. ::: ::: {.hypergen-refs .concepts} <h1>Concepts</h1> - {{< badge concept "futures-and-promises" >}}: covers futures for parallel computation and promises for cross-thread rendezvous ::: ## What you can now coordinate The idea to carry out of this unit is smaller than the machinery suggests. Nothing here is a traditional concurrency primitive. There are no mutexes and no critical sections. **Making values immutable removes almost all of the problem, and what remains is the single moment an identity takes a new value** — which is exactly what the four reference types define. ::: {.hypergen-refs .outcomes} <h1>Learning outcomes</h1> - {{< badge outcome "name-the-problems-with-shared-state" >}}: Name the anomalies that arise from shared mutable state, and why locking is not a real fix. - {{< badge outcome "separate-identity-from-value" >}}: Separate an identity from the succession of values it takes over time. - {{< badge outcome "explain-managed-references" >}}: Explain what a managed reference is and why Clojure provides four of them. - {{< badge outcome "use-refs-and-stm" >}}: Use refs inside transactions and explain the STM guarantees behind them. - {{< badge outcome "use-agents" >}}: Use agents for asynchronous, independent change, and handle their errors. - {{< badge outcome "use-atoms" >}}: Use atoms for synchronous, uncoordinated change to a single identity. - {{< badge outcome "use-vars-for-thread-local-state" >}}: Use vars and thread-local binding as the fourth kind of managed reference. - {{< badge outcome "apply-the-unified-access-model" >}}: Use the one interface that all four reference types share. - {{< badge outcome "choose-a-reference-type" >}}: Choose the right reference type for a given problem. - {{< badge outcome "use-futures-and-promises" >}}: Use futures and promises for parallelism that needs no shared state. ::: ::: {.hypergen-refs .concepts} <h1>Concepts</h1> - {{< badge concept "identity-vs-value" >}}: collects the reframing the whole unit rests on - {{< badge concept "unified-access-model" >}}: collects the four types under one interface ::: ## Conclusion - The problem is not state, and not even mutation — it is *shared* mutation. > Lost updates, dirty reads, unrepeatable reads, phantom reads. The real world changes; the trouble starts when threads share the changing thing. - Locking works and moves the burden of correctness onto you, permanently. > It reduces throughput, blocks readers unnecessarily, and depends on remembering to lock the right things in the right order — knowledge that cannot be expressed in the program. Then it adds deadlock, starvation, livelock and races. - Separating identity from value dissolves most of the problem. > An identity is a stable name for a succession of immutable values. State is the value of an identity at a point in time. If no value ever changes, reading can never be unsafe. - Persistent data structures make immutability affordable. > They preserve the previous version of themselves and share structure rather than copying, staying close to the performance of their mutable equivalents. - Four reference types, separated by two questions. > Coordinated? Synchronous? Refs coordinate under STM; atoms are the common case; agents are asynchronous and the only ones tolerating side effects; vars are thread-local and share nothing. - Optimistic concurrency means your function may run more than once. > True for `dosync` and for `swap!`, by different mechanisms. Which is why side effects belong in an agent, where they run once and only on success. ### Where next The next unit, **Evolving Clojure Through Macros**, changes subject entirely. It returns to a thread running since unit 1: your program is a list, the compiler reads it as data, and the same language is available at compile time and run time. Macros are what that fact makes possible. ::: {.hg-nav} [← State and the Concurrent World](index.qmd){.hg-prev} [Evolving Clojure Through Macros →](../7_evolving_clojure_through_macros/index.qmd){.hg-next} :::