SimulationLab ladder
Level 5 of 10 · Astrophysics
VERIFIEDKepler period error 0.025%. Leapfrog energy stays bounded while Euler drifts to 80%. Code runs 1,558× faster than plain Python.

L05 · Galactic Collision

Two spiral galaxies, 16,000 gravitating bodies, 2 billion years, 4 CPU cores, 3 minutes, all in Python compiled to SIMD machine code. The tidal tails Toomre & Toomre explained in 1972 appear on their own, and then the galaxies merge.

hero

Movie: output/collision.mp4 (1280×720, 20 s, H.264). Gold = galaxy 1, azure = galaxy 2. Brightness is the log (asinh) stretch of stellar surface density on black.


1. The question

Some galaxy pairs, like the Antennae (NGC 4038/9), the Mice (NGC 4676) and the Tadpole, have thin "tails" of stars more than 100 kpc long. In 1972 Alar and Juri Toomre showed these are tidal features. Gravity alone, acting during a close prograde passage, flings out the far side of each disk. Two questions:

  1. Physics: starting from two stable disk galaxies on a parabolic orbit, can a self-consistent N-body model reproduce tails, a bridge and the eventual merger? And does the merger timescale disagree with the point-mass (Kepler) prediction the way dynamical friction says it should?
  2. Computation: gravity is all-pairs, O(N²). How far can plain Python go, and which specific engineering steps (compilation, SIMD, threads, memory layout, precision) buy the speed?

2. The model

Gravity. Direct summation over all pairs with Plummer softening ε (G = 1):

$$\ddot{\mathbf r}_i=\sum_{j}\frac{m_j(\mathbf r_j-\mathbf r_i)}{\left(|\mathbf r_j-\mathbf r_i|^2+\varepsilon^2\right)^{3/2}},\qquad \varepsilon = 0.08 \;(280\ \text{pc})$$

The particles are not stars. Each one is a Monte Carlo sample of a smooth, collisionless phase-space density: a disk particle weighs 6×10⁶ M☉ and a halo particle 5×10⁷ M☉. Softening suppresses the unphysical close two-body encounters that such heavy "super-stars" would otherwise have.

Integrator. Kick-drift-kick leapfrog, fixed step dt = 0.05 (0.89 Myr):

$$\mathbf v_{n+\frac12}=\mathbf v_n+\tfrac{\Delta t}{2}\mathbf a(\mathbf x_n),\quad \mathbf x_{n+1}=\mathbf x_n+\Delta t\,\mathbf v_{n+\frac12},\quad \mathbf v_{n+1}=\mathbf v_{n+\frac12}+\tfrac{\Delta t}{2}\mathbf a(\mathbf x_{n+1})$$

Each galaxy (identical, 8,000 particles):

component profile parameters (N-body units)
stellar disk, 5,000 particles Σ(R) ∝ e^(−R/R_d), ρ(z) ∝ sech²(z/z₀) M_d = 1, R_d = 1, z₀ = 0.1, cut at 6 R_d
dark halo, 3,000 particles (live) Plummer ρ ∝ (1 + r²/b²)^(−5/2) M_h = 5, b = 2

Encounter. Two point masses on a parabolic orbit (e = 1), pericentre r_p = 5 (17.5 kpc), starting separation 20 (70 kpc). Galaxy 1 (gold) has its disk in the orbital plane, spinning prograde. Galaxy 2 (azure) is tilted 60°. The parabola also gives the analytic time to pericentre (Barker's equation), which is used as a check.

Units (G = 1): length 3.5 kpc, mass 3×10¹⁰ M☉, so velocity 192 km/s and time 17.8 Myr. Run: N = 16,000, 2,400 steps, 2.14 Gyr, one frame every 4 steps (600 frames).

3. M&S concepts you'll learn here

4. Tech stack

tool since role here
Python 3.12 L01 glue, the "pure Python" rung of the ladder
NumPy L01 state arrays, the broadcasting rung, histograms
Matplotlib L01 all figures, and the MP4 through FFMpegWriter
SciPy L02 Rotation (disk orientation, camera), gaussian_filter, cKDTree (adaptive smoothing)
SimPy / Mesa / NetworkX L03–L04 not needed here. Gravity is continuous-time physics, not events or agents
★ Numba L05 @njit compiles the force loop to LLVM machine code. fastmath + error_model="numpy" unlock AVX SIMD, and parallel=True + prange spreads it over threads. The same Python source runs at 4.9×10⁶ or 7.6×10⁹ pair interactions/s
★ ffmpeg (via imageio-ffmpeg) L05 H.264 MP4 (yuv420p, plays in any browser)
multiprocessing (stdlib) L04 4 worker processes render movie frames while the main process streams them to ffmpeg

Numba matters because it is the shortest path from "physics loop written like the equation" to "compiled, vectorised, multithreaded". You don't rewrite anything in C, and you don't switch to array tricks that blow up memory. The same pattern (a tight inner loop in @njit) returns in L06 (lattice Boltzmann) and L10 (ray-marched shading).

5. Industry & scientific tools

tool what it is who uses it how this project maps to it
GADGET-4 TreePM + FMM gravity, SPH hydro, hierarchical time steps, MPI cosmology groups (Millennium, IllustrisTNG heritage) our KDK leapfrog is GADGET's integrator. It replaces direct sum with tree/PM and gives each particle its own power-of-two time step
AREPO moving-mesh hydro + TreePM gravity IllustrisTNG, Auriga galaxy simulations same gravity story, plus gas physics (star formation, feedback) we leave out
SWIFT task-based FMM gravity + SPH, very scalable EAGLE-XL, FLAMINGO (Durham, Leiden) the FMM we mention as the O(N) upgrade; its task scheduler is the grown-up version of prange
REBOUND N-body library (WHFast symplectic, IAS15, tree) planetary dynamics, exoplanets our Kepler/leapfrog verification is its bread and butter; WHFast is a symplectic splitting like leapfrog
AMUSE Python framework coupling many astro codes (Bonsai, Gadget2, PhiGRAPE, SeBa) star-cluster, galaxy and multi-physics studies "rebuild it here": the same ICs could drive a GPU tree code (Bonsai) from Python
galpy galactic-dynamics library (potentials, orbits, DFs) Milky Way dynamics where you'd get proper equilibrium disk/halo DFs instead of our epicyclic approximations
NBODY6++GPU / PeTar direct-sum Hermite + regularisation, GPU star clusters (collisional) here O(N²) is still the right choice, because every close encounter matters
LAMMPS / GROMACS molecular dynamics materials, biochemistry the MD cousins: same velocity-Verlet (= leapfrog) integrator, with cut-off short-range forces and PME (a PM method) for long range

6. Results

The encounter

sequence

After the first pericentre (0.29 Gyr) each disk throws out a long, curved tidal tail from its far side and a bridge of stars toward its partner. That is the Toomre & Toomre (1972) signature. The prograde, in-plane gold disk makes a long, wide tail that curls back on itself. The tilted azure disk makes a narrow tail plus a diffuse fan, because its stars feel the tide out of their own plane. The two sets of debris also trade stars, so there is gold around the azure galaxy and azure around the gold one. The galaxies climb to apocentre (71 kpc at 1.03 Gyr), fall back, pass close again at ~1.7 Gyr, and merge into one hot, roughly spheroidal remnant. The tails keep expanding the whole time, because much of their material is unbound.

diagnostics

Left: the separation of the two nuclei (median position of each galaxy's innermost disk particles). Until pericentre the N-body orbit follows the analytic Kepler parabola. Measured pericentre: 0.29 Gyr at 16.3 kpc. Kepler prediction: 0.28 Gyr at 17.5 kpc. After that the curves split: the point-mass parabola escapes to infinity, but the live halos have taken orbital energy through dynamical friction, so the galaxies turn round at 71 kpc and merge. The nuclei stay within 2 kpc of each other from 1.97 Gyr on. Total energy is still conserved (right panel). It just moved from the orbit into the internal random motion of stars and dark matter.

Right: total energy error of the 16,000-body run (float32 forces, fixed dt). It stays at ~10⁻⁴ through the encounter. After the merger it creeps up to 5.6×10⁻⁴, because the merged nucleus is denser and its orbital times get closer to the fixed dt = 0.89 Myr. An earlier run at dt = 0.71 Myr stayed below 1.2×10⁻⁴ all the way. So even a symplectic integrator needs the time step to resolve the fastest orbits, and that is why production codes give every particle its own time step.

Verification

kepler

check expected got pass
two-body Kepler period (m₁ = 1, m₂ = 0.5, a = 1, e = 0.5, 500 steps/orbit) 2π√(a³/GM) = 5.130199 5.131472 (error 2.5×10⁻⁴ = 0.025%) < 1% ✔
leapfrog energy error, 20 orbits bounded (no growth) max 4.2×10⁻⁴, identical peak in first and last orbit ✔
explicit Euler energy error, 20 orbits, same dt drifts 0.80 after 20 orbits (orbit spirals outward) ✔ (>100× leapfrog)
collision run energy conservation ≪ 1% max 5.6×10⁻⁴ (0.056%) ✔
first pericentre vs. Kepler parabola (point masses) 0.28 Gyr, 17.5 kpc 0.29 Gyr, 16.3 kpc ✔ within 4% / 7% (extended halos are not point masses)

Why symplectic matters. Leapfrog does not conserve the true energy H exactly. It exactly conserves a "shadow" Hamiltonian H̃ = H + O(Δt²), because each kick and each drift is itself an exact Hamiltonian flow. So the energy error oscillates, spiking at every pericentre where the step is coarsest, but it never accumulates. Euler is not symplectic. It adds a little energy on each pericentre passage, and the staircase in the plot is those passages stacking up. After 20 orbits the Euler "planet" has lost 80% of its binding energy and is on a much larger orbit. For galaxies (≈10–100 orbits) and planetary systems (10⁹ orbits) that is the difference between physics and garbage. Leapfrog is also time-reversible and costs exactly one force evaluation per step, the same as Euler. (The small period error comes from leapfrog's O(Δt²) phase error, which shows up as slow apsidal precession of the orbit.)

Performance ladder

benchmark

Same physics, same machine (8-core / 16-thread x86 CPU with AVX2), capped at 4 threads, measured on a quiet machine. Each point is the best of 3 samples. Rates are pair interactions per second at the largest N timed for each rung:

rung largest N timed pair interactions/s speed-up vs pure Python gain over previous rung
pure Python (lists) 512 4.9×10⁶ 1× –
NumPy broadcasting 2,048 3.3×10⁷ 7× 7×
Numba @njit 8,192 3.2×10⁸ 65× 10× (vs NumPy)
+ fastmath, error_model="numpy" (SIMD) 16,384 1.47×10⁹ 300× 4.6×
+ parallel=True, 4 threads 32,768 5.0×10⁹ 1,030× 3.4×
+ float32 pair maths (production kernel) 65,536 7.6×10⁹ 1,558× 1.5×
parallel, AoS (N, 3) layout, float64 32,768 1.25×10⁹ 255× 4.0× slower than SoA

What each rung teaches:

  1. Pure Python pays the interpreter on every + and *: ~5×10⁶ pairs/s.
  2. NumPy broadcasting moves the loops into C, but it builds (3, N, N) temporaries. The work becomes memory-bound and the RAM use is O(N²) (N = 4096 would need > 1 GB). It is only 7× faster, and its throughput actually falls once N > 128, when those temporaries stop fitting in cache.
  3. @njit compiles the same Python function to machine code: 65× in one line. The first call pays 0.1–0.6 s of JIT compilation, which is why the benchmark warms up before it times anything.
  4. fastmath=True, error_model="numpy": the compiled loop was still scalar. Numba's default Python error model puts a ZeroDivisionError check inside the loop, and strict IEEE forbids reordering the sum. Remove both and LLVM emits AVX vsqrtpd/vdivpd on 4 doubles at once (checked with accel_simd.inspect_asm()), which gives 4.6×. Before this fix, the serial fastmath rung was no faster than plain @njit.
  5. parallel=True + prange splits the i-loop across 4 threads with no race conditions, because each thread owns its own rows. That gives 3.4× on 4 threads, but only once N ≳ 1000. At N = 64 it is no faster than one thread, because waking the threads costs as much as the work. On a saturated machine it is worse: a parallel region is only as fast as its slowest thread. In one run where other jobs had all 16 hardware threads busy, the 4-thread rung fell behind the single-threaded SIMD rung for N below ~10⁴, and the full run took 12.4 min instead of 3.
  6. float32 does the pair maths in single precision (8 lanes per AVX register) but keeps the sums and the state in float64. This is the GPU-code trick (Bonsai, and the GPU Gems N-body kernel). It is the production kernel.
  7. Dashed line, AoS layout: the same parallel kernel reading an (N, 3) array-of-structs instead of separate x/y/z arrays (structure-of-arrays). The strided loads defeat vectorisation, and it runs 4.0× slower: 4 threads of AoS are no faster than 1 thread of SoA. Memory layout is a performance decision.

The collision itself ran at 6.5×10⁹ pair interactions/s (39.6 ms per step at N = 16,000, including the Python-side integrator and bookkeeping).

Why production codes don't do this: O(N²) vs. Barnes–Hut, FMM, TreePM

The right panel is the whole story. Cost grows as N², so at our best measured 7.6×10⁹ pairs/s:

N one force evaluation (direct sum, 4 cores) typical use
1.6×10⁴ (this run) 0.034 s (measured: 0.040 s per full step) galaxy-pair toy model
10⁶ ~2 min a single modern galaxy simulation
10¹⁰ ~400 years a cosmological run (Millennium had 10¹⁰, FLAMINGO 3×10¹¹)

And a run needs 10⁴–10⁵ steps. Production codes change the algorithm, not the constant factor:

Direct summation still wins where every encounter matters and accuracy is paramount: collisional star clusters (NBODY6++GPU, PeTar, with GPUs doing exactly our inner loop) and planetary systems. For a collisionless galaxy the particles are statistical samples anyway, so the approximate tree force costs nothing in fidelity.

7. Run it

From the repo root:

uv run python L05_galactic_collision/sim.py          # full run: 188 s on a quiet 8-core machine
uv run python L05_galactic_collision/sim.py --quick  # smoke test: 5,000 bodies, 0.7 Gyr, no benchmark, ~35 s
                                                     # (note: --quick overwrites output/ with its smaller versions)

Output goes to output/: hero.png, collision.mp4, sequence.png, collision_diagnostics.png, verify_kepler_energy.png, benchmark.png. The run prints the benchmark table in Markdown. Timing breakdown (final run): benchmark 23 s, verification 1 s, collision 96 s (2,400 steps), stills + MP4 68 s (600 frames rendered by 4 processes). When other jobs saturate the CPU, expect 2–4× longer (we measured 12.4 min).

8. Code tour (sim.py)

function what it does
accel the kernel: direct-sum softened gravity on structure-of-arrays. It is plain Python, so it is also the pure-Python rung
accel_jit, accel_simd, accel_par the same function compiled three ways (njit, + fastmath/numpy error model, + parallel)
accel_aos, accel_numpy the array-of-structs and NumPy-broadcasting comparison rungs
potential, energy O(N²/2) softened potential energy (parallel reduction) + kinetic
gravity wraps a kernel into force(x). The float32 option is the mixed-precision mode
leapfrog, euler integrators as generators that yield after every step
verify Kepler period + leapfrog vs. Euler energy, asserts, and the figure
benchmark times every rung at N = 64 … 65,536, then prints the table and draws the chart
plummer, galaxy equilibrium ICs: Plummer DF sampling, exponential disk, measured rotation curve, Toomre-Q dispersions
collision_ics orients the two disks (scipy Rotation) and puts them on the parabolic orbit
collide the production run: stores frames, nucleus separation, energy
render adaptive-smoothing asinh image, one hue per galaxy (kNN widths via cKDTree)
zoom_schedule, make_movie, make_stills, diagnostics camera, MP4 (4-process frame pool feeding FFMpegWriter), hero/sequence, diagnostic plots

9. Simplifications & ceilings

10. Go further

  1. Write Barnes–Hut in Numba. Build an octree, walk it with opening angle θ, then plot force error vs θ and find the N where the tree beats accel_par. It should be around a few thousand.
  2. Go to the GPU. Port accel to numba.cuda or CuPy with shared-memory tiling (Nyland et al. 2007). Expect 10¹⁰–10¹¹ pairs/s and 10⁵ bodies in real time.
  3. Rebuild it in an industry tool. Generate the same galaxies with GalIC or MakeNewDisk and run them in GADGET-4 or SWIFT. Or drive the GPU tree code Bonsai from AMUSE in Python with 10⁶ particles and compare the tails.
  4. Toomre's atlas / match the Antennae. Sweep disk inclinations and pericentre (a multiprocessing pool as in L04) and compare against the Identikit models of real interacting pairs (Barnes & Hibbard 2009, Privon et al. 2013).
  5. Add physics. Use an NFW halo + bulge from galpy distribution functions, then SPH gas to see merger-driven inflows and starbursts (Mihos & Hernquist 1996).

11. References

Every output

Source code

sim.py 540 lines
"""L05 Galactic Collision: direct-sum N-body in Numba, leapfrog vs Euler, and a Toomre-style merger.

Run from the repo root:   uv run python L05_galactic_collision/sim.py          (full run, ~5 min)
                          uv run python L05_galactic_collision/sim.py --quick  (smoke test, ~1 min)
"""
import os

os.environ.setdefault("NUMBA_NUM_THREADS", "4")  # shared machine: cap threads BEFORE numba loads
os.environ.setdefault("OMP_NUM_THREADS", "4")
import math
import sys
import time
import timeit
from multiprocessing import Pool
from pathlib import Path

import imageio_ffmpeg
import matplotlib

matplotlib.use("Agg")
import matplotlib.pyplot as plt
import numpy as np
from matplotlib.animation import FFMpegWriter
from numba import njit, prange
from scipy.ndimage import gaussian_filter, gaussian_filter1d
from scipy.spatial import cKDTree
from scipy.spatial.transform import Rotation

QUICK = "--quick" in sys.argv
OUT = Path(__file__).parent / ("output/quick" if QUICK else "output")   # smoke tests never clobber the real media
plt.rcParams["animation.ffmpeg_path"] = imageio_ffmpeg.get_ffmpeg_exe()

# N-body units: G = 1, length = 3.5 kpc (disk scale length), mass = 3e10 Msun (one stellar disk).
L_KPC, M_SUN = 3.5, 3e10
V_KMS = math.sqrt(4.3009e-6 * M_SUN / L_KPC)  # G in kpc (km/s)^2 / Msun -> ~192 km/s
T_MYR = L_KPC * 3.0857e16 / V_KMS / 3.156e13  # kpc->km, s->Myr        -> ~17.8 Myr

# Collision set-up (sized so the full run fits in ~5 min on 4 threads)
N_DISK, N_HALO = (1500, 1000) if QUICK else (5000, 3000)  # per galaxy
EPS = 0.08  # Plummer softening (~280 pc)
DT = 0.05  # ~0.9 Myr. ponytail: one global dt, per-particle hierarchical steps (GADGET, SWIFT) for cusps
T_END = 40.0 if QUICK else 120.0  # ~2.1 Gyr
FRAME_EVERY = 4
RP, D0 = 5.0, 20.0  # pericentre of the point-mass parabola and starting separation (N-body lengths)


# ----------------------------------------------------------------------------- gravity kernels
def accel(px, py, pz, m, eps2, ax, ay, az):
    """Direct-sum acceleration on every particle (G = 1, Plummer softening), O(N^2).

    One source, five speeds: run as plain Python on lists, or compiled by the njit variants below.
    Structure-of-arrays (separate x, y, z) keeps the inner j-loop on contiguous memory -> SIMD.
    The i == j term is harmless: dx = 0 and eps2 > 0.
    """
    n = len(px)  # ponytail: O(N^2) pairs, Barnes-Hut tree / FMM / TreePM once N >> 1e5
    for i in prange(n):
        xi, yi, zi = px[i], py[i], pz[i]
        sx = sy = sz = 0.0
        for j in range(n):
            dx = px[j] - xi
            dy = py[j] - yi
            dz = pz[j] - zi
            r2 = dx * dx + dy * dy + dz * dz + eps2
            w = m[j] / (r2 * math.sqrt(r2))
            sx += dx * w
            sy += dy * w
            sz += dz * w
        ax[i] = sx
        ay[i] = sy
        az[i] = sz


accel_jit = njit(accel)  # compiled, strict IEEE maths, 1 thread
# + fastmath (sums may be reordered) and the numpy error model (no ZeroDivisionError branch in the loop):
# both are needed before LLVM will vectorise the j-loop with AVX (4 doubles / 8 floats per instruction)
accel_simd = njit(fastmath=True, error_model="numpy")(accel)
accel_par = njit(parallel=True, fastmath=True, error_model="numpy")(accel)  # + i-loop over NUMBA_NUM_THREADS


@njit(parallel=True, fastmath=True)
def accel_aos(x, m, eps2, a):
    """Same maths on an (N, 3) array-of-structs: strided loads, so the compiler struggles to vectorise."""
    for i in prange(x.shape[0]):
        sx = sy = sz = 0.0
        for j in range(x.shape[0]):
            dx = x[j, 0] - x[i, 0]
            dy = x[j, 1] - x[i, 1]
            dz = x[j, 2] - x[i, 2]
            r2 = dx * dx + dy * dy + dz * dz + eps2
            w = m[j] / (r2 * math.sqrt(r2))
            sx += dx * w
            sy += dy * w
            sz += dz * w
        a[i, 0] = sx
        a[i, 1] = sy
        a[i, 2] = sz


def accel_numpy(x, m, eps2):
    """NumPy broadcasting: builds (3, N, N) temporaries, so memory caps N at a few thousand."""
    d = x[:, None, :] - x[:, :, None]  # d[k, i, j] = x_j - x_i
    r2 = (d * d).sum(0) + eps2
    return (d * (m / (r2 * np.sqrt(r2)))).sum(2)


@njit(parallel=True, fastmath=True)
def potential(px, py, pz, m, eps2):
    """Total softened potential energy: sum over pairs of -m_i m_j / sqrt(r^2 + eps^2)."""
    n = len(px)
    w = 0.0
    for i in prange(n):
        s = 0.0
        for j in range(i + 1, n):
            dx = px[j] - px[i]
            dy = py[j] - py[i]
            dz = pz[j] - pz[i]
            s += m[j] / math.sqrt(dx * dx + dy * dy + dz * dz + eps2)
        w -= m[i] * s
    return w


def gravity(m, eps, kernel=accel_par, dtype=np.float64):
    """force(x) -> accelerations, both (3, N). dtype=float32 is the GPU-style mixed-precision mode:
    pair maths in float32, sums in float64, state (x, v) stays float64."""
    mk, e2 = m.astype(dtype), dtype(eps * eps)

    def force(x):
        xk = x.astype(dtype)
        a = np.empty_like(xk)
        kernel(xk[0], xk[1], xk[2], mk, e2, a[0], a[1], a[2])
        return a

    return force


def energy(x, v, m, eps):
    return 0.5 * np.sum(m * (v * v).sum(0)) + potential(x[0], x[1], x[2], m, eps * eps)


# ----------------------------------------------------------------------------- integrators
def leapfrog(x, v, force, dt, nsteps):
    """Kick-drift-kick leapfrog (symplectic, time-reversible, 2nd order). Updates x, v in place."""
    a = force(x)
    for n in range(1, nsteps + 1):
        v += 0.5 * dt * a
        x += dt * v
        a = force(x)
        v += 0.5 * dt * a
        yield n, x, v


def euler(x, v, force, dt, nsteps):
    """Explicit (forward) Euler: 1st order and NOT symplectic -> energy drifts secularly."""
    for n in range(1, nsteps + 1):
        a = force(x)
        x += dt * v
        v += dt * a
        yield n, x, v


# ----------------------------------------------------------------------------- verification
def verify():
    """Two-body Kepler orbit: period vs 2 pi sqrt(a^3 / G M), and leapfrog vs Euler energy error."""
    m, a_, e, eps = np.array([1.0, 0.5]), 1.0, 0.5, 1e-6  # eps << r: effectively Newtonian
    M = m.sum()
    T = 2 * np.pi * np.sqrt(a_**3 / M)
    r0, v0 = a_ * (1 + e), np.sqrt(M * (1 - e) / (a_ * (1 + e)))  # start at apocentre
    x0 = np.zeros((3, 2))
    u0 = np.zeros((3, 2))
    x0[0] = [-m[1] / M * r0, m[0] / M * r0]  # centre-of-mass frame
    u0[1] = [-m[1] / M * v0, m[0] / M * v0]
    orbits, per = 20, 500
    dt, n = T / per, orbits * per
    force = gravity(m, eps, accel_jit)
    E0 = energy(x0, u0, m, eps)
    res = {}
    for name, stepper in (("leapfrog", leapfrog), ("Euler", euler)):
        rel, dE = [], []
        for _, x, v in stepper(x0.copy(), u0.copy(), force, dt, n):
            rel.append(x[:2, 1] - x[:2, 0])
            dE.append(abs(energy(x, v, m, eps) / E0 - 1))
        res[name] = np.array(rel), np.array(dE)
    rel = res["leapfrog"][0]
    theta = np.unwrap(np.arctan2(rel[:, 1], rel[:, 0]))
    t = dt * np.arange(1, n + 1)
    k = int(theta[-1] // (2 * np.pi))
    T_meas = np.interp(2 * np.pi * k, theta, t) / k  # time of the k-th full revolution / k
    err_T = abs(T_meas / T - 1)
    lf, eu = res["leapfrog"][1], res["Euler"][1]
    print(f"[verify] Kepler period: analytic {T:.6f}, leapfrog {T_meas:.6f} ({k} orbits), error {err_T:.2e}")
    print(f"[verify] |dE/E| after {orbits} orbits: leapfrog max {lf.max():.2e} (last orbit {lf[-per:].max():.2e}),"
          f" Euler final {eu[-1]:.2e}")
    assert err_T < 0.01, "Kepler period off by more than 1%"
    assert lf[-per:].max() < 2 * lf[:per].max(), "leapfrog energy error is growing (should stay bounded)"
    assert eu[-1] > 100 * lf.max(), "Euler should drift far beyond leapfrog"

    fig, (a1, a2) = plt.subplots(1, 2, figsize=(12, 5), gridspec_kw={"width_ratios": [1, 1.4]})
    for (name, (r, _)), c in zip(res.items(), ("#4cc9f0", "#f72585")):
        a1.plot(r[:, 0], r[:, 1], lw=0.6, color=c, label=name)
    a1.plot(0, 0, "*", color="w", ms=10)
    a1.set(aspect="equal", xlabel="x (relative, N-body length)", ylabel="y (relative)",
           title=f"Relative orbit, e = {e}, {orbits} orbits, {per} steps/orbit")
    a1.legend(loc="upper left")
    a2.semilogy(t / T, lf, color="#4cc9f0", lw=0.8, label="leapfrog (symplectic): bounded")
    a2.semilogy(t / T, eu, color="#f72585", lw=1.2, label="explicit Euler: secular drift")
    a2.set(xlabel="time (orbital periods)", ylabel="|E(t)/E0 - 1|",
           title=f"Energy error. Period check: {T_meas:.5f} vs {T:.5f} ({err_T:.1e})")
    a2.legend(loc="lower right")
    a2.grid(alpha=0.2)
    fig.tight_layout()
    fig.savefig(OUT / "verify_kepler_energy.png", dpi=130)
    plt.close(fig)


# ----------------------------------------------------------------------------- benchmark
def benchmark():
    """Interactions per second for each rung of the performance ladder, N = 64 ... 65536."""
    rng = np.random.default_rng(1)
    e2 = EPS * EPS
    tiers = [  # name, largest N worth timing, make(x, m) -> zero-arg callable
        ("pure Python (lists)", 512,
         lambda x, m: lambda: accel(*x.tolist(), m.tolist(), e2, *([0.0] * len(m) for _ in range(3)))),
        ("NumPy broadcasting", 2048, lambda x, m: lambda: accel_numpy(x, m, e2)),
        ("Numba @njit", 8192, lambda x, m: lambda f=gravity(m, EPS, accel_jit): f(x)),
        ("+ fastmath (SIMD)", 16384, lambda x, m: lambda f=gravity(m, EPS, accel_simd): f(x)),
        ("+ parallel, 4 threads", 32768, lambda x, m: lambda f=gravity(m, EPS, accel_par): f(x)),
        ("+ float32 (production)", 65536, lambda x, m: lambda f=gravity(m, EPS, accel_par, np.float32): f(x)),
        ("parallel, AoS (N,3) layout", 32768,
         lambda x, m: lambda xa=np.ascontiguousarray(x.T): accel_aos(xa, m, e2, np.empty_like(xa))),
    ]
    Ns = [2**k for k in range(6, 17)]
    rates = {}
    for name, nmax, make in tiers:
        rates[name] = []
        for N in [N for N in Ns if N <= nmax]:
            x, m = rng.normal(size=(3, N)), np.full(N, 1.0 / N)
            f = make(x, m)
            t0 = time.perf_counter()
            f()  # warm-up (includes JIT compile on the very first call)
            if N == Ns[0] and "Python" not in name and "NumPy" not in name:
                print(f"[bench] {name}: first call incl. JIT compile {time.perf_counter() - t0:.2f} s")
            n = max(1, int(0.05 / timeit.timeit(f, number=1)))  # batch tiny calls to >= 50 ms per sample
            t = min(timeit.repeat(f, number=n, repeat=3)) / n  # best of 3: least disturbed by other jobs
            rates[name].append((N, N * N / t))
        N, r = rates[name][-1]
        print(f"[bench] {name:28s} N={N:6d}  {r:10.3e} interactions/s  ({N * N / r * 1e3:9.2f} ms per force eval)")

    base = rates["pure Python (lists)"][-1][1]
    print("\n| rung | largest N timed | interactions/s | speed-up vs pure Python |\n|---|---|---|---|")
    for name, rr in rates.items():
        print(f"| {name} | {rr[-1][0]} | {rr[-1][1]:.2e} | {rr[-1][1] / base:,.0f}x |")
    print()

    fig, (a1, a2) = plt.subplots(1, 2, figsize=(13, 5.2))
    colors = ["#adb5bd", "#f4a261", "#2a9d8f", "#4cc9f0", "#7b2cbf", "#f72585", "#7b2cbf"]
    for (name, rr), c in zip(rates.items(), colors):
        N, r = np.array(rr).T
        ls = "--" if "AoS" in name else "-"
        a1.loglog(N, r, ls, marker="o", ms=4, color=c, label=name)
        a2.loglog(N, N * N / r, ls, marker="o", ms=4, color=c, label=name)
    Nr = np.array([64, 65536])
    a2.loglog(Nr, 5e-9 * Nr**2, ":", color="w", alpha=0.6, label=r"$\propto N^2$")
    a1.set(xlabel="N (particles)", ylabel="pair interactions per second",
           title="Throughput: each rung of the ladder (4 threads max)")
    a2.set(xlabel="N (particles)", ylabel="wall time per force evaluation (s)",
           title="Cost of one force evaluation: direct sum is O(N$^2$)")
    for a in (a1, a2):
        a.grid(alpha=0.25)
        a.grid(alpha=0.07, which="minor")
    a1.legend(fontsize=8, loc="lower right")
    fig.tight_layout()
    fig.savefig(OUT / "benchmark.png", dpi=130)
    plt.close(fig)
    return rates


# ----------------------------------------------------------------------------- initial conditions
def unit_vectors(n, rng):
    u = rng.normal(size=(3, n))
    return u / np.linalg.norm(u, axis=0)


def plummer(n, M, b, rng):
    """Isotropic Plummer sphere, Aarseth, Henon & Wielen (1974) recipe."""
    r = b / np.sqrt(rng.uniform(0, 0.99, n) ** (-2 / 3) - 1)  # ponytail: outer 1% of mass cut off
    q = np.empty(0)
    while q.size < n:  # rejection-sample q = v / v_esc from g(q) ~ q^2 (1 - q^2)^3.5
        u, w = rng.random(n), 0.1 * rng.random(n)
        q = np.r_[q, u[w < u**2 * (1 - u**2) ** 3.5]]
    v = q[:n] * np.sqrt(2 * M / np.sqrt(r * r + b * b))
    return r * unit_vectors(n, rng), v * unit_vectors(n, rng)


def galaxy(n_disk, n_halo, rng, Md=1.0, Rd=1.0, z0=0.1, Rmax=6.0, Mh=5.0, b=2.0, Q=1.3):
    """Exponential stellar disk (sech^2 vertical) inside a live Plummer dark-matter halo."""
    Rg = np.linspace(0, Rmax, 2000)
    cum = 1 - (1 + Rg / Rd) * np.exp(-Rg / Rd)  # enclosed mass of an exponential disk
    R = np.interp(rng.random(n_disk) * cum[-1], cum, Rg)
    phi = rng.uniform(0, 2 * np.pi, n_disk)
    z = z0 * np.arctanh(rng.uniform(-1, 1, n_disk))
    xd = np.array([R * np.cos(phi), R * np.sin(phi), z])
    xh, vh = plummer(n_halo, Mh, b, rng)
    x = np.c_[xd, xh]
    m = np.r_[np.full(n_disk, Md / n_disk), np.full(n_halo, Mh / n_halo)]

    # Rotation curve measured from the particles themselves: mean radial pull on massless test rings
    rr, ang = np.meshgrid(np.linspace(0.05, Rmax, 60), np.linspace(0, 2 * np.pi, 32, endpoint=False))
    ring = np.array([rr * np.cos(ang), rr * np.sin(ang), 0 * rr]).reshape(3, -1)
    a = gravity(np.r_[m, np.zeros(ring.shape[1])], EPS)(np.c_[x, ring])[:, -ring.shape[1]:]
    vc2 = np.maximum(-(a[0] * ring[0] + a[1] * ring[1]).reshape(rr.shape).mean(0), 1e-6)
    r1 = rr[0]
    Om2 = vc2 / r1**2
    kappa = np.sqrt(np.maximum(r1 * np.gradient(Om2, r1) + 4 * Om2, 1e-6))  # epicyclic frequency
    Sig = Md / (2 * np.pi * Rd**2) * np.exp(-r1 / Rd)
    sR = Q * 3.36 * Sig / kappa  # Toomre Q = sigma_R kappa / (3.36 G Sigma)
    sphi = sR * kappa / (2 * np.sqrt(Om2))  # epicyclic ratio
    sz = np.sqrt(np.pi * Sig * z0)  # isothermal sheet
    vc, sR, sphi, sz = (np.interp(R, r1, f) for f in (np.sqrt(vc2), sR, sphi, sz))
    # ponytail: no asymmetric-drift correction, the disk relaxes a little in the first orbit
    vR = sR * rng.normal(size=n_disk)
    vphi = vc + sphi * rng.normal(size=n_disk)
    vd = np.array([vR * np.cos(phi) - vphi * np.sin(phi), vR * np.sin(phi) + vphi * np.cos(phi),
                   sz * rng.normal(size=n_disk)])
    v = np.c_[vd, vh]
    return x - x @ m[:, None] / m.sum(), v - v @ m[:, None] / m.sum(), m  # centre-of-mass frame


def collision_ics(rng, rp=RP, d0=D0, tilt1=(0, 0), tilt2=(60, 30)):
    """Two identical galaxies on a parabolic orbit (Toomre & Toomre 1972). tilt = (inclination, azimuth)
    in degrees of each disk's spin relative to the orbital angular momentum (+z). (0, 0) = prograde."""
    xs, vs, ms, gal, disk, core = [], [], [], [], [], []
    for g, tilt in enumerate((tilt1, tilt2)):
        x, v, m = galaxy(N_DISK, N_HALO, rng)
        rot = Rotation.from_euler("xz", tilt, degrees=True)
        xs.append(rot.apply(x.T).T)
        vs.append(rot.apply(v.T).T)
        ms.append(m)
        gal.append(np.full(len(m), g))
        disk.append(np.arange(len(m)) < N_DISK)
        core.append(disk[-1] & (np.linalg.norm(x, axis=0) < 0.5))  # nucleus tracers
    M1, M2 = ms[0].sum(), ms[1].sum()
    p = 2 * rp  # semi-latus rectum of a parabola
    nu = -np.arccos(p / d0 - 1)  # true anomaly at separation d0, before pericentre
    rel_x = d0 * np.array([np.cos(nu), np.sin(nu), 0])
    rel_v = np.sqrt((M1 + M2) / p) * np.array([-np.sin(nu), 1 + np.cos(nu), 0])
    for i, s in ((0, -M2), (1, M1)):  # both galaxies about the common centre of mass
        xs[i] += (s / (M1 + M2) * rel_x)[:, None]
        vs[i] += (s / (M1 + M2) * rel_v)[:, None]
    t_peri = 0.5 * np.sqrt(p**3 / (M1 + M2)) * (np.tan(-nu / 2) + np.tan(-nu / 2) ** 3 / 3)  # Barker
    return np.hstack(xs), np.hstack(vs), np.hstack(ms), np.hstack(gal), np.hstack(disk), np.hstack(core), t_peri


# ----------------------------------------------------------------------------- the collision
def collide():
    rng = np.random.default_rng(42)
    x, v, m, gal, disk, core, t_peri = collision_ics(rng)
    N = len(m)
    nsteps = int(round(T_END / DT))
    core = [np.flatnonzero(core & (gal == g)) for g in (0, 1)]
    force = gravity(m, EPS, accel_par, np.float32)
    frames, t_frames, t_E, E, sep = [], [], [0.0], [energy(x, v, m, EPS)], []
    t0 = time.perf_counter()
    for step, x, v in leapfrog(x, v, force, DT, nsteps):
        if step % FRAME_EVERY == 0:
            frames.append(x[:, disk].astype(np.float32))
            t_frames.append(step * DT)
            c = [np.median(x[:, idx], axis=1) for idx in core]
            sep.append(np.linalg.norm(c[1] - c[0]))
        if step % 50 == 0:
            t_E.append(step * DT)
            E.append(energy(x, v, m, EPS))
        if step % 500 == 0:
            el = time.perf_counter() - t0
            print(f"[collide] step {step}/{nsteps}  t = {step * DT * T_MYR:6.0f} Myr  "
                  f"{el:6.1f} s  ({el / step * 1e3:.1f} ms/step)")
    wall = time.perf_counter() - t0
    E = np.array(E)
    dE = np.abs(E / E[0] - 1)
    print(f"[collide] N = {N}, {nsteps} steps in {wall:.1f} s ({wall / nsteps * 1e3:.1f} ms/step, "
          f"{N * N * nsteps / wall:.2e} interactions/s)")
    print(f"[collide] energy error max |dE/E| = {dE.max():.2e}, final {dE[-1]:.2e}")
    return dict(frames=np.array(frames), t=np.array(t_frames), gal=gal[disk], sep=np.array(sep),
                t_E=np.array(t_E), dE=dE, t_peri=t_peri, N=N, m_disk=m[0], M=m.sum(), wall=wall)


# ----------------------------------------------------------------------------- rendering
COLORS = np.array([[1.0, 0.60, 0.25], [0.30, 0.62, 1.0]], np.float32)  # galaxy 1 gold, galaxy 2 azure
VIEW = Rotation.from_euler("zx", [90, -30], degrees=True)  # turn tails along the wide axis, tilt 30 deg


def render(p, gal, m_disk, half_w, h, w):
    """Surface density on black with an asinh (log-like) stretch, one hue per galaxy -> (h, w, 3) uint8.

    Two layers per galaxy: diffuse light with SPH-style adaptive smoothing (Gaussian width set by the
    distance to the 8th-nearest neighbour, so sparse tails blur into streams and dense cores stay
    sharp), plus every particle as a sharp 'star'. Pixel scales are relative to 720p.
    """
    q = VIEW.apply(p.T).T
    half_h, s = half_w * h / w, h / 720
    rc = np.c_[(q[1] / half_h + 1) * h / 2, (q[0] / half_w + 1) * w / 2]  # (row, col) in pixels
    to_sigma = m_disk / (2 * half_w / w) ** 2  # particles per pixel -> surface density (N-body units)
    stretch = lambda S, S0: np.arcsinh(S * to_sigma / S0) / np.arcsinh(0.3 / S0)  # 1 at Sigma = 0.3
    img = np.zeros((h, w, 3), np.float32)
    for g, c in enumerate(COLORS):
        pg = rc[gal == g]
        hist = lambda sel: np.histogram2d(pg[sel, 0], pg[sel, 1], bins=(h, w),
                                          range=((0, h), (0, w)))[0].astype(np.float32)
        d8 = cKDTree(pg).query(pg, k=8)[0][:, -1]
        width = np.clip(np.round(np.log2(np.maximum(0.5 * d8 / s, 1))), 1, 3)  # classes: 2, 4, 8 px
        glow = sum(gaussian_filter(hist(width == k), s * 2.0**k, truncate=3) for k in (1, 2, 3))
        stars = gaussian_filter(hist(slice(None)), 0.7 * s)
        img += (stretch(glow, 2e-3) + 0.6 * stretch(stars, 1e-2))[..., None] * c
    return (255 * (1 - np.exp(-1.6 * img))).astype(np.uint8)  # tone map: bright cores go white


def zoom_schedule(frames):
    """Half-width of the view per frame: 1.8 x the 90th-percentile extent, smoothed, kept in [16, 34]."""
    ext = [np.percentile(np.maximum(abs(q[0]), 16 / 9 * abs(q[1])), 90) for q in (VIEW.apply(f.T).T for f in frames)]
    return np.clip(gaussian_filter1d(1.8 * np.array(ext), 30, mode="nearest"), 16, 34)


def _render_args(a):
    return render(*a)


def make_movie(run, path, w=1280, h=720, fps=30):
    """Frames are rendered by 4 worker processes; the main process pipes them to ffmpeg in order."""
    frames, t, gal, m_disk = run["frames"], run["t"], run["gal"], run["m_disk"]
    hw = zoom_schedule(frames)
    fig = plt.figure(figsize=(w / 100, h / 100), dpi=100, facecolor="k")
    ax = fig.add_axes((0, 0, 1, 1))
    ax.set_axis_off()
    im = ax.imshow(np.zeros((h, w, 3), np.uint8), origin="lower", extent=(0, w, 0, h), interpolation="none")
    ax.set(xlim=(0, w), ylim=(0, h))
    txt = ax.text(24, h - 42, "", color="w", fontsize=15, family="monospace")
    ax.text(24, h - 66, "gold: galaxy 1, disk in the orbital plane   azure: galaxy 2, disk tilted 60 deg",
            color="#8d99ae", fontsize=10)
    ax.text(w - 24, 20, f"N = {run['N']:,} bodies  |  direct summation + leapfrog  |  Numba, 4 threads",
            ha="right", color="#8d99ae", fontsize=10)
    bar, = ax.plot([], [], color="w", lw=2)
    ax.text(24, 32, "20 kpc", color="w", fontsize=10)
    writer = FFMpegWriter(fps=fps, bitrate=5000, extra_args=["-pix_fmt", "yuv420p"])
    with Pool(4) as pool, writer.saving(fig, str(path), dpi=100):
        jobs = ((f, gal, m_disk, s, h, w) for f, s in zip(frames, hw))
        for img, tt, s in zip(pool.imap(_render_args, jobs, chunksize=2), t, hw):
            im.set_data(img)
            txt.set_text(f"t = {tt * T_MYR / 1000:4.2f} Gyr")
            bar.set_data([24, 24 + 20 / L_KPC / (2 * s) * w], [24, 24])  # 20 kpc in pixels
            writer.grab_frame(facecolor="k")
    plt.close(fig)


def make_stills(run):
    frames, t, gal, m_disk, sep = run["frames"], run["t"], run["gal"], run["m_disk"], run["sep"]
    hw = zoom_schedule(frames)
    # hero: ~230 Myr after first pericentre, when the tails and the bridge are at their most striking
    i = min(int(np.argmin(sep[: len(sep) // 2])) + int(13 / (DT * FRAME_EVERY)), len(frames) - 1)
    W, H = 2400, 1350
    fig = plt.figure(figsize=(W / 150, H / 150), dpi=150, facecolor="k")
    ax = fig.add_axes((0, 0, 1, 1))
    ax.set_axis_off()
    ax.imshow(render(frames[i], gal, m_disk, 1.2 * hw[i], H, W), origin="lower", interpolation="none")
    ax.text(0.02, 0.945, "Galactic collision", transform=ax.transAxes, color="w", fontsize=22)
    ax.text(0.02, 0.905, f"two disk galaxies in live dark-matter halos, {run['N']:,} bodies, "
            f"t = {t[i] * T_MYR / 1000:.2f} Gyr", transform=ax.transAxes, color="#8d99ae", fontsize=11)
    fig.savefig(OUT / "hero.png", dpi=150, facecolor="k")
    plt.close(fig)

    fig, axs = plt.subplots(2, 3, figsize=(15, 6.6), facecolor="k", layout="constrained")
    for a, j in zip(axs.flat, np.linspace(0, len(frames) - 1, 6).astype(int)):
        s = hw[j] * L_KPC
        a.imshow(render(frames[j], gal, m_disk, hw[j], 360, 640), origin="lower",
                 extent=(-s, s, -s * 9 / 16, s * 9 / 16), interpolation="none")
        a.set_title(f"t = {t[j] * T_MYR / 1000:.2f} Gyr", color="w", fontsize=11)
        a.tick_params(colors="#8d99ae", labelsize=8)
        a.set_xlabel("x (kpc)", color="#8d99ae", fontsize=9)
        a.set_ylabel("y (kpc)", color="#8d99ae", fontsize=9)
    fig.suptitle("Approach, first pericentre, tidal tails + bridge, apocentre, second passage, merger remnant",
                 color="w")
    fig.savefig(OUT / "sequence.png", dpi=110, facecolor="k")
    plt.close(fig)
    return i


def diagnostics(run):
    """Nucleus separation vs the point-mass (Keplerian) parabola, and energy conservation of the run."""
    t, sep, M, t_peri = run["t"], run["sep"], run["M"], run["t_peri"]
    rp, p = RP, 2 * RP
    D = np.linspace(-np.tan(np.arccos(p / D0 - 1) / 2), 6, 2000)  # Barker's equation for the parabola
    tk = t_peri + 0.5 * np.sqrt(p**3 / M) * (D + D**3 / 3)
    i = int(np.argmin(sep[: len(sep) // 2]))
    j = i + int(np.argmax(sep[i:]))
    k = int(np.flatnonzero(sep > 2 / L_KPC)[-1]) + 1  # nuclei stay within 2 kpc after this
    merged = f"from t = {t[k] * T_MYR / 1e3:.2f} Gyr" if k < len(t) else "not yet (run too short)"
    print(f"[diag] first pericentre t = {t[i] * T_MYR / 1e3:.2f} Gyr at {sep[i] * L_KPC:.1f} kpc "
          f"(Kepler parabola: {t_peri * T_MYR / 1e3:.2f} Gyr at {RP * L_KPC:.1f} kpc); "
          f"apocentre {sep[j] * L_KPC:.0f} kpc at {t[j] * T_MYR / 1e3:.2f} Gyr; "
          f"nuclei within 2 kpc for good: {merged}")
    fig, (a1, a2) = plt.subplots(1, 2, figsize=(13, 4.8))
    a1.plot(t * T_MYR / 1000, sep * L_KPC, color="#f4a261", lw=2, label="N-body: live halos")
    ok = tk <= t[-1]
    a1.plot(tk[ok] * T_MYR / 1000, rp * (1 + D[ok] ** 2) * L_KPC, "--", color="#4cc9f0",
            label="point masses (Kepler parabola)")
    a1.set(xlabel="time (Gyr)", ylabel="nucleus separation (kpc)",
           title="Dynamical friction: halos soak up orbital energy -> merger")
    a1.legend()
    a2.semilogy(run["t_E"][1:] * T_MYR / 1000, run["dE"][1:], color="#f72585")
    a2.set(xlabel="time (Gyr)", ylabel="|E(t)/E0 - 1|",
           title=f"Energy conservation, KDK leapfrog, dt = {DT * T_MYR:.2f} Myr, float32 forces")
    for a in (a1, a2):
        a.grid(alpha=0.2)
    fig.tight_layout()
    fig.savefig(OUT / "collision_diagnostics.png", dpi=130)
    plt.close(fig)


def main():
    OUT.mkdir(parents=True, exist_ok=True)
    plt.style.use("dark_background")
    T0 = time.perf_counter()
    stamp = lambda what: print(f"[time] {what} done, {time.perf_counter() - T0:.0f} s elapsed")
    print(f"units: length {L_KPC} kpc, velocity {V_KMS:.0f} km/s, time {T_MYR:.2f} Myr, mass {M_SUN:.0e} Msun")
    if not QUICK:
        benchmark()  # first, so the JIT compile times it prints are real
        stamp("benchmark")
    verify()
    stamp("verification")
    run = collide()
    stamp("collision")
    diagnostics(run)
    i = make_stills(run)
    print(f"[render] hero frame t = {run['t'][i] * T_MYR / 1000:.2f} Gyr")
    make_movie(run, OUT / "collision.mp4")
    print(f"[render] MP4: {len(run['frames'])} frames, {(OUT / 'collision.mp4').stat().st_size / 1e6:.1f} MB")
    stamp("rendering (total runtime)")


if __name__ == "__main__":
    main()