spectrogram
The spectrograms of a lane's clips on one canvas and one WebGL context, computed on the GPU by gl-spectrogram on one of four frequency axes, with the band zoom, the pointer's cell and the frequency axis laid out for a lane.
pnpm add @audio-ui/gl
import { zoomBand, panBand, loadScales, createSpectrogramLane, readHertz, /* … */ } from "@audio-ui/gl/spectrogram";| Export | Kind | What it is |
|---|---|---|
SpectrogramScale | type | The frequency axes a spectrogram is spaced on: octaves, mel, equal space per auditory filter, or hertz. |
SpectrogramBand | interface | The frequencies a spectrogram shows, as shares of its axis from 0 (the floor) to 1 (the Nyquist limit), whatever the axis spaces its rows by: the clips draw this band, the axis labels it. |
zoomBand | function | The band factor times as wide (under 1 zooms in), the share at of it (0 at its foot) staying where it is. |
panBand | function | The band moved by share of its own width, up or down, kept inside the axis. |
loadScales | function | The axes the rows are spaced on, for labels that match them: scales[name].at(f, lo, hi) is where f hertz sits between lo and hi, 0 to 1. |
SpectrogramHit | interface | What the pointer is over on a spectrogram: the cell's frequencies and level, the floor of the levels drawn, and where the pointer is. |
SpectrogramInputs | interface | What a spectrogram lane draws. |
SpectrogramHooks | interface | What a spectrogram lane tells its host. |
createSpectrogramLane | function | Draws the spectrograms of a lane's clips on canvas, which sits in the lane's box, each clip in its own part of it: their frequencies over the stretch of the lane the view shows, computed on the GPU by gl-spectrogram at every zoom, in the text colour over the ground under it. |
readHertz | const | A frequency the pointer reads, as the axis writes it: three significant digits, five characters at most (12.3k, 523, 52.3). |
hertzLabel | const | A frequency as the axis writes it, no unit: 500, 1.25k, 20k. |
SpectrogramAxisLayout | interface | What spectrogramAxis lays out: a frequency's place, the labels with their ticks, the ticks between, and where the pointer is. |
spectrogramAxis | function | Lays out a spectrogram's frequency axis for a lane height px tall, as audio.js draws it: labels roundest first (20k, 2k, 200, then between as room allows), each with its tick at the row its frequency sits on by the axis the picture is spaced on, and shorter ticks between, the band in view respected. |
hasWebGL2 | function | Whether this browser can draw with WebGL 2: the lanes that need it say no, and the page draws them another way. |
Lane | interface | A lane that is drawing. |
LaneClip | interface | A clip of a lane to draw: where it lies, how long it is, and how it reads its recording. |
LaneSamples | interface | A recording's samples, mixed to one: how many frames it holds, and a reader of the frames from from to to. |
LaneSpectra | interface | A recording's spectra: a column every hop frames, each bin's level to Nyquist over windows of size frames, a byte each, (dB + 150) · 1.6, 0 silence: what gl-spectrogram draws a recording it does not hold from. |
Usage
createSpectrogramLane draws the spectrograms of a lane's clips on a canvas in the lane's box, each clip in its own parts of it. Instead of peaks it draws the frequencies of the recording, each column an FFT computed on the GPU by gl-spectrogram, so a steady tone is one row thin and a click one column thin at any zoom.
import { createSpectrogramLane } from "@audio-ui/gl/spectrogram";
const lane = createSpectrogramLane(
canvas,
{ band, clips, scale: "log", zoom: pixelsPerSecond }, // "log" octaves, "mel", "erb" or "lin" hertz
{ onHit: (hit) => { /* the cell under the pointer, or null */ } }
);
lane.update({ band, clips, scale: "mel", zoom });
lane.destroy();onHit gives the cell the pointer is over: its low and high frequencies, its level in dB, the floor of the levels drawn (a cell under it has nothing to read), and the pointer's place.
Streamed recordings
A recording streamed from a file is never read whole. Its samples give spectra: a column every hop frames, each bin's level to Nyquist over Hann windows of size frames, a byte each, as spectrogramSpectra of @audio-ui/engine/analysis makes them from its samples, once, and kept beside it (SPECTRA_SIZE, 2048 frames, for both by default). Zoomed out the lane draws from them; once a column of the view is narrower than one of theirs it reads the frames it shows, a chunk of CHUNK_FRAMES (65536) at a time, and the renderer computes those columns from the samples.
import { spectrogramSpectra } from "@audio-ui/engine/analysis";
// Once: read a stretch at a time from any reader, at once or as a promise.
const spectra = await spectrogramSpectra({ frames, read: readFrames });
const samples = { frames, read: readFrames, spectra: () => spectra };A full-scale sine reads 0 dB on its bin; the byte keeps a level to 0.6 dB.
The band
band is the frequencies shown, as shares of the axis from 0 (the floor) to 1 (the Nyquist limit), whatever the axis spaces its rows by: anything with getSnapshot, set and subscribe, such as createSource([0, 1]) of @audio-ui/core. Shift and the wheel over the picture zoom it about the pointer. zoomBand and panBand are the arithmetic, to drive it from your own controls.
The axis
spectrogramAxis lays out the frequency axis for a lane of a given height, as audio.js draws it: labels roundest first (20k, 2k, 200, 20, then between as room allows), each with its tick at the row its frequency sits on by the axis the picture is spaced on, shorter ticks between, the band in view respected. Zoomed in, it labels the round values in view, 1-2-5 a decade across a wide span, even steps across a narrow one. A pointer frequency is marked and the labels near it give way. loadScales gives the axes ((await loadScales())[scale]), hertzLabel and readHertz write a frequency as the axis and as the pointer's reading do.