pnpm add @audio-ui/gl
import { SPLIT_ROW_PX, LEAF_FRAMES, waveformLeaves, leavesPeak, splitsChannels, /* … */ } from "@audio-ui/gl/waveform";| Used by | use-loudest |
|---|
| Export | Kind | What it is |
|---|---|---|
WaveformBands | interface | The three signals a sound is split into to colour it by its frequencies: what it has low, between and high. |
WaveformClip | interface | A clip of the lane, with what its waveform needs besides what every lane clip has. |
WaveformLook | type | How the lane draws a waveform: as one, or as the three bands of the sound one over the other, the lows warm and the highs cool. |
WaveformLayout | type | How a recording with several channels is laid out: mix draws them as one waveform; split draws each channel in a row of its own, as a DAW does on a tall track; auto splits when each row would be at least SPLIT_ROW_PX tall, and mixes below that, as a short track shows one lane. |
SPLIT_ROW_PX | const | The least height of a channel's row, in CSS px, for auto to split. |
WaveformInputs | interface | What a waveform lane draws. |
WaveformLaneOptions | interface | How a waveform lane draws. |
LEAF_FRAMES | const | The frames a leaf of waveformLeaves sums up. |
waveformLeaves | function | The leaves of samples, from its first frame: [min, max, Σx², count] of each 256 frames, the last leaf of what is left, NaN left out (a leaf of nothing else is [Infinity, -Infinity, 0, 0]). |
leavesPeak | function | The loudest sample the leaves sum up, 0 to 1 for a recording at full scale at most: what a streamed clip's peak is. |
WaveformRow | interface | What a clip draws in one row of the lane: a layer of its recording, its samples, the loudest of them, and the row's place and height. |
splitsChannels | function | Whether a clip draws each channel in a row of its own at this layout, look and height. |
waveformRows | function | The rows a clip draws in a lane height px tall, with the samples of each: a channel apiece (channels, read), its recording whole, or its three bands. |
createWaveformLane | function | Draws the waveforms of a lane's clips on canvas, which sits in the lane's box, each clip in its own parts of it, on the GPU by gl-waveform: every pixel column shows the exact lowest and highest of its samples with the RMS inside, and zoomed in a line through the samples, however long the recording. |
hasWebGL2 | function | Whether this browser can draw with WebGL 2: the lanes that need it say no, and the page draws them another way. |
Lane | interface | A lane that is drawing. |
LaneClip | interface | A clip of a lane to draw: where it lies, how long it is, and how it reads its recording. |
LaneSamples | interface | A recording's samples, mixed to one: how many frames it holds, and a reader of the frames from from to to. |
Usage
createWaveformLane draws the waveforms of a lane's clips on a canvas in the lane's box, each clip in its own parts of it. Drawn by gl-waveform, every pixel column shows the exact lowest and highest of its samples with the RMS inside, and zoomed in it is a line through the samples with a dot on each: at any zoom, over recordings of any length. The color is the canvas' text color.
import { createWaveformLane } from "@audio-ui/gl/waveform";
const lane = createWaveformLane(canvas, {
clips, // WaveformClip[]: a LaneClip with the recording's `peak`, and `bands` for the band look
layout: "auto", // a row a channel once the rows are tall enough; or "split", "mix"
look: "plain", // or "bands"
zoom: pixelsPerSecond,
});
lane.update({ clips, look: "bands", zoom });
lane.destroy();A clip's peak is its recording's loudest sample, 0 to 1: the waveform is drawn as tall as the loudest reaches the lane's height, as a quiet recording still fills its lane. The clip's gain scales what is drawn, as it scales what plays.
Many lanes
A browser keeps some sixteen WebGL contexts and loses the oldest past that. createWaveformLane(canvas, inputs, { shared: true }) draws on the page's one context instead, a canvas never on the page, and copies the picture onto canvas, a 2D one: a page draws as many lanes as it has clips, one a clip as an arrangement draws them, and lanes reading one recording share its renderers, each let go of once no lane draws it. Leave zoom out for a lane that fits its clips to its box, and a clip's peak for the lane to read it, from its leaves or its samples. A lane of its own lets go of its context when it is destroyed (releaseContext of @audio-ui/gl/lane), unless it starts again on the same canvas at once, as React does in development.
Streamed recordings
A recording streamed from a file is never read whole. Its samples give leaves: the recording summed up 256 frames at a time, [min, max, Σx², count] each, as waveformLeaves makes them from its samples, once, as it is written, and kept beside it. Zoomed out the lane draws from them alone, exact at a leaf a device pixel or more; past that it reads the frames it shows, a chunk of CHUNK_FRAMES (65536) at a time, and lets go of the oldest chunks it no longer shows. leavesPeak gives the clip's peak from them.
import { leavesPeak, waveformLeaves } from "@audio-ui/gl/waveform";
// Once, while the recording is written: whole leaves a stretch at a time, laid end to end.
const leaves = waveformLeaves(mix);
const samples = {
frames,
leaves: () => leaves, // or a promise, read from where they are kept
read: (from, to) => readFrames(from, to), // a chunk at a time, at once or as a promise
};
const peak = leavesPeak(leaves);Each channel of a clip's channels gives its own leaves for its row.
Channels
A recording with several channels is drawn as a DAW draws it: a row for each channel once the rows are tall enough to read, one waveform below that, as a short track shows one lane. layout is "auto" by default and splits when each row would be at least SPLIT_ROW_PX (40) CSS px tall; "split" and "mix" force it. Give a clip its channels, one reader each, for the rows; samples stays their mix, which the one waveform, the band look and the spectrogram draw. The rows share the clip's peak, so channels are compared at the same height, and are told apart by a pixel of ground. splitsChannels and waveformRows give the layout for a lane height, to lay out what goes beside it.
By band
With look: "bands" a clip is drawn as the three bands of its sound one over the other: the lows warm behind, the mids in the ink's colour, the highs cool in front, as DJ software colours a sound. A clip's bands() gives the sound split in three (splitBands of @audio-ui/engine/analysis), with the loudest sample of each band so each is drawn as tall as its own loudest reaches. It is called once a recording and may split lazily: the split holds three copies of the mix. A recording streamed from a file gives each band as a reader instead ({ frames, read, leaves }), whose leaves are made with the others in the same pass (createBandSplit of @audio-ui/engine/analysis) and whose stretches are split as the view reads them; a band given no peak is drawn as tall as its leaves' loudest.
Reading backwards
gl-waveform has no range that reads backwards: a stretch whose frames fall is not drawn. A reversed clip is the host's own drawing, and a page draws it that way too where WebGL 2 is missing.