theinformer

A plugin and standalone software that analyzes the incoming audio and computes a range of audio descriptors, which are sent as OSC messages

View on GitHub

Table of contents

The Informer

A VST3/LV2/AU plugin and standalone software that analyzes incoming audio and computes a range of audio descriptors, which are sent as OSC (Open Sound Control) messages. These can be used in other software, such as for sound visualization or real-time audio analysis. The Informer is compatible with Windows, Linux, and macOS.

The Informer plugin screenshot

Complementing The Informer, there is also The Case Officer, a Max for Live device that receives data from The Informer so that they can be used as modulation sources within Live. Additionally, there is a C++ header-only library available for integrating the descriptor calculation algorithms into your own software.

Currently, these descriptors have been implemented:

Amplitude descriptors

Spectral descriptors

specentropy svg

Please note: when the normalize parameter is enabled, all descriptors are adjusted to ensure they fall within the [0.0, 1.0] range. While some descriptors naturally adhere to this range or have well-defined boundaries (e.g., those typically limited to [0.0, Nyquist frequency]), others (such as kurtosis, skewness, and slope) are adjusted based on empirical observations. As a result, the normalize option is best suited for artistic purposes where exact precision is not essential, focusing instead on preventing values from falling outside the expected range, rather than for detailed sound analysis. In addition to the descriptors mentioned above, a simplified spectrum is provided, consisting of a user-specified number of equal-octave bands (ranging from 2 to 16 via the user interface). For each of these bands, the reported value corresponds to the square root of the highest magnitude among all the original frequency bands of the full spectrogram that fall within that equal-octave band.

Pre-built binaries

Compiled binaries for Linux, Windows and macOS can be found in the Releases section.

How to build the devices

If you prefer to build the devices yourself, first of all Grab the source with git clone https://github.com/valeriorlandini/theinformer.git --recursive.

Then, follow the instructions below.

Plugin/standalone

cd theinformer and then create the necessary build files with:

Navigate to the build folder with cd build

Next run cmake --build . --config Release

The compiled binaries can be found inside TheInformer_artefacts/Release (or simply TheInformer_artefacts in Linux) folder.

Max/MSP externals

cd theinformer/InformerMax and then follow the next instructions according to your operating system. In both cases, once you built the package, copy the folder InformerMax in Documents/Max 9/Packages (or Max 8, according to your Max/MSP version).

Mac

Run cmake -G Xcode ..

Next run cmake --build . --config Release or open the Xcode project from this “build” folder and use the GUI.

Note: you can add the -jX option where X is the number of cores to use (e.g. -j4). This can help speed up your builds, though it may sometimes interleave the error output, making troubleshooting more challenging.

If you are running on a Mac M1+ machine, you will likely see an error cannot be loaded due to system security policy when loading your externals in Max. To resolve this, you can ad-hoc codesign your external with codesign --force --deep -s - myobject.mxo.

Windows

You can run cmake --help to get a list of the options available.

Visual Studio 2022:

cmake -G "Visual Studio 17 2022" ..

Visual Studio 2019:

cmake -G "Visual Studio 16 2019" ..

Visual Studio 2017:

cmake -G "Visual Studio 15 2017 Win64" ..

Having generated the projects, you can now build by opening the .sln file in the build folder with the Visual Studio app (just double-click the .sln file) or you can build on the command line like this:

cmake --build . --config Release

Note: you can add the -jX option where X is the number of cores to use (e.g. -j4). This can help speed up your builds, though it may sometimes interleave the error output, making troubleshooting more challenging.

VCV Rack module

cd theinformer/RackPlugin/TheInformer and then run:

make dist RACK_DIR=[path to Rack dir]

Where [path to Rack dir] is the path (relative or absolute) to the folder where you uncompressed Rack SDK (that you can download here) or Rack source code.

A file with .vcvplugin extension will be created: move this into your Rack modules folder and open VCV Rack.

The Case Officer Max for Live device

The Case Officer is a Max for Live device that receives data from The Informer so that they can be used as modulation sources within Live. It runs on Live 12 (Windows/macOS), and can be found in the Max4Live folder of this repository.

image

Informer C++ library

In Library folder, there is informer.h, a MIT-licensed C++ header-only library to use the plugin algorithms in any application. The library can be used in two ways: by directly calling the provided functions or by creating an instance of the implemented class and then computing and retrieving the descriptors from there.

The library is simply imported with the inclusion of its header:

#include "informer.h"

For the first option, there are two namespaces inside the Informer namespace: Amplitude and Frequency. Once you have an iterable container with floating point values (of any type) representing a buffer, you can compute the amplitude descriptors according to the following example:

std::vector<double> myBuffer = /* your buffer */

auto rms = Informer::Amplitude::rms(myBuffer);

For descriptors like kurtosis and skewness, which uses the mean and the variance, these values can be passed as optional arguments or, if not provided, they are computed by the function. For frequency descriptors, the functions expect an iterable container with floating point values representing the magnitudes of each bin obtained from the Fourier transform, for example:

std::vector<double> fftMag = /* your FFT magnitudes */

auto irregularity = Informer::Frequency::irregularity(fftMag);

There is a built-in function to calculate the normalized magnitudes if you have the result of a real valued FFT in the canonical form of (real[0], real[SR/2], real[1], imag[1], real[2], imag[2], ...):

// With realFftResult being a container with the result of a real valued FFT
std::vector<double> fftMag = Informer::Frequency::magnitudes(realFftResult);

For descriptors needing the sample rate, this can be specified (otherwise it is set to 44100 Hz). When a descriptor uses other descriptors (such as kurtosis and skewness), these can be passed as parameters, otherwise they are computed. Finally, for descriptors expressed in Hertz, like centroid and spread, the frequencies of the FFT bins can be passed as a vector to speed up the function (that would compute them otherwise). A utility function precompute_frequencies is available for this scope:

double sampleRate = 44100.0;
unsigned int fftSize = 8192;

auto precomputed_frequencies = Informer::Frequency::precompute_frequencies(fftSize, sampleRate);

For class implementation, it can be instantiated by passing a buffer and a series of FFT magnitudes:

// Sample rate defaults to 44100.0, rolloff point defaults to 0.85,
// previous FFT magnitudes to 0.0 (with same current FFT magnitudes size)
// Last parameter tells to compute the descriptors immediately
// If buffer or magnitudes are not passed, they default to empty vectors
// and their corresponding descriptors are not computed
// fftSize parameter can be used to specify the FFT size when magnitudes
// are not passed. If fftSize is not passed or set to 0, FFT size is
// (magnitudes size - 1) * 2
Informer::Informer<float> informer(buffer, fftMag, sampleRate, rolloffPoint, previousFftMags, fftSize, true);

// Retrieve descriptors (once they have been computed)
auto peak = informer.get_time_descriptor("peak");
auto centroid = informer.get_frequency_descriptor("centroid"));

// Store new audio values
informer.set_buffer(newBuffer);
informer.set_magnitudes(newMagnitudes);
// If you have the result of a real valued FFT, use this function
// and magnitudes are automatically computed
informer.set_stft(newStft);

// Compute new descriptors
informer.compute_descriptors();

pyinformer: Informer Python bindings

In PyInformer folder, there is the necessary stuff to create Python bindings to the C++ library, so that you can use all the functions of the library in Python.

To build and install the bindings, inside the main repository folder:

pip install ./PyInformer

(Expand this section to build only without automatic installation) To build the bindings, inside `PyInformer` folder: * `cmake -S . -B build -G "Visual Studio 17 2022"` on Windows (adjust the Visual Studio version if you have an older one.) * `cmake -S . -B build -G "Unix Makefiles"` on Linux * `cmake -S . -B build -G Xcode` on Mac Navigate to the build folder with `cd build` Next run `cmake --build . --config Release` You will find a dynamic library file that begins with `pyinformer.cpython`: place inside your Python library folder or inside your Python project folder.

You can begin to use the library right away with:

import pyinformer

There are two submodules, pyinformer.amplitude and pyinformer.frequency. Once you have a list of floats or a NumPy 1D array representing a buffer with its samples or the magnitudes of a real FFT, you can do for example:

import numpy as np

# Generate a random buffer and scale it to [-1, 1] range
example_buffer = np.random.rand(4096) * 2.0 - 1.0

zero_crossing_rate = pyinformer.amplitude.zerocrossing(example_buffer)

Arguments to be passed to the various functions are the same and respect the same order of the corresponding C++ ones.

The Python library has also a class-based implementantion, mirroring the one provided by the C++ counterpart. For example, provided that you have a buffer and its corresponding FFT magnitudes:

import pyinformer 

descriptors = pyinformer.Informer(sample_rate=sr, stft_size=n_fft)

# Buffer and magnitudes can be also set during class initialization
descriptors.set_buffer(buffer)
descriptors.set_magnitudes(magnitudes)

descriptors.compute_descriptors()
centroid = descriptors.get_frequency_descriptor('centroid')

informer.js: Informer JavaScript bindings

In Informer.js folder, there is a WASM implementation of the library, with both the C++ source code to build it with Emscripten and the compiled module and JS file. Documentation coming soon, in the meanwhile you can check a demo here, available also inside demo subfolder of Informer.js.

License

JUCE-based software (VST/AU/LV2 plugins and standalone) and Rack module are licensed under GPLv3, while the rest of the project (libraries, Max for Live device and Max/MSP externals) is MIT licensed.