Use Cases and Testing

Synopsis

A major challenge when adopting an API as flexible and complex as ARA is understanding the potential use cases that will be encountered across all the possible host/plug-in combinations, and to design and test implementations accordingly.
This chapter describes various use cases that each host and plug-in should be aware of in order to build high quality ARA integrations. Intense quality insurance is key to a stable and thriving overall ARA platform and great user experience - passing this checklist as applicable is therefore also a prerequisite for Celemony granting developers the right to use its ARA Audio Random Access trademark for marketing a ARA-enabled products as such.
As a living document, this chapter will be regularly updated whenever common issues in current implementations become apparent.

As both hosts and plug-ins have very different features and capabilities, many of the following use cases may not apply in a specific host/plug-in combination. For example, a restoration plug-in will typically not bother with musical timing, time-stretching or MIDI notes. Nevertheless, all features which make sense in the given context shall be properly supported by all hosts and plug-ins that pick up ARA.

While testing, developers should also utilize the ARA Test Host and ARA Test Plug-In examples provided with the ARA SDK as well as Melodyne, which all feature extensive testing and validation as outlined in the previous chapter.

Render Timing

  • Is sample timing correct in all cases? (It's easy to encounter an occasional single-sample offset due to rounding errors when feeding the plug-in's sample time)
  • Is the latency of post-ARA realtime effects and audio hardware properly compensated for?
    • Is the playback still sample accurate across different tracks with different latencies?
    • For meters or playback location displays, is visual latency compensation working? (see VST3's Steinberg::Vst::IAudioPresentationLatency and AudioUnit's kAudioUnitProperty_PresentationLatency)
  • If implemented in the host, do user-accessible track delay and/or region delay parameters still work properly when using ARA?

Musical Timing

  • If both host and plug-in use musical song timing (tempo and signatures), is it properly exchanged? This can be tested both with Melodyne or the experimental JUCE_ARA plug-in demo.
    • Is the beat timeline drawn in the plug-in identical to the one drawn in the host?
    • Does the plug-in follow edits in the host timeline properly?
    • Does the grid align with the metronome/click track?
    • Can your implementation handle multiple tempo changes and multiple signature changes?
    • Is the musical timing always in sync when the latency compensation is active (both directions)?
  • If both host and plug-in use musical timing for audio sources, is it properly synchronized in both directions?
    • If the host knows the file tempo and signature before the plug-in is added, does the plug-in read that information properly?
    • If not known before inserting, and the plug-in can analyze this, is the analysis result picked up by the host?
    • Does the plug-in update properly if the audio source tempo or signatures are edited in the host, and vice versa?
  • If supported by the host, can audio source tempo and signature analysis provided by the plug-in copied to the song timeline? For example, Studio One implements this by letting users drag events (i.e. playback regions) to the timeline.

Time Stretching

  • Does time-stretching work properly? Using an example audio file with a drum track that starts at 120 bpm then has a ritardando to 90 bpm:
    • can the user have this drum file analyzed in the plug-in and make it fit a song with constant 120 bpm? (As a host, have the audio analyzed by the plug-in, then read the timeline to find out the number of beats in the file, then in the host timeline pick a range with the same number of beats, then set up an according playback region and activate the TimestretchReflectingTempo flag
    • Starting from the results of the previous test, can the user stretch the file to halftime by either entering a stretch factor in the host or by dragging (if supported by the host)? Does the result sound correct in both cases?
    • Starting from the results of the first test, can the user use Melodyne's tempo dialog to re-interpret the tempo as halftime (so that the region/event is extended properly)?
    • Can the user do the opposite of the first test and make the song timeline fit the drum file? (As a host, have the audio analyzed by the plug-in, then read the timeline and copy the tempo curve to the song timeline, then set up a playback region that does not modify the file upon playback)

Signal Flow And Routing

  • Are crossfades set in the host rendered correctly?
  • Are content based fades rendered correctly (if implemented by the plug-in and supported by the host)?
  • Does region/event solo/mute/volume behave as expected?
  • Does track solo/mute/volume behave as expected?
  • Is bypass behaving properly for ARA? (If the host chooses to bypass ARA plug-ins, it will need to apply any transformation done by the plug-in on its own - this may be tough esp. with regards to the the TimestretchReflectingTempo flag. The easier choice is to not implement bypass for ARA on the host side and instead leave that up to the plug-in.)

Maintaining The ARA Model

  • Does undo/redo in the host work as expected if ARA-fied regions/events are involved? (Note: it may be necessary to keep some ARA objects around for Undo even if not used elsewhere, see deactivateAudioSource/ModificationForUndoHistory)
  • If the host has a concept of alias versus copy of regions/events, do edits in the plug-in reflect accordingly across alias regions/events while not affecting copied regions/events?
  • If the host has a concept of track versions/alternatives and supports multiple audio modifications per audio source, does it correctly create individual modifications per track version/alternative?

Audio-MIDI Conversion

  • If implemented on both sides, does audio-to-MIDI work properly?
    • Is the timing of the MIDI 100% in sync with the audio?
    • Are the MIDI pitches correct? (It's easy to be off by an octave here because there are different interpretations between Yamaha and Roland of what pitch a "C1" has)

Key Signatures And Chords

  • Are enharmonic equivalents properly distinguished based on the circle of fifth index?
  • Are the interval roles properly evaluated to distinguish e.g. from 2 from 9?
  • If the name is provided or evaluated, are the music related unicode characters (sharp, flat, triangle etc.) properly handled?
  • When reading chords, and the internal chord model is covering only a subset of the possible ARA chords, are there reasonable fallbacks for the cases not covered?

Persistence

  • Can documents be archived and restored properly with all edits in tact (see also ARA Test Host testArchiving())?
    • When doing a bounce, is the output identical before saving and after restoring?
  • When restoring, does your host/plug-in fail gracefully if media files are missing (audio sources samples not available)?
    • Does the document recover properly if media files are re-assigned after they were missing upon restoring?
  • Are archiving progress notifications being sent / received properly?
  • Is drag/drop and copy/paste between documents properly including the respective ARA object state (see also ARA Test Host testDragAndDrop())?
  • Are ARA audio file chunks supported (see also ARA Test Host testAudioFileChunkLoading())?
    • Audio files with chunks can be created by Melodyne 5 Standalone by storing detection data into the audio file. Is that detection data loaded properly when using Melodyne as ARA plug-in for such a file?
    • SpectraLayers 7 also creates ARA-enhanced files on demand when dragging layers out of the plug-in. These chunks request that the plug-in is opened automatically when the file is added, does this work?

Versioning

  • On the host side, are the supported API generations evaluated properly?
    • Is there a proper error message for the user if a plug-in is either too old or too new?
  • On the host side, are the versioned plug-in archive IDs properly stored and evaluated?
    • If a document is loaded which contains an incompatible plug-in chunk from a newer version of the plug-in, is there a proper error message containing the appropriate information about the newer plug-in? Is loading the incompatible chunk properly suppressed? This can be tested e.g. with Melodyne 5 vs. 4.

UI-Related Topics

  • Does rendering in stop mode work if the UI is open?
    • Do editing operations in the plug-in (e.g. dragging a blob up and down in Melodyne) provide audible editor rendering results in stop mode?
    • Does the playback cursor in the plug-in follow the playback cursor in your host in stop mode?
  • Does host playback control from the plug-in work work?
    • Are the cycle ranges properly synchronized between the plug-in and the host, with each side reflecting the changes on the other?
  • Is key command handling working properly?
    • Are all key commands properly handled by the plug-in when its view has the focus (regardless of whether they are bound to menu items in the plug-in's menus or "free" commands not bound to a menu)?
    • Are they properly ignored by the plug-in when has no focus?
    • Is there any conflicts when both the host and the plug-in implement the same key command, such as the standard key commands for copy/undo/paste or space bar for playback start?
    • Does it work across Windows, macOS, Linux (if applicable)?
  • In the given host/plug-in scenario, does it make sense to optionally integrate the plug-in view into the main window like Studio One, Cubase and other hosts do with Melodyne?
  • Does embedding the plug-in view into the host work fine in all places where it is implemented (separate window or embedded into a single-window-approach)?
    • Does resizing the view behave correctly?
    • Is repeatedly opening/closing views or switching between views of different plug-in instances working fine?
    • Does it work across Windows, macOS, Linux (if applicable)?
    • Is HiDPI supported properly?

General

  • Does the implementation scale well both in terms of CPU and memory usage when being used with many long audio files at the same time (e.g. a real-life test scenario with a 30 minutes life-recording of 20+ tracks)?
  • When reading UTF8-encoded strings from the other side, are any normalization requirements (e.g. for the text layout engine) properly enforced?
  • Does the plug-in scan as an ARA enabled plug-in in the host?
    • For AudioUnit plug-in developers, is the `ARA"` tag" set in the Info.plist file?