Audio in C++


This topic describes the "Audio in C++" sample application implementation.

Samples

This tutorial describes how to implement a simple audio playback application that can load and play multiple audio files. The user can load WAV sounds and start, pause, and stop their playback.

Figure 1. Audio in C++ application

For information on how to access the sample application cheat sheet and run the application, see Sample-based Tutorials.

In the sample application source code, pay attention to the following member variables in the AudioInstance class:

  • audio_ is a pp::Audio object.
  • header_ is a pointer to a WAV file header.
  • file_data_bytes_ is the WAV file data.
  • sound_instances_ is a vector of sound instances read from the WAV file.
  • active_sounds_ is an array of the active sound instances.
  • file_names_ is a vector of WAV file names.

Initializing the Instance and Context

To implement audio playback, you must initialize the instance and the audio context:

  1. In the class constructor, initialize the required members, callback factory, and the Logger class:

    AudioInstance::AudioInstance(PP_Instance instance) :
  pp::Instance(instance),
  file_number_(0),
  active_sounds_number_(0),
  callback_factory_(this) {
  file_names_.reserve(kNumberOfInputSounds);
  Logger::InitializeInstance(this);
}

  2. Initialize the audio context.

    1. Define the audio buffer size by retrieving the recommended sample frame count for the given audio parameters, using the RecommendSampleFrameCount() function.
    2. Initialize the audio playback configuration object with the retrieved sample frame count.
    3. Initialize the pp::Audio instance with the AudioConfig object and the AudioCallback() function pointer. The AudioCallback() function implements the main audio playback logic: it is called repeatedly during playback, and mixes the audio data from multiple files.
    bool AudioInstance::Init(uint32_t /*argc*/, const char* /*argn*/[],
  const char* /*argv*/[]) {
  // Retrieve the recommended sample frame count
  sample_frame_count_ =
    pp::AudioConfig::RecommendSampleFrameCount(this,
      PP_AUDIOSAMPLERATE_44100,
      SAMPLE_FRAME_COUNT);
  // Create an audio resource configuration with a 44.1 kHz sample rate
  // and the recommended sample frame count
  pp::AudioConfig audio_config = pp::AudioConfig(this,
    PP_AUDIOSAMPLERATE_44100,
    sample_frame_count_);
  audio_ = pp::Audio(this,  // Pointer to pp::Instance
    audio_config,
    AudioCallback,
    this); // argument of type void* for the AudioCallback call
  return true;
}

Implementing Playback Controls

When a playback control button is clicked, the JavaScript application component uses the PostMessage() method to notify the Native Client (NaCl) plugin.

In the NaCl plugin, parse the received message using the HandleMessage() function and pass the data to the appropriate playback control function:

void AudioInstance::HandleMessage(const pp::Var& var_message) {
  if (var_message.is_string()) {
    const std::string message = var_message.AsString();
    if (StartsWith(message, kPlayCommand)) {
      // Start playback
      Play(GetIdFromMessage(message, kPlayCommand));
    } else if (StartsWith(message, kPauseCommand)) {
      // Pause playback
      Pause(GetIdFromMessage(message, kPauseCommand));
    } else if (StartsWith(message, kStopCommand)) {
      // Stop playback
      Stop(GetIdFromMessage(message, kStopCommand));
    } else if (StartsWith(message, kLoadCommand)) {
      // Receive the file URL and load it
      PrepareReadingFile(message.substr(strlen(kLoadCommand)));
    }
  }
}

To implement the playback controls:

  • To load and read an audio file:

    1. Load the WAV file to a data buffer using the PrepareReadingFile() function.
    2. In the ReadWAVE() function, interpret the WAV data:
      1. Copy the input string to a char vector, and cast the pointer for the data vector to the WAVEFileHeader structure pointer. This structure checks whether the header is correctly formatted.
      2. If the header check succeeds, cast the address for the beginning of the rest of the data array to a smart pointer.
      3. Create a SoundInstance object to contain the audio data.
      4. Push the SoundInstance object to the sound_instances_ collection, from where it can be retrieved for playback.
    void AudioInstance::ReadWAVE(const std::string& data) {
  Logger::Log("Interpreting WAVE data of file %s",
    file_names_[file_number_].c_str());

  // Clean and create a buffer for interpreting file data
  file_data_bytes_.reset(new char[data.size() + 1]);

  // Copy file data to the buffer
  std::copy(data.begin(), data.end(), file_data_bytes_.get());
  file_data_bytes_[data.size()] = '\0';

  // Read the header data
  header_ = reinterpret_cast<WAVEFileHeader*>(file_data_bytes_.get());

  // Check for malformed header
  std::string header_error = CheckForWAVEHeaderErrors(*header_);
  if (!header_error.empty()) {
    Logger::Error("Unsupported file %s - bad header:\n%s",
      file_names_[file_number_].c_str(), header_error.c_str());

    PostMessage(CreateCommandMessage(kErrorMessage, file_number_));
    return;
  }
  std::unique_ptr<uint16_t> sample_data(reinterpret_cast<uint16_t*>(
    file_data_bytes_.get() + sizeof(WAVEFileHeader)));

  // Create a sound instance, fill its fields, and add it
  // to the sound instances vector
  std::shared_ptr<SoundInstance> instance(new SoundInstance(file_number_,
    header_->num_channels, (data.size() - sizeof(WAVEFileHeader)) /
    sizeof(uint16_t), &sample_data));
  sound_instances_.insert(std::pair<int, std::shared_ptr<SoundInstance> >(
    instance->id_, instance));

  // Notify the JavaScript component
  PostMessage(CreateCommandMessage(kFileLoadedMessage, file_number_));

  Logger::Log("File %s data loaded",
    file_names_[file_number_].c_str());
}

  • To start playback:

    1. Check whether any sounds are currently active and playing.
    2. If no sounds are active, start playback by using the StartPlayback() function to initiate the audio callback loop.
    3. Flag the sound as active and increment the active sounds counter.
    4. Notify the JavaScript component that playback has started.
    void AudioInstance::Play(int number) {
  // Start playing if nothing is active
  if (active_sounds_number_ == 0) {
    audio_.StartPlayback();
    Logger::Log("Playing started");
  }
  assert((number - 1 >= 0) || (number - 1 < kNumberOfInputSounds));
  active_sounds_[number - 1] = true;
  ++active_sounds_number_;

  // Notify the JavaScript component
  PostMessage(CreateCommandMessage(kPlayMessage, number));
}

  • To pause playback:

    1. Flag the sound as inactive and decrement the active sounds counter.
    2. If there are no active sounds, stop the audio callback loop using the StopPlayback() function.
    3. Notify the JavaScript component that playback has paused.
    void AudioInstance::Pause(int number) {
  // Set the sound as inactive
  --active_sounds_number_;
  assert((number - 1 >= 0) || (number - 1 < kNumberOfInputSounds));
  active_sounds_[number - 1] = false;

  // Stop playing if nothing is active
  if (active_sounds_number_ == 0) {
    audio_.StopPlayback();
    Logger::Log("Playing stopped");
  }
  // Notify the JavaScript component
  PostMessage(CreateCommandMessage(kPauseMessage, number));
}

  • To stop playback:

    1. Flag the sound as inactive and decrement the active sounds counter.
    2. If there are no active sounds, stop the audio callback loop using the StopPlayback() function.
    3. Reset the sample_data_offset_ variable, so the file plays from the beginning the next time playback starts.
    4. Notify the JavaScript component that playback has stopped.
    void AudioInstance::Stop(int number) {
  // Set the sound as inactive
  assert((number - 1 >= 0) || (number - 1 < kNumberOfInputSounds));
  if (active_sounds_[number - 1]) {
    --active_sounds_number_;
    active_sounds_[number - 1] = false;
  }

  // Stop playing if nothing is active
  if (active_sounds_number_ == 0) {
    audio_.StopPlayback();
    Logger::Log("Playing stopped");
  }

  // Reset the playback offset
  sound_instances_[number]->sample_data_offset_ = 0;

  // Notify the JavaScript component
  PostMessage(CreateCommandMessage(kStopMessage, number));
}

Implementing the Audio Callback Loop

The audio callback loop fills the audio buffer with sample data and mixes data from multiple sources in real time. During playback, the AudioCallback() function is called regularly. Each iteration fills the buffer with the next chunk of data.

  1. Cast the pointer to the AudioInstance object and buffer, and fill the buffer with zeroes.
  2. To maintain the same overall volume level regardless of the number of sounds playing simultaneously, calculate a volume modifier. The modifer is defined as the inverse square root of the active sound count.
  3. Fill the buffer with data. For each SoundInstance object in the sound_instances_ map:
    1. Check whether the end of the sound data has been reached.
    2. Write the data chunk located at the SoundInstance object data offset to the buffer using the SafeAdd() function. This constrains the buffer values to between a minimum and maximum value for each sample.
    3. Increment the data offset in the SoundInstance object. The data offset allows the buffer to resume filling from the last position the next time the AudioCallback() function is called.
  4. If the end of any SoundInstance data is reached, notify the JavaScript component.
void AudioInstance::AudioCallback(void* samples, uint32_t buffer_size,
  void* data) {

  // Instance pointer is passed when creating Audio resource
  AudioInstance* audio_instance = reinterpret_cast<AudioInstance*>(data);

  // Fill the audio buffer
  int16_t* buff = reinterpret_cast<int16_t*>(samples);
  memset(buff, 0, buffer_size);

  assert(audio_instance->active_sounds_number_ > 0);
  // Compute the volume of the sound instances
  const double volume = 1.0 / sqrt(audio_instance->active_sounds_number_);
  // Prevent writing outside the buffer
  assert(buffer_size >= (sizeof(*buff) * MAX_CHANNELS_NUMBER *
    audio_instance->sample_frame_count_));

  for (SoundInstances::iterator it =
    audio_instance->sound_instances_.begin(); it !=
      audio_instance->sound_instances_.end(); ++it) {
    std::shared_ptr<SoundInstance> instance(it->second);

    // Main loop for writing samples to audio buffer
    for (size_t i = 0;  i < audio_instance->sample_frame_count_; ++i) {
      // If there are samples to play
      if (audio_instance->active_sounds_[instance->id_ - 1] &&
        instance->sample_data_offset_ < instance->sample_data_size_) {
        audio_instance->SafeAdd(&buff[2 * i], volume *
          (int16_t)instance->sample_data_[instance->sample_data_offset_]);
        // For mono sound, each sample is passed to both channels, but for
        // stereo sound, samples are written successively to all channels
        if (instance->channels_ == 2) {
          ++instance->sample_data_offset_;
        }
        audio_instance->SafeAdd(&buff[2 * i + 1],  volume *
          (int16_t)instance->sample_data_[instance->sample_data_offset_]);
        ++instance->sample_data_offset_;
      }
    }

    // When the end of a sound is reached, stop playing it
    if (instance->sample_data_offset_ == instance->sample_data_size_) {
      // Pepper API calls are normally avoided in audio callbacks,
      // as they can cause audio dropouts when the audio callback thread 
      // is swapped out
      // Audio dropout is not a concern here because the audio has ended
      audio_instance->PostMessage(audio_instance->CreateCommandMessage(
        kReturnStopMessage, instance->id_));
    }
  }
}