Audio in C++
This topic describes the "Audio in C++" sample application implementation.
Samples
This tutorial describes how to implement a simple audio playback application that can load and play multiple audio files. The user can load WAV sounds and start, pause, and stop their playback.
For information on how to access the sample application cheat sheet and run the application, see Sample-based Tutorials.
In the sample application source code, pay attention to the following member variables in the AudioInstance
class:
audio_
is app::Audio
object.header_
is a pointer to a WAV file header.file_data_bytes_
is the WAV file data.sound_instances_
is a vector of sound instances read from the WAV file.active_sounds_
is an array of the active sound instances.file_names_
is a vector of WAV file names.
Initializing the Instance and Context
To implement audio playback, you must initialize the instance and the audio context:
-
In the class constructor, initialize the required members, callback factory, and the
Logger
class:AudioInstance::AudioInstance(PP_Instance instance) : pp::Instance(instance), file_number_(0), active_sounds_number_(0), callback_factory_(this) { file_names_.reserve(kNumberOfInputSounds); Logger::InitializeInstance(this); }
-
Initialize the audio context.
- Define the audio buffer size by retrieving the recommended sample frame count for the given audio parameters, using the
RecommendSampleFrameCount()
function. - Initialize the audio playback configuration object with the retrieved sample frame count.
- Initialize the
pp::Audio
instance with theAudioConfig
object and theAudioCallback()
function pointer. TheAudioCallback()
function implements the main audio playback logic: it is called repeatedly during playback, and mixes the audio data from multiple files.
bool AudioInstance::Init(uint32_t /*argc*/, const char* /*argn*/[], const char* /*argv*/[]) { // Retrieve the recommended sample frame count sample_frame_count_ = pp::AudioConfig::RecommendSampleFrameCount(this, PP_AUDIOSAMPLERATE_44100, SAMPLE_FRAME_COUNT); // Create an audio resource configuration with a 44.1 kHz sample rate // and the recommended sample frame count pp::AudioConfig audio_config = pp::AudioConfig(this, PP_AUDIOSAMPLERATE_44100, sample_frame_count_); audio_ = pp::Audio(this, // Pointer to pp::Instance audio_config, AudioCallback, this); // argument of type void* for the AudioCallback call return true; }
- Define the audio buffer size by retrieving the recommended sample frame count for the given audio parameters, using the
Implementing Playback Controls
When a playback control button is clicked, the JavaScript application component uses the PostMessage()
method to notify the Native Client (NaCl) plugin.
In the NaCl plugin, parse the received message using the HandleMessage()
function and pass the data to the appropriate playback control function:
void AudioInstance::HandleMessage(const pp::Var& var_message) {
if (var_message.is_string()) {
const std::string message = var_message.AsString();
if (StartsWith(message, kPlayCommand)) {
// Start playback
Play(GetIdFromMessage(message, kPlayCommand));
} else if (StartsWith(message, kPauseCommand)) {
// Pause playback
Pause(GetIdFromMessage(message, kPauseCommand));
} else if (StartsWith(message, kStopCommand)) {
// Stop playback
Stop(GetIdFromMessage(message, kStopCommand));
} else if (StartsWith(message, kLoadCommand)) {
// Receive the file URL and load it
PrepareReadingFile(message.substr(strlen(kLoadCommand)));
}
}
}
To implement the playback controls:
-
To load and read an audio file:
- Load the WAV file to a data buffer using the
PrepareReadingFile()
function. - In the
ReadWAVE()
function, interpret the WAV data:- Copy the input string to a char vector, and cast the pointer for the data vector to the
WAVEFileHeader
structure pointer. This structure checks whether the header is correctly formatted. - If the header check succeeds, cast the address for the beginning of the rest of the data array to a smart pointer.
- Create a
SoundInstance
object to contain the audio data. - Push the
SoundInstance
object to thesound_instances_
collection, from where it can be retrieved for playback.
- Copy the input string to a char vector, and cast the pointer for the data vector to the
void AudioInstance::ReadWAVE(const std::string& data) { Logger::Log("Interpreting WAVE data of file %s", file_names_[file_number_].c_str()); // Clean and create a buffer for interpreting file data file_data_bytes_.reset(new char[data.size() + 1]); // Copy file data to the buffer std::copy(data.begin(), data.end(), file_data_bytes_.get()); file_data_bytes_[data.size()] = '\0'; // Read the header data header_ = reinterpret_cast<WAVEFileHeader*>(file_data_bytes_.get()); // Check for malformed header std::string header_error = CheckForWAVEHeaderErrors(*header_); if (!header_error.empty()) { Logger::Error("Unsupported file %s - bad header:\n%s", file_names_[file_number_].c_str(), header_error.c_str()); PostMessage(CreateCommandMessage(kErrorMessage, file_number_)); return; } std::unique_ptr<uint16_t> sample_data(reinterpret_cast<uint16_t*>( file_data_bytes_.get() + sizeof(WAVEFileHeader))); // Create a sound instance, fill its fields, and add it // to the sound instances vector std::shared_ptr<SoundInstance> instance(new SoundInstance(file_number_, header_->num_channels, (data.size() - sizeof(WAVEFileHeader)) / sizeof(uint16_t), &sample_data)); sound_instances_.insert(std::pair<int, std::shared_ptr<SoundInstance> >( instance->id_, instance)); // Notify the JavaScript component PostMessage(CreateCommandMessage(kFileLoadedMessage, file_number_)); Logger::Log("File %s data loaded", file_names_[file_number_].c_str()); }
- Load the WAV file to a data buffer using the
-
To start playback:
- Check whether any sounds are currently active and playing.
- If no sounds are active, start playback by using the
StartPlayback()
function to initiate the audio callback loop. - Flag the sound as active and increment the active sounds counter.
- Notify the JavaScript component that playback has started.
void AudioInstance::Play(int number) { // Start playing if nothing is active if (active_sounds_number_ == 0) { audio_.StartPlayback(); Logger::Log("Playing started"); } assert((number - 1 >= 0) || (number - 1 < kNumberOfInputSounds)); active_sounds_[number - 1] = true; ++active_sounds_number_; // Notify the JavaScript component PostMessage(CreateCommandMessage(kPlayMessage, number)); }
-
To pause playback:
- Flag the sound as inactive and decrement the active sounds counter.
- If there are no active sounds, stop the audio callback loop using the
StopPlayback()
function. - Notify the JavaScript component that playback has paused.
void AudioInstance::Pause(int number) { // Set the sound as inactive --active_sounds_number_; assert((number - 1 >= 0) || (number - 1 < kNumberOfInputSounds)); active_sounds_[number - 1] = false; // Stop playing if nothing is active if (active_sounds_number_ == 0) { audio_.StopPlayback(); Logger::Log("Playing stopped"); } // Notify the JavaScript component PostMessage(CreateCommandMessage(kPauseMessage, number)); }
-
To stop playback:
- Flag the sound as inactive and decrement the active sounds counter.
- If there are no active sounds, stop the audio callback loop using the
StopPlayback()
function. - Reset the
sample_data_offset_
variable, so the file plays from the beginning the next time playback starts. - Notify the JavaScript component that playback has stopped.
void AudioInstance::Stop(int number) { // Set the sound as inactive assert((number - 1 >= 0) || (number - 1 < kNumberOfInputSounds)); if (active_sounds_[number - 1]) { --active_sounds_number_; active_sounds_[number - 1] = false; } // Stop playing if nothing is active if (active_sounds_number_ == 0) { audio_.StopPlayback(); Logger::Log("Playing stopped"); } // Reset the playback offset sound_instances_[number]->sample_data_offset_ = 0; // Notify the JavaScript component PostMessage(CreateCommandMessage(kStopMessage, number)); }
Implementing the Audio Callback Loop
The audio callback loop fills the audio buffer with sample data and mixes data from multiple sources in real time. During playback, the AudioCallback()
function is called regularly. Each iteration fills the buffer with the next chunk of data.
- Cast the pointer to the
AudioInstance
object and buffer, and fill the buffer with zeroes. - To maintain the same overall volume level regardless of the number of sounds playing simultaneously, calculate a volume modifier. The modifer is defined as the inverse square root of the active sound count.
- Fill the buffer with data. For each
SoundInstance
object in thesound_instances_
map:- Check whether the end of the sound data has been reached.
- Write the data chunk located at the
SoundInstance
object data offset to the buffer using theSafeAdd()
function. This constrains the buffer values to between a minimum and maximum value for each sample. - Increment the data offset in the
SoundInstance
object. The data offset allows the buffer to resume filling from the last position the next time theAudioCallback()
function is called.
- If the end of any
SoundInstance
data is reached, notify the JavaScript component.
void AudioInstance::AudioCallback(void* samples, uint32_t buffer_size,
void* data) {
// Instance pointer is passed when creating Audio resource
AudioInstance* audio_instance = reinterpret_cast<AudioInstance*>(data);
// Fill the audio buffer
int16_t* buff = reinterpret_cast<int16_t*>(samples);
memset(buff, 0, buffer_size);
assert(audio_instance->active_sounds_number_ > 0);
// Compute the volume of the sound instances
const double volume = 1.0 / sqrt(audio_instance->active_sounds_number_);
// Prevent writing outside the buffer
assert(buffer_size >= (sizeof(*buff) * MAX_CHANNELS_NUMBER *
audio_instance->sample_frame_count_));
for (SoundInstances::iterator it =
audio_instance->sound_instances_.begin(); it !=
audio_instance->sound_instances_.end(); ++it) {
std::shared_ptr<SoundInstance> instance(it->second);
// Main loop for writing samples to audio buffer
for (size_t i = 0; i < audio_instance->sample_frame_count_; ++i) {
// If there are samples to play
if (audio_instance->active_sounds_[instance->id_ - 1] &&
instance->sample_data_offset_ < instance->sample_data_size_) {
audio_instance->SafeAdd(&buff[2 * i], volume *
(int16_t)instance->sample_data_[instance->sample_data_offset_]);
// For mono sound, each sample is passed to both channels, but for
// stereo sound, samples are written successively to all channels
if (instance->channels_ == 2) {
++instance->sample_data_offset_;
}
audio_instance->SafeAdd(&buff[2 * i + 1], volume *
(int16_t)instance->sample_data_[instance->sample_data_offset_]);
++instance->sample_data_offset_;
}
}
// When the end of a sound is reached, stop playing it
if (instance->sample_data_offset_ == instance->sample_data_size_) {
// Pepper API calls are normally avoided in audio callbacks,
// as they can cause audio dropouts when the audio callback thread
// is swapped out
// Audio dropout is not a concern here because the audio has ended
audio_instance->PostMessage(audio_instance->CreateCommandMessage(
kReturnStopMessage, instance->id_));
}
}
}