30
Apr 12
16:20
iOS: encoding to AAC with the Extended Audio File Services gotchas
I recently implemented encoding to AAC from raw PCM in memory (as opposed to from a file) using the extended audio file services, which have their calls prefixed with ExtAudioFile*. ExtAudioFile stuff basically wraps a converter with the standard AudioFile functionality. It was a bit hard to track down the correct documentation, but it’s not that far off from using the regular AudioFile stuff you’d do when writing a WAV file, for instance.
Then, a few days later, AAC encoding suddenly stopped working, without my code changing at all. Debugging led me to find that this only happened on my iPhone 4S, and not my iPod or 3GS, nor the simulator. This problem existed with apps from other developers too, including the helpful Michael Tyson’s TPAAC converter project on github. I was getting a long amount of blocking (10-30 seconds) and then an error from sometimes ExtAudioFileSetProperty on the kExtAudioFileProperty_ClientDataFormat and if not that, the ExtAudioFileWrite call. I have no clue why sometimes the ExtAudioFileSetProperty would go through. In all cases though, the audio on my app was dead after this call failed until I restarted the app. This made me think it was something to do with AudioSessions (as Michael Tyson points out can cause issues), but turns out this wasn’t the case for me. FYI my OSError code was 268451843, which doesn’t even translate to anything in core audio.
After reverting to swearing at the docs that are the notorious API documentation, and exploring various dead ends such as modifying my interrupt handler for the AudioService as well as trying it on the main thread just in case there was some race case issue, I came across this life saving stack overflow post by a mystery user1021430 who I am feeling very fond for (since he has an ambiguous handle I can imagine what I want). His post mentioned that it seems for certain dual core devices like the 4S, the hardware encoder is finicky, and it is possible to switch to a software encoder to fix these issues. I had no clue about this from the docs, and really think this guy that posted the answer probably has the most undervalued reputation (1).
Anyway, here’s my finalized (and unpolished, including progressbar updates that you should remove to use) code that exports to aac from a buffer. The FilterSound and FilterAudioBuffer classes are ommited, but it basically just helps fill out a buffer and shouldn’t be hard to understand.
-(BOOL)saveAACFileWithSound:(FilterSound*)s filename:(NSString*)filename progressView:(UIProgressView*)progView
{
OSStatus err = noErr;
NSString *path = [self pathForAudioFileWithFilename:filename];
NSURL *url = [NSURL fileURLWithPath:path];
char errCode[5];
uint64_t doneSamples = 0;
uint64_t totalSamples = [s approxTotalSamples];
AudioStreamBasicDescription format;
ExtAudioFileRef outFile;
memset(&format, 0, sizeof(format));
format.mFormatID = kAudioFormatMPEG4AAC;
//format.mFormatFlags = kMPEG4Object_AAC_Main;
//format.mSampleRate = kFilterSampleRate; // the encoder for compressed formats uses a different sample rate (see docs)
format.mChannelsPerFrame = 2;
err = ExtAudioFileCreateWithURL((CFURLRef)url,
kAudioFileM4AType,
&format,
NULL,
kAudioFileFlags_EraseFile,
&outFile);
if (err != noErr) {
strncpy(errCode,(const char*) &err, 4);
errCode[4] = 0;
NSLog(@"AudioFileCreateWithURL path=%@ Error %li = %s", path, err, errCode);
return NO;
}
// sometimes the dual core devices like iPhone 4S will freak out on the next set property call unless you do this.
// What's more the device will be broken.
// No idea why this is.
UInt32 codecManf = kAppleSoftwareAudioCodecManufacturer;
ExtAudioFileSetProperty(outFile, kExtAudioFileProperty_CodecManufacturer, sizeof(UInt32), &codecManf);
// setup the client format since we want to convert from PCM
AudioStreamBasicDescription clientFormat;
memset(&clientFormat, 0, sizeof(format));
clientFormat.mFormatID = kAudioFormatLinearPCM;
// WAV should be little endian
clientFormat.mFormatFlags = kLinearPCMFormatFlagIsSignedInteger |
kLinearPCMFormatFlagIsPacked;//~kAudioFormatFlagIsBigEndian;
clientFormat.mSampleRate = kFilterSampleRate;
clientFormat.mBitsPerChannel = sizeof(AudioSampleType) * 8; // AudioSampleType == 16 bit signed ints
clientFormat.mChannelsPerFrame = 2;
clientFormat.mFramesPerPacket = 1;
clientFormat.mBytesPerFrame = (clientFormat.mBitsPerChannel / 8) * clientFormat.mChannelsPerFrame;
clientFormat.mBytesPerPacket = clientFormat.mBytesPerFrame * clientFormat.mFramesPerPacket;
err = ExtAudioFileSetProperty(outFile, kExtAudioFileProperty_ClientDataFormat, sizeof(clientFormat), &clientFormat);
if (err != noErr){
NSLog(@"ExtAudioFileSetProperty error %li", err);
return NO;
}
size_t outNumPackets = 0;
size_t numBytes = 0;
FilterAudioBuffer* audio_buf;
audio_buf = [[FilterAudioBuffer alloc] initWithChannels:2
frames:kSaveWavBufferSamples
shouldZero:NO];
AudioBufferList writeBuffer;
while (![s shouldBeRemoved]) {
// we use the same buffer each iteration, so we need to wipe it clean.
memset(audio_buf->buffer, 0, sizeof(SInt16) * 2 * kSaveWavBufferSamples);
// have the sound object fill out the next sound
// for this function the term 'packet' is a frame for uncompressed formats.
// fortunately writeToBuffer returns the number of frames written
// (This can be shorter for buffer playback at the end when the sound size doesnt
// divide evenly into kSaveWavBufferSamples
outNumPackets = [s writeToBuffer:(SInt16*)audio_buf->buffer samples:kSaveWavBufferSamples];
numBytes = outNumPackets * sizeof(SInt16) * 2;
writeBuffer.mNumberBuffers = 1;
writeBuffer.mBuffers[0].mNumberChannels = clientFormat.mChannelsPerFrame;
writeBuffer.mBuffers[0].mDataByteSize = numBytes;
writeBuffer.mBuffers[0].mData = audio_buf->buffer;
err = ExtAudioFileWrite(outFile, outNumPackets, &writeBuffer);
doneSamples += outNumPackets;
float prog;
prog = doneSamples / ((float) totalSamples);
[self performSelectorOnMainThread:@selector(updateProgress:)
withObject:[NSArray arrayWithObjects:[NSNumber numberWithFloat:prog],
progView, nil]
waitUntilDone:NO];
if (err != noErr){
NSLog(@"AudioFileWritePackets error %li", err);
break;
}
}
[audio_buf release];
ExtAudioFileDispose(outFile);
return err == noErr;
}
