This is a basic outline that I will expand on. I just finished debugging this stuff and want to write the main points before I forget my references and main issues.
Summary: two basic gotchas of full duplex (record from mic and output to speaker) audio IO on the iphone that I ran into are:
- The AudioUnits IO callback requires you to handle input buffers and output buffers in a very different fashion for each, whereas in the easy to setup core audio callback I used on OS X, they were handled in a similar fashion (both buffers as parameters to the callback). As they are buffers, mishandling them as I did caused my program to crash. Solution: use the parameter buffer for output, and AudioUnitRender() to obtain the input.
- Sometimes my callbacks were working, verified by my printf debug statements, but no audio output could be heard – the speakers were silent despite a perfectly good-looking setup and debugging session. Cause: DC output or some other condition caused the iphone speakers to go into a muted state even when volume was on. Solution: plug and unplug a headphone.
- When recording from the mic the audio will default to the headset speaker by the earpiece instead of the large and much louder main speaker by the dock. Cause: this is an apple default. Solution: Use AudioSession API to change the default.
Read on for the details.
I’ve implemented Audio IO using the native API on a few systems including old mac (pre OS X), new mac (Core Audio), and Windows. But since then I’ve been getting lazy and just using portaudio for everything, since it does what I want and gets a reasonable latency in most cases.
I just finished implementing my audio IO for an iPhone app I’ve been working on, and I somehow managed not to kill myself. The documentation is not very good, but I don’t think that’s the main issue I was running into. What lost me about 10 hours of time was a few gotchas that I hope writing here will save others some time.
The basics.
To start off, I wanted a reasonable low latency audio IO with callbacks like I am used to. On the iPhone it seems you need to use AudioUnits for this.
At first I tried to implement an AudioGraph with a Mixer and a RemoteIO audio unit in that, but after a while I realized I could just use a single RemoteIO and no graph. See this site by Michael Tyson for more info. This one will show you how to set up an audio IO mic-in headset-out RemoteIO unit that is probably suitable for most applications that need recording. However I needed to modify it a bit for my purposes.
There are a number of other useful sites for introduction to iPhone audio IO including:
- Tim Bolstad made a pretty legit and working (I used this to start) audio IO example that uses Audio Graphs. As I said, this was over-complicating my situation so I stopped using this, but if you need more than a few AudioUnits, this is the way to go.
- AurioTouch is a pretty thorough example by apple, but due to its complexity I recommend the above two guides first.
- Nobihisa Nagano is a Japanese guy (that is also a supercollider user) who can provide a different approach to AudioUnits if you happen to read Japanese. Perhaps google translate can also help you here.
It’s actually getting quite late, so I’ll stop here for now and hopefully continue later.