- – - – - – - – - M I C H A E L – C H I N E N – - – - – - -

In debugging a particular problem when using someone else’s framework, one often needs to head to documentation for help. At this point one can go to the documentation, or google his exact problem. I tend to go for the latter as a first attempt. Then there are times where I do both for hours and still cannot find the solution though docs or google hits. These are really frustrating, so when I finally fix them I’ve decided to post the solutions here in case they can help someone else. They may need clarification. Just ask and I’ll quickly respond if I can remember.

When you say the same word over and over it starts to become meaningless. Mix this together with metaclasses and reflection. Then take objective-C and add it to C++. You then have an objective-C @Class construct for forward declaration, the Class object, the NSObject class method, and the C++ class keyword. It was only a matter of time before this would drive somebody crazy and lead me to waste an hour of debugging the wrong thing.

Here is the definition of the class method in objective-C which you can include in objective C++ without warnings or errors:

- (Class)class

‘class’ doesn’t mean anything in regular c or objective-c, but my projects often involve C++ classes, so I’m usually working in .mm files.
This lowercase method name matches the C++ keyword, but is allowed by the compiler in objective-c++, presumably because the compiler looks at context to apply c keywords to.

However this is not allowed:

-(void)giveMeAClass:(Class)class

I spent about an hour and a half trying to figure out how to forward-declare or include the class declaration. Because this was in a class header that also had NSObject, which declares the ‘class’ method above (which uses Class as a return value), this went against my intuition but unfortunately when going into debugging mode, sometimes I (and admit it, you too!) get fixated on one path and make some very silly decisions along the way that end up preventing a breadth first approach that would get the solution faster.

I did learn some interesting things along the way. Such as, Class is a typedef pointer to a struct, which is why it is one of the only (possibly the only) objective-c object that doesn’t use an asterix when passing it around.

But the real problem was not ‘Class’. It was ‘class’. ‘class’ is a c++ keyword, as mentioned earlier, and you cannot use it for parameter names, only for method names. I should have seen this, but the ‘class’ method case through me off.

The correct solution (as NSObject also uses) is just to call your Class variable names aClass.

-(void)giveMeAClass:(Class)aClass

Arg.

Guess who wins in signed vs unsigned integer conversion? I guessed wrong today, or rather, I’ve been assuming wrong for quite a while.

consider this example:

   std::vector<SomeClass*> myVec;
   int countdown = -1;
   // ... (fill the vector)
   if (countdown < myVec.size())
      countdown++;  // this will never get hit

myVec.size() returns size_t, which is an unsigned int and countdown is an int, so to compare these two, the compiler will implicitly cast. You will get warnings when you do this kind of comparison, so listen to them. I assumed in the conversion of primitive values, signed int has precedence. That line casts countdown to an unsigned int, meaning that it has a very high value (0xfffffff), so the conditional is always false.

So the if statment was evaluating to

   if ((unsigned int) -1 < 1) // pretend there is 1 element in the array
      countdown++;

which is

   if (0xFFFFFFFF < 1) // false (same as  UINT_MAX < 1)
      countdown++;

The correct code then is:

   if (countdown < (int) myVec.size())
      countdown++;  // this will now be run

Here are the rules for implicit type conversion from the C99 standard:

6.3.1.8 Usual arithmetic conversions

If both operands have the same type, then no further conversion is needed.
Otherwise, if both operands have signed integer types or both have unsigned integer types, the operand with the type of lesser integer conversion rank is converted to the type of the operand with greater rank.
Otherwise, if the operand that has unsigned integer type has rank greater or equal to the rank of the type of the other operand, then the operand with signed integer type is converted to the type of the operand with unsigned integer type.
Otherwise, if the type of the operand with signed integer type can represent all of the values of the type of the operand with unsigned integer type, then the operand with unsigned integer type is converted to the type of the operand with signed integer type.
Otherwise, both operands are converted to the unsigned integer type corresponding to the type of the operand with signed integer type.

I don't understand the statement after the bold section. But it is clear that unsigned types have precedence over same ranked types.

This is a basic outline that I will expand on. I just finished debugging this stuff and want to write the main points before I forget my references and main issues.

Summary: two basic gotchas of full duplex (record from mic and output to speaker) audio IO on the iphone that I ran into are:

1. The AudioUnits IO callback requires you to handle input buffers and output buffers in a very different fashion for each, whereas in the easy to setup core audio callback I used on OS X, they were handled in a similar fashion (both buffers as parameters to the callback). As they are buffers, mishandling them as I did caused my program to crash. Solution: use the parameter buffer for output, and AudioUnitRender() to obtain the input.
2. Sometimes my callbacks were working, verified by my printf debug statements, but no audio output could be heard – the speakers were silent despite a perfectly good-looking setup and debugging session. Cause: DC output or some other condition caused the iphone speakers to go into a muted state even when volume was on. Solution: plug and unplug a headphone.
3. When recording from the mic the audio will default to the headset speaker by the earpiece instead of the large and much louder main speaker by the dock. Cause: this is an apple default. Solution: Use AudioSession API to change the default.

Read on for the details.

I’ve implemented Audio IO using the native API on a few systems including old mac (pre OS X), new mac (Core Audio), and Windows. But since then I’ve been getting lazy and just using portaudio for everything, since it does what I want and gets a reasonable latency in most cases.

I just finished implementing my audio IO for an iPhone app I’ve been working on, and I somehow managed not to kill myself. The documentation is not very good, but I don’t think that’s the main issue I was running into. What lost me about 10 hours of time was a few gotchas that I hope writing here will save others some time.

The basics.
To start off, I wanted a reasonable low latency audio IO with callbacks like I am used to. On the iPhone it seems you need to use AudioUnits for this.
At first I tried to implement an AudioGraph with a Mixer and a RemoteIO audio unit in that, but after a while I realized I could just use a single RemoteIO and no graph. See this site by Michael Tyson for more info. This one will show you how to set up an audio IO mic-in headset-out RemoteIO unit that is probably suitable for most applications that need recording. However I needed to modify it a bit for my purposes.

There are a number of other useful sites for introduction to iPhone audio IO including:

• Tim Bolstad made a pretty legit and working (I used this to start) audio IO example that uses Audio Graphs. As I said, this was over-complicating my situation so I stopped using this, but if you need more than a few AudioUnits, this is the way to go.
• AurioTouch is a pretty thorough example by apple, but due to its complexity I recommend the above two guides first.
• Nobihisa Nagano is a Japanese guy (that is also a supercollider user) who can provide a different approach to AudioUnits if you happen to read Japanese. Perhaps google translate can also help you here.

It’s actually getting quite late, so I’ll stop here for now and hopefully continue later.

Today I had to publish an iOS app written in actionscript. There are a number of almost good guides like this one, which would be good if they just had more pictures.At first I used Flash Builder 4.5.1, which didn’t do the trick. I then used CS 5, and had everything building and running fine via the emulator. But when it came time to publish after generating the provisioning profile it just wouldn’t work. CS 5 kept saying “could not access the digital certificate. could not load keystore file (password may be incorrect)”. I recreated the .p12 file over and over and checked that I was using the distribution certificate instead of the dev one.
Eventually I thought maybe that my password was too complex, having characters like ‘!’ and ‘$’, but these are normal password chars, so I just stupidly assumed that this was not the issue. An hour later after still having no solution I changed the password to simple alphanumeric chars, and it worked. I didn’t see any google links from my searches that lead me to this fix, but I’m posting it here. This is a pretty lame bug, adobe.

You should see the 'live' icon on your stream after you finish this tutorial

There are really no good tutorials for this that I could find, so here’s a tutorial on how to get justin.tv up on ubuntu. This should work with other linux distros. Before you start calculating the millions you’ll be making from ad revenue, lets first get you working with a stream. Then you can broadcast your sick rts skills or live coding sessions, as you see fit and rake in the dough.

http://apiwiki.justin.tv/mediawiki/index.php/Linux_Broadcasting_API

is the main site for how to do this, but it isn’t very detailed and doesn’t go over desktop streaming.

We’re going to use ffmpeg to stream the desktop and capture our audio as well.

1. First you need a justin.tv account, so go create one.

2.Then go to this page (while logged in) and click the ‘show’ link to view your stream key. This is sort of like a password, so don’t give it out.

3. Install ffmpeg if you don’t have it (you probably do,) but if you don’t for debian/ubuntu it’s the following:

sudo apt-get install ffmpeg libavcodec-extra-52

4.Start the stream.

INRES="1920x1080" # input resolution
OUTRES="1024x576"
FPS="20" # target FPS
QUAL="fast"  # one of the many FFMPEG preset
STREAM_KEY=live_231xxxxxxxxx #get your stream key as described above.

Then run ffmpeg with

ffmpeg -f x11grab -s "$INRES" -r "$FPS" -i :0.0  -f alsa -ac 2 -i hw:0,0 -vol 4096 -vcodec libx264 -vpre "$QUAL" -s "$OUTRES"  -acodec libmp3lame -ab 128k -threads 0   -f flv "rtmp://live.justin.tv/app/$STREAM_KEY flashver=FMLE/3.0\20(compatible;\20FMSc/1.0)"

I modified this command from a forum post that used the one below. My changes were to not use pulseaudio and to boost the volume to twice as much (256 is default, so adjust accordingly)
some people had success with pulseaudio, but my ubuntu/wine config must not be setup to use it.
If audio fails, try this command instead.

ffmpeg -f x11grab -s "$INRES" -r "$FPS" -i :0.0  -f alsa -ac 2 -i hw:0,0 -vcodec libx264 -vpre "$QUAL" -s "$OUTRES"  -acodec libmp3lame -ab 96k -vol 4096 -ar 22050 -threads 0   -f flv "rtmp://live.justin.tv/app/$STREAM_KEY flashver=FMLE/3.0\20(compatible;\20FMSc/1.0)"

The above command uses your audio output (what comes out of your speakers) as the audio to be streamed. To record from a mic, you can use pulse (I think it is installed by default):

ffmpeg -f x11grab -s "$INRES" -r "$FPS" -i :0.0  -f alsa -ac 2 -i pulse -vcodec libx264 -vpre "$QUAL" -s "$OUTRES"  -acodec libmp3lame -ab 96k -vol 4096 -ar 22050 -threads 0   -f flv "rtmp://live.justin.tv/app/$STREAM_KEY flashver=FMLE/3.0\20(compatible;\20FMSc/1.0)"

This command should start your stream on justin.tv.
I am not sure why my audio is 1/16th what it should be and I need to scale the volume that much.

Next tutorial will be about how to integrate a webcam into a desktop streaming setup.

Update: I installed the pulse audio device/volume selector, and then ran the second command which uses pulse. While the stream is open I need to change from duplex stereo to output analog only, then change back. It’s weird that this fixes the filtering/low volume issue, but it does get around it. I only needed to make the swap once and it all works great.

sudo apt-get install padevchooser

I have recently switched to youtube because justin.tv/twitch tv doesn’t support saving or transfering videos to youtube if the content is not games. So while I may still use twitch for starcraft 2 and live streaming, I now use the following very similar command to record to mp4, then upload. I find I get much better framerates like this.
This command records the file to my videos folder with a timestamp attached. It wouldn’t take much more work to automate this upload process.

ffmpeg -f x11grab -s "$INRES" -r "$FPS" -i :0.0  -f alsa -ac 2 -i pulse -vcodec libx264 -vpre "$QUAL" -s "$OUTRES"  -acodec libmp3lame -ab 96k -vol 4096 -ar 22050 -threads 0 "$HOME/Videos/JLPTN1/cast720-$(date +%Y_%m_%d_%k_%M).mp4"

For some reason I needed $HOME instead of ~ to get the filepath to work.

Update2: I recently installed FFmpeg 0.8.10 which uses libavcodec 53.8.0, and this works for the mp4 capture, but fails silently with the rtmp output to justin.tv not going anywhere (Even after I take out the no longer valid -vpre and flashver=FMLE/3.0\20(compatible;\20FMSc/1.0) – if you leave the flashver in you get an error that says [rtmp @ 0x9be5fc0] Server error: Authentication Failed.
. My solution was to have two versions of ffmpeg – one compiled from source in /usr/local/bin/ and one using the safe libavcodec 52.72.2 – this is the one that apt-get gave me in /usr/bin/, and whenever I need justin.tv I use the older one.