Justin.tv desktop streaming in linux

You should see the 'live' icon on your stream after you finish this tutorial

There are really no good tutorials for this that I could find, so here’s a tutorial on how to get justin.tv up on ubuntu. This should work with other linux distros. Before you start calculating the millions you’ll be making from ad revenue, lets first get you working with a stream. Then you can broadcast your sick rts skills or live coding sessions, as you see fit and rake in the dough.

http://apiwiki.justin.tv/mediawiki/index.php/Linux_Broadcasting_API
is the main site for how to do this, but it isn’t very detailed and doesn’t go over desktop streaming.

We’re going to use ffmpeg to stream the desktop and capture our audio as well.

1. First you need a justin.tv account, so go create one.

2.Then go to this page (while logged in) and click the ‘show’ link to view your stream key. This is sort of like a password, so don’t give it out.

3. Install ffmpeg if you don’t have it (you probably do,) but if you don’t for debian/ubuntu it’s the following:

sudo apt-get install ffmpeg libavcodec-extra-52

4.Start the stream.

INRES="1920x1080" # input resolution
OUTRES="1024x576"
FPS="20" # target FPS
QUAL="fast"  # one of the many FFMPEG preset
STREAM_KEY=live_231xxxxxxxxx #get your stream key as described above.

Then run ffmpeg with

ffmpeg -f x11grab -s "$INRES" -r "$FPS" -i :0.0  -f alsa -ac 2 -i hw:0,0 -vol 4096 -vcodec libx264 -vpre "$QUAL" -s "$OUTRES"  -acodec libmp3lame -ab 128k -threads 0   -f flv "rtmp://live.justin.tv/app/$STREAM_KEY flashver=FMLE/3.0\20(compatible;\20FMSc/1.0)" 

I modified this command from a forum post that used the one below. My changes were to not use pulseaudio and to boost the volume to twice as much (256 is default, so adjust accordingly)
some people had success with pulseaudio, but my ubuntu/wine config must not be setup to use it.
If audio fails, try this command instead.

ffmpeg -f x11grab -s "$INRES" -r "$FPS" -i :0.0  -f alsa -ac 2 -i hw:0,0 -vcodec libx264 -vpre "$QUAL" -s "$OUTRES"  -acodec libmp3lame -ab 96k -vol 4096 -ar 22050 -threads 0   -f flv "rtmp://live.justin.tv/app/$STREAM_KEY flashver=FMLE/3.0\20(compatible;\20FMSc/1.0)" 

The above command uses your audio output (what comes out of your speakers) as the audio to be streamed. To record from a mic, you can use pulse (I think it is installed by default):

ffmpeg -f x11grab -s "$INRES" -r "$FPS" -i :0.0  -f alsa -ac 2 -i pulse -vcodec libx264 -vpre "$QUAL" -s "$OUTRES"  -acodec libmp3lame -ab 96k -vol 4096 -ar 22050 -threads 0   -f flv "rtmp://live.justin.tv/app/$STREAM_KEY flashver=FMLE/3.0\20(compatible;\20FMSc/1.0)" 

This command should start your stream on justin.tv.
I am not sure why my audio is 1/16th what it should be and I need to scale the volume that much.

Next tutorial will be about how to integrate a webcam into a desktop streaming setup.

Update: I installed the pulse audio device/volume selector, and then ran the second command which uses pulse. While the stream is open I need to change from duplex stereo to output analog only, then change back. It’s weird that this fixes the filtering/low volume issue, but it does get around it. I only needed to make the swap once and it all works great.

sudo apt-get install padevchooser

I have recently switched to youtube because justin.tv/twitch tv doesn’t support saving or transfering videos to youtube if the content is not games. So while I may still use twitch for starcraft 2 and live streaming, I now use the following very similar command to record to mp4, then upload. I find I get much better framerates like this.
This command records the file to my videos folder with a timestamp attached. It wouldn’t take much more work to automate this upload process.

ffmpeg -f x11grab -s "$INRES" -r "$FPS" -i :0.0  -f alsa -ac 2 -i pulse -vcodec libx264 -vpre "$QUAL" -s "$OUTRES"  -acodec libmp3lame -ab 96k -vol 4096 -ar 22050 -threads 0 "$HOME/Videos/JLPTN1/cast720-$(date +%Y_%m_%d_%k_%M).mp4"

For some reason I needed $HOME instead of ~ to get the filepath to work.

Update2: I recently installed FFmpeg 0.8.10 which uses libavcodec 53.8.0, and this works for the mp4 capture, but fails silently with the rtmp output to justin.tv not going anywhere (Even after I take out the no longer valid -vpre and flashver=FMLE/3.0\20(compatible;\20FMSc/1.0) – if you leave the flashver in you get an error that says [rtmp @ 0x9be5fc0] Server error: Authentication Failed.
. My solution was to have two versions of ffmpeg – one compiled from source in /usr/local/bin/ and one using the safe libavcodec 52.72.2 – this is the one that apt-get gave me in /usr/bin/, and whenever I need justin.tv I use the older one.

SMB filesharing on mac (works with windows clients)

I have an old powerbook g4 (2004,) which was my main computer until last year. Apple products are expensive but I think it was pretty worth it’s money. Now I have a Core Duo (2006) macbook and a sweet homebrew core 2 quad pc whose main purpose in life is to fix audacity bugs on windows and make my actions per minute in starcraft higher. It is connected to my sound system so I want to be able to hear my library music from it. Little did I know, this small desire would bring to an epic smb quest.

WIFI is broken on both my new(er) macbook and pc, so I use the old powerbook as a network hub, sharing internet from wifi via ethernet. I want to use it to share files between mac and pc, so I tried using the “Windows File Sharing” option in the Sharing panel, which just boots up an smb server.

However it doesn’t work with PCs and has apparently crappy authentication. There are numerous posts about people trying to set the authentication to ntvlm1 by modifying the smb.conf. I tried this, and many other similar things, to no avail. Then I came across one solution which is to compile and install the latest smb.

It works now after messing with the damn thing for a good 4 or 5 hours. Since there are no obvious other guides on the net for the few folks who want to do this on mac, here are the steps and problems i ran into. You’ll need basic knowledge of the terminal and probably need the xcode developer tools installed to do this.

1. I typed “smb source download” and got version 3.2.6 or so.

Next steps in the terminal
2. Then I cd’d to the directory “source3” in the smb folder and typed

./configure --prefix=/usr;make;sudo make install

-note: there is also “source4”, but i believe this is an unstable branch.
-2nd note: also don’t use –prefix=/sur/local/sbin like some guy on a forum posts says he does because this will end up creating an (additional) sbin and lib directory within /usr/local/sbin.
-note: this takes about 15minutes to half an hour on a slow pc.

3. Now we need to edit the configuration file for smb.
In the terminal type:

emacs /usr/lib/smb.conf

and change

passdb backend = opendirectorysam

(or whatever it is) to

passdb backend = smbpasswd'

4.Above that line add 2 new lines :

smb passwdfile = /usr/lib/samba/smbpass.passwd
security = user

5.I also changed

guess account = unknown

to

guess account = youruseraccountname

(replace it with your user name of course)

6.Now save with control-x then control-s. quit emacs with control-x control-c

7.Now it’s time to start the client. in the terminal, type

sudo smbd -d

which starts the smb daemon.

8.You now need to add the password (this can only be done when smb is running, apparently.) Type

sudo smbpasswd -a yourmacusername

then enter whatever password you want (This can be different from your mac password.)

9.Connect from your mac/pc. If your ip is 192.168.1.50, then connect to smb://192.168.1.50/yourmacusername. On windows you can do this by mapping a network drive when you right click ‘My Computer’. On mac it’s in the ‘go’ menu->connect to server

-note if things don’t work out for you and you want to test and see error messages, use

sudo smb --debug-level=10 -i

if you do this you’ll need to kill it before starting the daemon (and vice versa). You may also need the kill command if smbd hangs.

I have some additional steps because I wanted to share an external USB drive, which is not possible by using symbolic links or mac aliases in your public folder.
to do this:
1.edit smb.conf again and add the lines

[extusb]
comment = extusb
path = /Volumes/My Passport
browseable = yes
read only = no

Replace “My Passport” with the name of your drive.
To access it just go to smb://192.168.1.50/extusb (if your ip is 192.168.1.50) and use the same password you provided above.

Then everything pretty much works. The latency is pretty crappy so I had to adjust VLC file cache size to play audio/video well. It’s probably not too hard to configure smb more to handle this, but I’m feeling smbd’d out right now. If someone figures it out thought let me know.

Some last notes:
My smb system was a 1.5ghz g4 on 10.4.11.
My smb clients were a macbook core2duo 2.0 ghz on 10.6.4 and 2.4 gHz Core 2 Quad on Vista x64
I read somewhere on a forum that the smb/opendirectory that comes with your mac is a modified version and not compatible with the newest public smb downloads.
I also kinda sorta know the proper way to handle daemons are with launchctl but I don’t know how to set it up, and I never restart this mac, so I haven’t looked into it.

sizeof dynamically allocated arrays in c

The MAX(a,b) macro bug I posted last was something I had done before. Today and Yesterday I spent a good few hours running over my old mistakes.

What’s a difference between:

unsigned char *dynArray = malloc(500);
unsigned char trueArray[500];

well, Today’s arrrrrrg is that sizeof (dynArray) is just the size of a pointer, not the size of the array as you would get when doing sizeof(trueArray).

assert(sizeof(dynArray)  == sizeof(unsigned char*));      //ok
assert(sizeof(trueArray) == sizeof(unsigned char) * 500); //ok
assert(sizeof(dynArray)  == sizeof(trueArray));           //fails

It’s likely that you knew this, and I had learned this at one point, and thought ‘hmm, how interesting’ while scratching my chin. But I have obviously forgotten it as I was using sizeof(dynArray) to iterate/initialize my array. I have to give credit to the phrases pointer-array-equivalence and somehow my comfort of using malloc(sizeof(*myPointer)) to confusing me into the conclusion that sizeof(dynArray) would really evaluate to 500. I remember writing that line and saying to myself rather convincingly that this is the right way to get the size of an array. It thus royally sucked debugging the actual code since I tend to start reverting the snippets that I am least convinced of first. But debugging the confusion process is actually quite nice. And now I’m pretty sure I’m not going to do that gotcha again. That’s my consolation sherlock.

Macro blues and function reentry?

Knowing how to macro correctly is important in both the real world (starcraft) and when writing code.

in c you can define useful macros like

#define FFMAX(a,b) ((a) > (b) ? (a) : (b))

. The macros will be a little better than using even static inline functions because no extra memory is allocated for the temporary variables that are implicit in a function like

static inline int max(int a, int b) { return a > b ? a : b; }

But using macros causes, as I found, REEAALLY if-not-hard-stupid-to-find bugs if you are putting expressions with side effects as the macro arguments. If you don’t know, stupid-to-find means you feel increasingly stupid during and after the finding.

I had a function with side effects that was in the macro, and I when I got some strange results I turned to valgrind and gdb to help find the problem. I also added a bunch of printfs. All this told me was that the function was being executed twice on the same line even though I had written it just once, and it wasn’t near a loop. I had gotten so used to ignoring the debugger when it jumps to the MAX(a, b) macro that I didn’t even think to look at the definition, which was of course the root of the problem. The problem is that macros do text substitution on its argument list, and not evaluation as normal functions do. This means that when I called

FFMAX(smallintval, my_side_effecting_function())

it expanded naturally to

((smallintval) > (my_side_effecting_function()) ? (smallintval) : (my_side_effecting_function()))

and caused double side effects to happen instead of the single.
This guy here has explained it much better than I.

The real problem though, is that from watching the debugger jump into the same function twice from the same line, I expected that there was some crazy stack thrashing causing reentry of some sort or some The Matrix-style “deja vu” (what the hell was that about anyway?) So I ran valgrind memcheck on it to of course no avail, because there was no buffer overflows, no memory bashing, no elegant landing on the program counter memcpy mistakes.

I stared and played around with the code for about 8 hours straight; I got it, of course, by then going to sleep and looking at it with a fresh eyes.
Here is a memento screenshot:

Cocoa and pthreads

結論から言う。
If you’re building a cocoa app, don’t use pthread mutexes and pthread threads.  Use NSThread and NSLock.  I’ve written a few cocoa apps that use pthreads, but maybe I’ve just been lucky till now.

I just spent about 8 work hours trying to figure out why my NSTextStorage object wouldn’t concatenate properly with my NSMutableAttributed, occasionally crashing the NSTextView. A lot of google searches pointed to memory stomping so I debugged with both GuardMalloc and NSZombieEnabled and set breakpoints on my raiseException methods. No beans.

Then I saw that maybe I should be surrounding my appendAttributedString calls to NSMutableAttributedString (and NSTextStorage) with -[NSMutableAttributedString beginEditing] and -[NSMutableAttributedString endEditing].  Did this.  Same result.

I was updating the text in my NSTextView from the consumer main thread by popping off strings from a worker producer thread.  I was doing this in a thread-way using pthread threads and pthread mutexes to make sure the shared objects were handled safely.  Then I noticed a mailing list post and Apple doc that says you need to create at least one NSThread before autoreleasing works correctly.  So I changed my pthread threads to NSThread threads, which are just pthreads that have a bit more on top.  Things still didn’t work.  Lastly out of desperation I changed my pthread mutexes to NSLocks.  Then everything worked.  It looks like somehow the pthread mutexes were slipping.  I don’t know why, but for now I’ve stopped the bleeding.