(Michael Chinen)

Tatsumi Ryusui and Starcraft II and emacs

Over the last year in Berlin I worked on five collaborative music/performance/art projects that I’ve been meaning to post.

In the summer as a part of 48 Stunden Neukölln I did a collaboration with Tatsumi Ryusui (myspace).

Normally Tatsumi does experimental electronics improvisation and I do computer music or sonification stuff, but we were asked to play something a bit more poppy.

Tatsumi keeps everything real and is always exhibiting the utmost level of radness so I couldn’t pass this opportunity up. (After all, computer musicians secretly or publicly don’t like computer music and are always longing to find some pop outlet side-project to get some console for their self-imposed complexes.) Just kidding. I actually heard someone make this accusation for the general case. I must be still upset to post it here.

Tatsumi in white cow, me in aloha attire (it was friday):

Untitled from Tatsumi Ryusui on Vimeo.

Tatsumi played guitar and sung kayoukyoku, which is kind of between japanese folk and pop.
I made a effects processing program that has the interface inspired by emacs and Starcraft II.

You shouldn’t be freaking out. Starcraft II is a totally legit model for user interface. In general games are to interface as porn was to internet (although I can’t vouch for the correct SAT answer). As a competitive game it requires players to command with accuracy and dexterity while processing a huge amount of input. it is perhaps the most demanding in terms of user actions per minute (clicks, keystrokes). And it is essentially a performance environment, which is why some of the most popular channels are about watching shoutcasts of Starcraft II like Day9, HuskyStarcraft, and HDStarcraft. And not to mention it has a great community that amazingly includes genuinely cool genuinely interested genuinely cute girls like Press Heart To Continue. Yes, my love for starcraft is probably subject to second-hand embarassment. I’m not ashamed, so please learn to deal with it yourself or get in.

In using (text) programming as a medium for composition I often feel the need to be able to incorporate envelopes to control parameters such as volume/etc. This kind of thing the mouse is great for, and we’re pretty good at using it. It is of course imperfect for ‘drawing’ and I suck at ‘drawing’ and drawing to begin with (just how MouseX.kr and MouseY.kr in supercollider often brings snickers), but if processed correctly has many advantages for providing this need. No one’s saying you have to do a direct motion to parameter mapping. Also, we are all super fucking good at using the mouse now. If you have practiced an instrument and somehow got decent at it, you know how much time it takes. Chances are you spent a comparable amount of time using a mouse. Yet it’s somehow odd to think of it as practice.

Emacs is also great for its key input to do what you want as fast as possible. In general for commands you don’t touch the mouse and just press a combo of keys to execute commands.

The program I made was essentially a large effects box that allowed effect like delay/distortion/phaser/downsampling, mixing, zooming in on buffers, and quick parameter to mouse mapping based on keys/mouse position.
It looked like this:

It was done with OpenFrameworks, a minimal cross-platform C++ framework which I’ve been using in some Institute for Algorhythmics projects.

We’re going to play again on November 14 at Schlessiches Tor. I’ll post an update and info about other projects soon.

Macro blues and function reentry?

Knowing how to macro correctly is important in both the real world (starcraft) and when writing code.

in c you can define useful macros like 

#define FFMAX(a,b) ((a) > (b) ? (a) : (b))

. The macros will be a little better than using even static inline functions because no extra memory is allocated for the temporary variables that are implicit in a function like

static inline int max(int a, int b) { return a > b ? a : b; }

But using macros causes, as I found, REEAALLY if-not-hard-stupid-to-find bugs if you are putting expressions with side effects as the macro arguments. If you don’t know, stupid-to-find means you feel increasingly stupid during and after the finding.

I had a function with side effects that was in the macro, and I when I got some strange results I turned to valgrind and gdb to help find the problem. I also added a bunch of printfs. All this told me was that the function was being executed twice on the same line even though I had written it just once, and it wasn’t near a loop. I had gotten so used to ignoring the debugger when it jumps to the MAX(a, b) macro that I didn’t even think to look at the definition, which was of course the root of the problem. The problem is that macros do text substitution on its argument list, and not evaluation as normal functions do. This means that when I called

FFMAX(smallintval, my_side_effecting_function())

it expanded naturally to

((smallintval) > (my_side_effecting_function()) ? (smallintval) : (my_side_effecting_function()))

and caused double side effects to happen instead of the single.
This guy here has explained it much better than I.

The real problem though, is that from watching the debugger jump into the same function twice from the same line, I expected that there was some crazy stack thrashing causing reentry of some sort or some The Matrix-style “deja vu” (what the hell was that about anyway?) So I ran valgrind memcheck on it to of course no avail, because there was no buffer overflows, no memory bashing, no elegant landing on the program counter memcpy mistakes.

I stared and played around with the code for about 8 hours straight; I got it, of course, by then going to sleep and looking at it with a fresh eyes.
Here is a memento screenshot:

Cocoa and pthreads

結論から言う。
If you’re building a cocoa app, don’t use pthread mutexes and pthread threads.  Use NSThread and NSLock.  I’ve written a few cocoa apps that use pthreads, but maybe I’ve just been lucky till now.

I just spent about 8 work hours trying to figure out why my NSTextStorage object wouldn’t concatenate properly with my NSMutableAttributed, occasionally crashing the NSTextView. A lot of google searches pointed to memory stomping so I debugged with both GuardMalloc and NSZombieEnabled and set breakpoints on my raiseException methods. No beans.

Then I saw that maybe I should be surrounding my appendAttributedString calls to NSMutableAttributedString (and NSTextStorage) with -[NSMutableAttributedString beginEditing] and -[NSMutableAttributedString endEditing].  Did this.  Same result.

I was updating the text in my NSTextView from the consumer main thread by popping off strings from a worker producer thread.  I was doing this in a thread-way using pthread threads and pthread mutexes to make sure the shared objects were handled safely.  Then I noticed a mailing list post and Apple doc that says you need to create at least one NSThread before autoreleasing works correctly.  So I changed my pthread threads to NSThread threads, which are just pthreads that have a bit more on top.  Things still didn’t work.  Lastly out of desperation I changed my pthread mutexes to NSLocks.  Then everything worked.  It looks like somehow the pthread mutexes were slipping.  I don’t know why, but for now I’ve stopped the bleeding.

move to sourceforge

Many of the fucking sound projects have been integrated with institute of algorhythmics and are now available through a sourceforge svn repository. This is great as users can now download current source and play with the apps.

I’m aware that some users don’t know how to use svn.  So here’s a tutorial on how to download and compile our browser for mac.

How to get, build and run our source:

  1. Install XCode from apple. The version you need depends on which OS you are running (3.2 for 10.6, 3.1 for 10.5, or 2.5 for 10.4.) It’s a free download, but its huge and can take an hour to get, but if you are interested in programming there’s no reason you shouldn’t.  Go to this page and click on the “Mac Dev Center” button.  You’ll need to register a free account with apple to download it.
  2. If you don’t have fink, install it.  fink is a command-line tool that lets you download programs from source and magically builds them.  It also provides the option to download binaries via apt-get.  Installation should take about 10 minutes.  Yes, I know 10 minutes today is like three hours in late 20th century time, but it will really help you get some awesome tools including dict.
  3. Install svn using fink.  svn is a source control tool that lets many users work on the same code
    files at once, while keeping track of changes and progress.  There are GUI versions of it (rapidsvn or versions,) but I prefer to use it via terminal.  Here’s how to get it:Open the terminal (in /Applications/Utilities folder) and type:

    sudo apt-get install svn

    You can also type “sudo fink install svn” to build it from source, which you may have to do if a binary is not available.

  4. Download the repository source. If you want to you can make and cd to a new directory. Then download the source (creates a new folder in the current directory which you can see by typing pwd) 
     svn co https://algorhythmics.svn.sourceforge.net/svnroot/algorhythmics algorhythmics
  5. Now you can go back to the finder. Probably a new folder called algorhythmics exists in your home directory. Open it and open FuckingWebBrowser/FuckingWebBrowser.xcodeproj.
  6. Hit the build and go button or press command-R. The source then builds and then finally the program runs. If you want to tinker in the code you might be interested in a few files: FuckingAudioIO.cpp and FuckingSynth.h/cpp and its subclasses, which generate sound. If you are uneasy in c++ I recommend changing single number/parameter values and recompiling. If the program breaks after your change, simply revert back and try again.

Next, if you still want more fun, install dict using sudo fink install dict from the terminal. Now typing “dict artful” looks up through the net a bunch of meanings of the word and possibly save you from a silly word misuse that I sometimes still make. I like the tool because it’s really fast and light and I always have the terminal open.

Dict singing four versions of a popular sentence

Now you can open the project file NSSpeechSynthesizer.xcodeproj and build and run a singing recursive dictionary that uses dict to obliterate meaning and makes me wonder where it was in the first place.  It was hacked together from an apple speech example and is pretty crude. But one day I’ll get back to it and clean it up for a release.

There are likely to be problems for some people in this process. I will try to update troubleshooting as people post their issues. Until then happy hacking.

FuckingWebBrowser

FuckingWebBrowser sonification demo from Michael Takezo Chinen on Vimeo.

FuckingWebBrowser is a simple open-source WebKit based browser (for which there are hundereds of tutorials including many of annoying ones on youtube how to make in less than 2 minutes on a mac.) I added sonification, which converts the memory state into audio. The quality of sound is noise, but an overall structure is recognizable due to changes in the user interface (while the images load, while I mouse over/scroll, etc), which directly affect computer memory. From 5’30” on there is a special sneak preview of a tool codenamed FuckingFucker, which can attach to any process using its PID or BSD name and sonify a dissasembly of the process’s instructions at runtime (eg. and read the registers on the cpu). In this video FuckingFucker is attached to FuckingWebBrowser providing a double sonification.
FuckingFucker is still in development, and is a work in collaboration with Institute of Algorhythmics.

One can download the mac OS app or get the source of FuckingWebBrowser here. Of note, the related project, FuckingAudacity, is also up here. FuckingWebBrowser uses a simple portaudio callback to make sound. It has very little source code and can serve as a good example of how to recompile an open-source app into one that sonifies memory state. I intend to post a tutorial on exactly how to do this soon. A future release will also concern FuckingFucker, which provides the ability to sonify the runtime process of even closed-source apps.