(Michael Chinen)

Over the last year in Berlin I worked on five collaborative music/performance/art projects that I’ve been meaning to post.

In the summer as a part of 48 Stunden Neukölln I did a collaboration with Tatsumi Ryusui (myspace).

Normally Tatsumi does experimental electronics improvisation and I do computer music or sonification stuff, but we were asked to play something a bit more poppy.

Tatsumi keeps everything real and is always exhibiting the utmost level of radness so I couldn’t pass this opportunity up. (After all, computer musicians secretly or publicly don’t like computer music and are always longing to find some pop outlet side-project to get some console for their self-imposed complexes.) Just kidding. I actually heard someone make this accusation for the general case. I must be still upset to post it here.

Tatsumi in white cow, me in aloha attire (it was friday):

Untitled from Tatsumi Ryusui on Vimeo.

Tatsumi played guitar and sung kayoukyoku, which is kind of between japanese folk and pop.
I made a effects processing program that has the interface inspired by emacs and Starcraft II.

You shouldn’t be freaking out. Starcraft II is a totally legit model for user interface. In general games are to interface as porn was to internet (although I can’t vouch for the correct SAT answer). As a competitive game it requires players to command with accuracy and dexterity while processing a huge amount of input. it is perhaps the most demanding in terms of user actions per minute (clicks, keystrokes). And it is essentially a performance environment, which is why some of the most popular channels are about watching shoutcasts of Starcraft II like Day9, HuskyStarcraft, and HDStarcraft. And not to mention it has a great community that amazingly includes genuinely cool genuinely interested genuinely cute girls like Press Heart To Continue. Yes, my love for starcraft is probably subject to second-hand embarassment. I’m not ashamed, so please learn to deal with it yourself or get in.

In using (text) programming as a medium for composition I often feel the need to be able to incorporate envelopes to control parameters such as volume/etc. This kind of thing the mouse is great for, and we’re pretty good at using it. It is of course imperfect for ‘drawing’ and I suck at ‘drawing’ and drawing to begin with (just how MouseX.kr and MouseY.kr in supercollider often brings snickers), but if processed correctly has many advantages for providing this need. No one’s saying you have to do a direct motion to parameter mapping. Also, we are all super fucking good at using the mouse now. If you have practiced an instrument and somehow got decent at it, you know how much time it takes. Chances are you spent a comparable amount of time using a mouse. Yet it’s somehow odd to think of it as practice.

Emacs is also great for its key input to do what you want as fast as possible. In general for commands you don’t touch the mouse and just press a combo of keys to execute commands.

The program I made was essentially a large effects box that allowed effect like delay/distortion/phaser/downsampling, mixing, zooming in on buffers, and quick parameter to mouse mapping based on keys/mouse position.
It looked like this:

It was done with OpenFrameworks, a minimal cross-platform C++ framework which I’ve been using in some Institute for Algorhythmics projects.

We’re going to play again on November 14 at Schlessiches Tor. I’ll post an update and info about other projects soon.

Many of the fucking sound projects have been integrated with institute of algorhythmics and are now available through a sourceforge svn repository. This is great as users can now download current source and play with the apps.

I’m aware that some users don’t know how to use svn.  So here’s a tutorial on how to download and compile our browser for mac.

How to get, build and run our source:

1. Install XCode from apple. The version you need depends on which OS you are running (3.2 for 10.6, 3.1 for 10.5, or 2.5 for 10.4.) It’s a free download, but its huge and can take an hour to get, but if you are interested in programming there’s no reason you shouldn’t.  Go to this page and click on the “Mac Dev Center” button.  You’ll need to register a free account with apple to download it.
2. If you don’t have fink, install it.  fink is a command-line tool that lets you download programs from source and magically builds them.  It also provides the option to download binaries via apt-get.  Installation should take about 10 minutes.  Yes, I know 10 minutes today is like three hours in late 20th century time, but it will really help you get some awesome tools including dict.
3. Install svn using fink.  svn is a source control tool that lets many users work on the same code
   files at once, while keeping track of changes and progress.  There are GUI versions of it (rapidsvn or versions,) but I prefer to use it via terminal.  Here’s how to get it:Open the terminal (in /Applications/Utilities folder) and type:

   sudo apt-get install svn

   You can also type “sudo fink install svn” to build it from source, which you may have to do if a binary is not available.

4. Download the repository source. If you want to you can make and cd to a new directory. Then download the source (creates a new folder in the current directory which you can see by typing pwd)

    svn co https://algorhythmics.svn.sourceforge.net/svnroot/algorhythmics algorhythmics

5. Now you can go back to the finder. Probably a new folder called algorhythmics exists in your home directory. Open it and open FuckingWebBrowser/FuckingWebBrowser.xcodeproj.
6. Hit the build and go button or press command-R. The source then builds and then finally the program runs. If you want to tinker in the code you might be interested in a few files: FuckingAudioIO.cpp and FuckingSynth.h/cpp and its subclasses, which generate sound. If you are uneasy in c++ I recommend changing single number/parameter values and recompiling. If the program breaks after your change, simply revert back and try again.

Next, if you still want more fun, install dict using sudo fink install dict from the terminal. Now typing “dict artful” looks up through the net a bunch of meanings of the word and possibly save you from a silly word misuse that I sometimes still make. I like the tool because it’s really fast and light and I always have the terminal open.

Dict singing four versions of a popular sentence

Now you can open the project file NSSpeechSynthesizer.xcodeproj and build and run a singing recursive dictionary that uses dict to obliterate meaning and makes me wonder where it was in the first place.  It was hacked together from an apple speech example and is pretty crude. But one day I’ll get back to it and clean it up for a release.

There are likely to be problems for some people in this process. I will try to update troubleshooting as people post their issues. Until then happy hacking.

FuckingWebBrowser sonification demo from Michael Takezo Chinen on Vimeo.

FuckingWebBrowser is a simple open-source WebKit based browser (for which there are hundereds of tutorials including many of annoying ones on youtube how to make in less than 2 minutes on a mac.) I added sonification, which converts the memory state into audio. The quality of sound is noise, but an overall structure is recognizable due to changes in the user interface (while the images load, while I mouse over/scroll, etc), which directly affect computer memory. From 5’30” on there is a special sneak preview of a tool codenamed FuckingFucker, which can attach to any process using its PID or BSD name and sonify a dissasembly of the process’s instructions at runtime (eg. and read the registers on the cpu). In this video FuckingFucker is attached to FuckingWebBrowser providing a double sonification.
FuckingFucker is still in development, and is a work in collaboration with Institute of Algorhythmics.

One can download the mac OS app or get the source of FuckingWebBrowser here. Of note, the related project, FuckingAudacity, is also up here. FuckingWebBrowser uses a simple portaudio callback to make sound. It has very little source code and can serve as a good example of how to recompile an open-source app into one that sonifies memory state. I intend to post a tutorial on exactly how to do this soon. A future release will also concern FuckingFucker, which provides the ability to sonify the runtime process of even closed-source apps.

Audacity GUI performance from Michael Takezo Chinen on Vimeo.

Audacity is an open source sound editor. I modified the code so that the sound playback depends upon the physical memory state of the program. Because the memory is dependent upon the user interaction, the user interface (buttons, menus, etc) as well as the structure of user-driven processes (effects, importing files, etc) become a part of the sonification process. The noisy quality of the sound is due to the fact that chunks of uninitialized memory are continuously sent to the audio output.

This recording was done with a point and shoot camera, so the audio quality is not so good, but it gives you an idea of what I’m trying to do.

The program is called FuckingAudacity. How to go about building it and programs like this will be up shortly.  This basics of this process are quite simple and can be attached to other open source programs as well.  I will try to document this along with how to attach portaudio to a program if it doesn’t have sound i/o, allowing pretty much any c/c++ open source program to having fucking versions.

This process has actually been quite helpful for finding bugs (and fixing them,) because the performance aspect makes for more extreme use cases.  For example, I fixed a bug that existed in 1.3.10 where pressing play in a second project while another was already playing would cause the audio glitch and stop.  This process also has a healthy result  It is introducing me to more code, and thus more devel mailing lists/communities.   For example, I’ve also been playing with SuperCollider in a similar fashion.

If you are interested here a some related projects that I know of:

Shintaro Miyazaki’s work sonifies computer processes externally:

algoRHYTHMS everywhere Version 0.2 from Shintaro Miyazaki on Vimeo.

These two other projects were on slashdot：
http://www.codesounding.org/indexeng.html
http://cessu.blogspot.com/2008/09/have-you-listened-to-your-program-today.html

  It’s interesting that they attach sonification data to MIDI and some meter, which is a pretty specific sonification scheme.  One of them uses valgrind, which I am interested.  Unfortunately it doesn’t seem to work so well on my 5 year old computer for GUIs.  But I want to find a way to work with a debugger that allows hotfixing/live program analysis (e.g. the call stack) as an extra layer of performance.

I’ve before puzzled over what it would take to create an extended wave that looks like DC offset or low frequency square wave (not an actual impossible square wave with infinite harmonics, but just one that mostly looks squarish.)  This is a case where the input signal and the speaker cone(s) movement and the resulting air pressure wave can look quite different from each other.

Another common time that square waves comes up is when clipping happens in the amp/interface.  This is a known speaker killer, but maybe not in all cases as I originally thought.

Today I actually googled the topics and found the following decent discussions of them.
http://www.diyaudio.com/forums/multi-way/5699-cant-reproduce-square-wave.html

http://www.bcae1.com/2ltlpwr.htm

http://www.rocketroberts.com/techart/spkr.htm

The diyaudio thread is huge, with some interesting parts.  Here are the main points I liked:

• If a speaker receives a DC, it will begin to move at a more or less constant speed proportional to the amount of voltage.  It will take a certain amount of time for the speaker cone to reach the maximum point it was designed for – which suggests that there are many square waves that will not result in blowing up your speakers.  These friendly square waves would then appear to be of relatively lower amplitude and/or higher frequency.  This is probably not so surprising, but now at least you might feel a little better about playing pan sonic at reasonable volumes.
• There is a derivative (as in calculus) effect between the input signal to the speaker, the speaker cone, and the waveform produced in the air.   You put a square wave into s speaker.  The cone moves in a triangular fashion.  This creates a square wave in the air.  Of course it doesn’t really matter so much except for phase when dealing with sinusoids with their uncreative deriviation and integration laws.

Of course, there are many square waves that will blow your speakers up.

Next, I want to find out what the hell wind is.