I have zero background in java and only a cursory knowledge of rendering, so this is what I could hack together. I also learned that MacOS X is not a great platform to develop java on.
I am currently working on a program to enable a conversation of sorts between a computer and person. Right now, it listens for a sound, then when someone make a noise, it randomly plays back one of its samples. The samples are the recordings of the sounds people made.
I call it cluster-converse because it was originally written to be a node in a cluster of these things, where they would share their collection of samples across the net. Someday, I hope to achieve that, but currently I have been diverted. It is written using Pure Data.
I have abandoned this project for now because I in the tests I ran, I did not get very good results. You can play with one of the tests here
I have started working on sonifying Polly's world. I thought a good place to start would be footsteps. Its currently done in Pure-Data using the data from Polly.java. I tried to simulate the weight of each falling footstep by tracking the change in the total Y value of Polly. The greater the total change in Y, the louder the footstep. The next step is to add sounds for head movements.
Launch Polly and then trigger the sounds to go with each action.
This soundfiles are designed to loop, providing an infinite footstep sound. Your soundplayer may screw up the looping by delaying before repeating...