Sound Garden, workshop at La Salle College of Art 1999
Ludger Brümmer
ludi@zkm.de, http://www.folkwang.uni-essen.de/~ludi
Sound tutorial

Granulare Synthesis


Fundamentals of the technique of granular synthesis:

Overview over the granular synthesis technique 


short introduction by Gerhard Behles.

the result of granular synthesis is a stream of so called grains. a grain is a piece of sound with a duration between 2 milliseconds and several hundreds of milliseconds (5 ms is a typical mean value); no matter what the under- lying sound material is, the grain has an envelope shape with some degree of fade-in- and fade-out time. these attack- and release portions of the envelope make sure that no truncation noises (clicks) will be heard, regardless of transients in the material. the choice of material to use for granular synthesis is basically independent of the model: sine waves, fm-sound and concrete sound (sound samples) or whatever. in order to achieve continuous sound output, grains are fired repeatedly. truax' model organizes grains in VOICES. one voice plays a grain, is silent for a while, plays a grain again, is silent again etc. this process can be controlled using two parameters: the grains DURATION and the DISTANCE between grains. the sum of the values for duration and distance is the PERIOD of a grain cycle. since typical values for distance and period lie in the milli- second range, a sensation of pitch will arise from the grain cycle, overlaying and interfering with the underlying material's pitch (if the material is pitched, that is). In fact, this process can be regarded as a form of AMPLITUDE MODULATION, and the sound that is obtained with short periods is often being associated with ring modulation. a source of statistical deviation is being introduced to enrich the quality of the synthesized sound: each time a grain is fired, a couple of random operations are invoked in order to determine this grains duration, its distance from the following grains, and maybe other parameters that influence what the grain sounds like; for example the PITCH or TRANSPOSITON of the underlying material. In fact, each of the relevant parameters, i.e. duration, distance, pitch, etc. is accessed by a pair of values: one FIX value and one RANGE value, where the result of a random operation is the "fix" value plus a random number between zero and the "range" value. so with a duration fix value of 10 ms and a duration range value of 5 ms you would be getting grains of a duration between 10 and 15 ms, or 12.5 ms average; with a steady, unbiased value for distance of, say, 20 ms this would result in a period varying between 30 and 35 ms, or in a grain FREQUENCY fluctuating between 33.3 Hz and 28.5 Hz. grain frequency in one voice is equivalent to the so called GRAINS PER SECOND value. at high grain frequencies, the effect of increasing duration and distance range values is often associated with adding NOISE. grain density is enriched by increasing the number of voices. each voice is doing the same thing (the process that has been described above) and is being controlled by the same set of parameters. so there is only one knob for grain duration fix no matter how many voices are sounding. in fact, as long as all range values remain zero, all voices will sound equal and in phase, so they will just sound like one voice at a higher amplitude. as the range value for, say, duration is increased, the onset of grains will be smeared among the voices, and the voices will be completely out of sync at some point. this is because new random deviations are being calculated with each grain's onset, and the grains in the different voices will be drifting apart as a consequence.

Modifications of the granular technique 


The technique that has been described above has been modified and extended in a variety of ways.
First, grains in one voice can overlap; a grain distance value around zero results in an overlap of four grains in each voice. In this manner, sound can be continuous in one voice, since no "holes" or "breaks" are heard.
Apart from using random deviation, a spreading of values among voices can be applied for a couple of parameter groups. The "spreading" value will equally distributed among the voices and then be added to the fix value. Using four voices, a spreading value of four will result in 1 being added to the fix value for the first voice, two for the second etc. A spreading value of ten will add two to the first voice, four to the second etc.
Grains are made from excerpts of the input sound. By picking excerpts in the order and distance that corresponds to the input sound, the input sound can be "re-assembled". By altering speed and direction of the input sound's traversal, sound can be slowed up or down arbitrarily. The choice of input sound excerpts can also be randomized.
The amplitude level of the input signal excerpt that a grain is made of can be measured and used to bias the program's behavior in variety of ways.

Phrenos
resulting pitch structure of a recursive algorithm


Example of fractal Event Coding as in the piece: 
Le tombeau de Maurice
Example of the Variable Structures as in 
The gates of H.
Examples of repetition structures in -Thrill-
1. Building up structure created out of the same repetitive pattern using different sounds

2. symmetric structure

3. asymmetric structure

4. accent with a building down structure

5. crescendo accent with building down structure

 
 

Physical Modeling



Flowchart of a flute model, created by Perry Cook, Princeton

Genesis an Example of a modular physical modeling software.



Slight introduction

Genesis is a software package driven by the physical modeling language "Cordis Anima" (Claude Cadoz, Anni Luciani, Jean Loup Florens. See Proceedings ICMC 1995 Banff, Computer Music Journal 17.1)
Cordis Anima is a generalized approach to physical modeling. It uses generalized particle physics paradigms based on the physical interaction between punctual masses. These masses are linked to each other and perform linear elastic and viscous interactions combined with finite state automata processes allowing the description of any kind of non linear interaction. One can build interaction like articulated objects, collisions, dry friction, adherence, sticking...
Some software functions and a GUI is added to create a software which feeds the specific needs of people working in music and acoustic. For example, the movement of some masses can be monitored and written to a floating point sound file. The sound file then appears in a sound editor where it can be listened and modified.
Besides these tools for handling audio data an animation procedure is available. The Animation shows the action of the current patch in a very much slower speed then the actual structure vibrates. This is done to make it a tool for understanding the interaction between the vibrating structures.

The theory behind cordis anima


The approach of the Cordis anima using particle physics for creating complex physical models is ruled by the paradigm of the interaction of punctual masses. This means equivalent to the physical existence of substances that single molecule like units create the frame of a physical object. These molecule like units which I refer to as masses are linked together to create an interaction which represents the flow of different forms of power between the masses. Power or force is a sum of different elements according to Newton's laws. These elements are speed, which in a discrete world is created out of the context between last position current position and time interval between these positions and weight. The connection between the punctual masses is installed with links serving the purpose of transforming the different states of force. Masses perform a transformation of force into speed i.e. position while links perform the translation of speed into force.
The displacement of 2 masses creates forces into two directions. The interaction force of the force between the 2 masses is correlated to the quality of the connection (viscosity or elasticity). Viscosity would reduce the force each time the link is being updated. The elasticity can be seen as the container of the force: when a spring is being stretched it contains the force injected into the system. As a consequence the 2 masses would move into opposite directions (one with positive force, one with negative force) until the energy is drawn out of the system by the amount of friction. This process would periodically change the value of force, while the two masses change the direction of movement.

Beside the quality of interaction the quality of the patch describes the resulting sound.

So lets describe how to build a model from scratch. Take an empty patch field and align the masses in a 2 dimensional field. Now place the masses on the patch and connect them with REF links. The patch will describe the flow of energy through the system. Now define the K-value (elasticity) and the Z-value (viscosity). If you forget to do so the two masses would have no connection since the Links have the default value of 0. After the definition of the patch 2 points of monitoring have to be defined. The movement of these 2 masses are written into a sound file and then played. Its the same what in every instrument is done: never the complete string would perform the pressure waves. In a guitar, violin or a piano it is a specific point which is connected to the resonator. This point leads the movements of the string to the surface of the resonator which then brings the air into vibration. A patch must contain a SOL. This is the reference point for the vibration, it is like the fixed points of a string or a membrane. Its a point with indefinite weight.
The next aspect would follow the question how to make the model moving. There are 2 basic ways to inject energy into the system since the system itself does not contain initial energy:
1. by Displacement of a Mass
2. by an initial force
The task are performed with the Etat initial menu, click on Observation and then on Etat initial. Change either the position of a mass or the force.
If you got so fare, please don't hesitate to save you patch so that you do not loose the current work. Do this with the Enregister Sous function in the Fichier menu. Here you can as well load a patch into Genesis.

Now you want to look if the model does what it should. So trigger the Regarder in the Observation menu. The model should move there. If not, you forgot something. The next step would be to create a sound with it. Do it with the Ecoute button again in the observation menu. If you want to calculate a longer sound, just click on Duree Simulation and tell how long the sound should be. Some more tricks to ease the work to create larger patches I will tell you during the introduction of the course.

The normal link performs a linear mapping between input force and output force  f.e. the higher the input force given a specific elasticity or viscosity the higher the output force. When you think of the hammer of a piano you would have a hammer which is not at all connected to the string. But in the moment it would strike the string a rigid connection between both object would appear. To model this behavior, a Conditional Link is introduced into Genesis. The conditional link applies areas of nonlinear behavior between these value fields. For example the following characteristic would cause the link to break if the applied force would increase a specific value. The normal conditional link has a connection between 2 masses in case the position is 0 or negative. With the example of string and hammer this example shows as follows: In case the hammer is in a positive position, i.e. above the string it would have no connection. In case the hammer hits the string at position 0 the position defined with the K and Z value take in effect until they both separate. Of course during the moment of the hammer hits the string a complex interaction occurs. But this is typical for a physical defined interaction and give the attack in the sound the natural character.

input force ..../|
                   ../ .|
                   / ...|______output force or position delta

Here a sentence of Claude Cadoz about the conditional link "which allows to move one or more points of the structure out of the balance position, and then to release the link when the tensed force exceeds a predetermined threshold." {this is similar to a plucked string] "The second assumes a minimal modelisation of a striker, made of one or several masses carried along by a function of gestural command, m and linked at the structure excitation points by an interaction conditional link. (ICMC Proceedings 1994 Page 16/17).

To build up the knowledge about the parameters controlling a vibrating system see here how the different parameter are connected connected with each other. But if you don't understand: it just says that the Weight of a Mass, plus the amount of viscosity and the amount of elasticity has to be added and related to the Movement of the Mass to generate the value of the force.

Mp'' + Zp' + Kp = F


M=inertia (Traegheit) Z=viscosity K=elastisity F=external force p' and p'' are the first and second derivates of the postion p.

Since the network of interaction is created by separate link and Mass Modules it is possible to apply complex parameter modification to each desired element of the network.


functionality of the bi-directional communication between 2 masses (sorry for the german expression) Kraft means force and Bewegung means movement.

A special form represents the conditional link which can functions in specific circumstances. It allows to include friction as a specific way to transmit energy. A bow f.e. has moments where the string and the hair create solid connection followed by a short time segment where the string slips and the bow is separated from the string. This effect could be created with a conditional link which allows the delivery of energy from one mass to the other only until a threshold of movement or distance is reached.
Since the space of the masses and its links is one dimensional, the representation of the position of the mass is not equivalent to the 3 dimensional position the mass has in the simulation. This means that the model does not take in advance geometric aspects like angle.
What especially is interesting in the resulting sound out of such a physical model is the fact its natural behavior and as well its possibilities of modification. Usually a sound if it is created in an artificial manor its modification shows some artifacts of the synthesis method.
These elements will in all case of decomposition be a physical model and behave like one. The modification of the patch and the destruction of links and masses will not destroy the functionality of the system. Since it can be explained by the fundamental type of the structure of this model.

Since all the elements got a very general character, the decomposition does not destroy aspects of the functionalism as a physical object. A more specific model with separate function modules works only in special circumstances and parameter constellations similar to physical reality.

translation of the used german expressions:
bewegliche Massen = moving masses,
statischer Punkt = static Point
Kraftluss = flow of force, it represents the flow of the Mass displacement in a string
Feder Dämpfer Verbindung = spring damper connection
 

One limitation of the models should be mentioned. Since an audio signal consists of a pressure wave the simulation may consist of a 1 dimensional space where the masses are connected in 2 dimensions but moving in one direction. Its a kind of tricky idea but the amount of calculation reduction possible with this limitation is immense. The example shows a model of a 2 dimensional connection of the masses moving in 1 dimension.
 
 



Overview of the seminars topics


1. Genesis for Sound or image synthesis
2. Composition/Video Analysis
3. algorithmic composition
3.1 Common Music
3.2 Csound
3.3 Mix
 
 
 
 

Genesis


GUI Layout and examples of animations of  a string


several states of a plucked string


 
 

Genesis animations of complex patches


membrane with complex energy injectors

multiple parallel strings

 
 






Score


Le tombeau de Maurice




 
 





Score 


La cloche sans vallees



 

What to know for algorithmic composition with Csound and Common Music


Csound and Common Music are free, you can take the software home and use it on  Mac/PC/Linux/Unix.
You can get updates at ftp://ccrma.stanford.edu/pub/Lisp/cm/ and  http://www.leeds.ac.uk/music/Man/c_front.html

How to get started?



1. Common Music
double click on the Common Music Icon
load the file i2.lisp into your lisp image with the menu
write your algorithm and evaluate it, see evaluate in the menu
write open MacDisc:Folder:output.sco to define the output name of your file. Don't forget to have a .sco at the end of the file name. You have to do this only once, or until you change the name of the output file. Enter
write mix foo1,foo2 0,0 and then Enter. This tells which algorithm you want to evaluate and to write events into the scorefile. You can have one algorithm or multiple if they are delimited by a colon.
when there was an error somewhere type: "propeller ."

2. Csound



double click on Csound
write in the name of your .sco and .orc file
and perform the operation

for Csound you should call the soundfile you want to read "soundin" followed by a dot and a number. For example: soundin.23 . You can change the number behind the dot. You call the specific soundfile only by calling its number.

Now we have a look at the Common Music algorithm (all reserved words are bold):


(algorithm foo i2 (length 20 dur .1 rhythm .1 amp 1 atk 0 rel 0
                   file-nr 23 skip 0 ampl 1 ampr 1 rvbsend .1 file-nr 23)
    (setf pitch (between 4 20)))

algorithm is the function name
foo is the name you decide to give the algorithm
i2 is the name of the csound instrument you chose
length tell how many notes this algorithm should create
dur = duration of the note
rhythm = the position of the note. For example writing .1 would create a repetition of 10 note per second
amp = amplitude between 0 - 1
atk = attack time in seconds
rel = rel time
file-nr = write the number of the file you want to use
skip = where to begin reading the soundfile 0 = beginning, .1 = after .1 seconds
ampl = amplitude of left channel, just in case the dynamic balance between left and right is not o.k.
ampr = amplitude of right channel
rvbsend = how much reverb should this event get

an the algorithm for reverb:


(algorithm foo99 i99 (length 1 rhythm 0 dur (+ (* 70 .1) 3.5) rvbtime 3.5))

take one event, leave rhythm 0 and for dur take the sum of all the events plus reverb time (+ all-events rvbtime). rvbtime tells the reverberation length, or better, the size of the room it simulates.

and about Lisp


its a language where you write the function at the beginning and then the arguments (+ 1 2) and you use parentheses to structure to functional unit. You can nest the parentheses how much you want: (- 10 (+ 2 3)) but you always have to close an opened parentheses. Relax, we will not use much of it, only if you are an experienced hacker than .....
To show the level of hierarchy, the expressions are indented: left is higher right is lower.

a more sophisticate example of an algorithm


(algorithm foo2 i2 (length 70 dur .1 rhythm .1 amp 1 atk 0 rel 0
                   file-nr 23 skip 0 ampl 1 ampr 1 file-nr 23)
        (vars (rhythm-var .1))
 (setf rhythm (incf rhythm-var .01))
 (setf pitch (item (items .5 .6 .7 .8 .9 1 2 3 4 5)))
 (setf rvbsend (between 0 .2))

between = pics a random value between the 2 numbers. Will be fractional if they a fractions.
(item (items a b c d)) = will read the elements in cycle as much as you want: a b c d a b c d
vars = variable declaration. When you need a variable declare it here and then you can use it.
(rhythm-var .1) = the variable rhythm-var will start with the value .1
incf = increments the value of the variable rhythm-var by 0.01 so that its getting larger larger each time  0.11, 0.12, 0.13, 0.14 ....
 
 

Creative Steps on the way to the video


To give an overview above all the steps of the creation of the video in this workshop have a look at this picture
You see that there are some horizontal arrows in the plot. They mention the steps where at least the visual group has to interact with the audio group. But of course it is encouraged to work much closer together or at least to observe each others work several time throughout the creative process.