DOI: 10.2478/V10051-011-0006-9
Educational Possibilities of the Project Colour Visualization of Music
Bojan Klemenc1, Peter Ciuha2, Franc Solina1
1Computer Vision Laboratory, Faculty of Computer and Information Science, University of Ljubljana, Tržaška cesta 25, 1000
Ljubljana, Slovenia, bojan.klemenc@fri.uni-lj.si, franc.solina@fri.uni-lj.si 2Academy of Fine Arts, University of Ljubljana, Erjavčeva cesta 23, 1000 Ljubljana, Slovenia, peter.ciuha@guest.arnes.si
We propose a system of colour visualization of music based on a system of colour signs, which are connected to musical tones. Tones, which are in harmonic relationships, are represented by related colours. First, we outline the foundations on which the system of colour signs is based - the mathematical model of harmony. We discuss several possibilities of visual representation of expressive elements of music - melody, composition, rhythm and harmony. These relationships enabled us to develop a computer program that employs these elements for visualization. The program mimics human perception in which the parts are determined by the perception of the whole. Furthermore, the program enables the development of tools that can enhance music understanding during listening or performing. Music performance can acquire a new quality with the use of interactive coloured musical instruments, which by using colours show the performer different possibilities for forming musical harmonies and thereby change the composing of music into a game and attractive colour-aural journey. Here we stumble upon a challenge for educational science and methodology: how to use such upcoming multimedia tools. These tools would bring the processes of learning and playing a game closer together since playing games is a child's most natural form of functioning. Furthermore, in the area of artistic creation we can once again establish a balance between our logical and intuitive nature.
Keywords: visualization, music, colours, learning, creativity
1 Introduction
In his book Optics, published in 1704, Isaac Newton connected seven tones of the octave with the seven colours of the rainbow. Probably the only explanation for his division of the rainbow into seven colours was his wish to achieve an alignment with the seven tones of the octave and possibly with the seven celestial bodies known at that time. He also described the colour wheel (Figure 1) and proposed several solutions for connecting the colour wheel to the tones (Collopy, 2009).
However, the study of harmony may have started even earlier. Pythagoras studied the nature of musical harmony in the 6th century BCE and discovered the first natural law based on small integer arithmetic. This influenced his students - the Pythagoreans and Plato heavily. They expanded the mathematical model to encompass the harmony of nature and the entire universe. A known example is Plato's work on Harmony of the spheres (Plato, The Republic, 380 BCE). In some way,
his discovery marks the start of science (Benson, 2006, Tramo et al., 2001).
Throughout history a lot of composers, musicians and syneasthetes sensed particular properties of music visually or with colours. Richard Wagner believed in Gesamtkunstwerk, a musical creation for all senses. Alexander Scriabin included a colour piano in his orchestral composition Prometheus: The Poem of Fire. The piano was supposed to project coloured light onto a screen in a concert hall, but Scriabin died before its realization. Pioneers of abstract painting, such as Wassily Kandinsky and Paul Klee, connected their art with musical language. However none of the systems for representation of music with colour came into widespread use, with the exception of some teaching methods, tools for visualization of music for the deaf and for entertainment.
Our project started with the desire to find a system for colouration of musical tones that could show fundamental harmonic relationships between the twelve semitones. Most assignments of colours to musical tones that are based on
Received: 5th January, 2011, received in revised from: 15th March, 2011, accepted: 24th April, 2011
1 A colour version of the paper is available from http://dx.doi.org/10.2478/v10051-0006-9


E

Figure 1: Newton's colour wheel with reconstructed colours1 and distribution of the colours in order of rising frequencies.
physical properties of sound distribute the tones linearly in order of rising frequencies or wavelengths (Figure 1). Such distribution can show melodic distance between tones and can be used for teaching songs and melodies, but it fails to explain that some tone combinations may, when played concurrently, sound consonant or dissonant. Using the aforementioned colouring method, the consonant combinations of tones are represented by very different colours and dissonant tone combinations can have very similar colours.
A more appropriate solution is to colour similarly sounding tones or tone combinations with similar colours. We devised a method for calculating a common colour for any combination of concurrent tones (Ciuha, Klemenc & Solina, 2010), which enables us to show musical harmony with colour. Further applications include the possibility to create coloured musical instruments that enable the performer to see the possibilities for forming musical harmonies with the help of colours, which in turn has educational value. On the other side,
our project could also open new ways of experiencing music for deaf people.
We created our system of colour signs based on the mathematical model of harmony. In visualization we need to visually represent expressive elements of music - melody, composition, rhythm and harmony. Meaningful solutions enable us to develop a computer program that uses these elements for visualization. The solution should be related to our perception in which the parts are determined by the perception of the whole. On that basis new musical tools and colour musical instruments could be created that would enhance understanding of music during listening or performing. Here we face new challenges for educational science. These tools could bring the processes of learning and playing games closer together, as playing is a child's most natural form of functioning. Finally, in the area of artistic creation we could re-establish a balance between our logical and intuitive nature.
The rest of the paper is organized as follows: in Section 2 we review the mathematical model of harmony. Section 3
Figure 2: Comparison of major and minor.
describes the proposed mapping between tones and colours. In Section 4, we study fundamental expressive elements of music and possibilities of their visualization. Section 5 presents our project "Colour visualization of music". Interactive colour musical instruments are outlined in Section 6. In Section 7, we discuss the possibility of using the system in education and conclusions are finally drawn in Section 8.
2 Mathematical model of musical harmony
In music language two fundamental types of scales and chords are called major and minor. A musical tone consists of a basic tone at a certain frequency and of higher harmonic components with frequencies that are integer multipliers of the basic frequency (Benson, 2006).
A major triad is very similar to the fourth, fifth and sixth multiplier of the basic frequency. We experience it as an enriched form of the basic tone (Figure 2: C major triad). Minor triad does not comply with that pattern, but is instead similar to one sixth, one fifth and one fourth of a common higher harmonic component of all three tones. It can be also interpreted as the 10th, 12th and 15th multiplier, which is the same as 60/6, 60/5, 60/4. In some way, the minor is a mirror form of the major (Benson, 2006, Parncutt, 1989). The musical character of the minor is defined by its middle tone (Figure 2: e minor), which means, that like in major, the relationship of 5:4 (or 15:12) is the most important relationship for the perception of a combination of concurrent tones. If the 10, 12 and 15 represent E, G and B, we get E minor, which is relative minor to G major (Balsach, 1997, Parncutt, 1989). E minor scale uses the same tones as the scale of G major. This holds true for majors and their relative minors.
From the times of ancients Greeks onwards we search for an explanation for the influence of music on humans and for the explanation for the music itself. Partial answer lies in the study of the mathematical relationships between musical elements. The relationships (ratios) deviate a bit from ideal values
in temperate scales since they have equal distances between all semitones.
If we choose the major and minor chord as the basis for the model of harmony, we want the colours of the visualised tones to be similar. In this way the performer or the listener could see that the tones he is using form a harmonic whole. The first model of harmony was proposed in the 6th century BCE by Pythagoras. He discovered that the most consonant sounding tone combinations are created by strings with special ratios of their lengths. The ratios of these lengths can be expressed by small integers. The string lengths are in inverse relationship with the frequencies of oscillation - shorter strings oscillate with a higher frequency (Benson, 2006). The ratio of 2:1 is called an octave and the tones comprising an octave are so similar that we can speak of octave equivalence. The next ratio is 3:2 or a fifth and can be used as a generator of a sequence (of an arbitrary number) of new tones. Pythagoras discovered that certain tones in the sequence are very similar and connected the sequence of tones into the circle of fifths. The fifth of the last fifth is almost equal to the first tone in sequence.
The number of fifths in the circle can differ, but often the simplest solutions are used: 5 tones (pentatonic), 7 tones of octave (diatonic scale) and 12 tones of chromatic scale (Parncutt, 1989). None of the circles of fifths is completely mathematically correct because results of multiplication by 3/2 can never give the same result as a multiplication by 2 (octave). This difference or error is distributed evenly among all the tones of the chromatic scale in the equally tempered scale. Among consonant relationships are relationships with ratios of frequencies of 2:1 (octave), 3:2 (fifth), 4:3 (fourth), 5:3 (sixth), 5:3 (third), 6:5 (minor third), 8:5 (minor sixth). They all have adequate approximations in the equal tempered scale. Less harmonic or non-harmonic ratios are 7:4 (16:9, 9:5, minor seventh), 7:5 (45:32, tritone), 9:8 (second), 15:8 (seventh) and 16:15 (minor second). These ratios have also worse approximations (Huron, 2008, Parncutt, 1989, Benson, 2006).
When we hear a musical tone, we perceive an array of different frequencies with integer multipliers of the base tone frequency. They are connected into a coherent harmonic whole
Figure 3: Coloured key spanning circle of thirds and the distribution of colours in an octave (see: http://dx.doi.org/10.2478/v10051-011-0006-9).
Figure 4: Comparison of colours of c minor (left) and C major (right).
by our aural system. Presence of order means that all frequencies came from the same sound source. Different independent sources on the other hand produce mathematically unrelated frequencies, which cause us to assure ourselves what is happening around us. We perceive them as unpleasant and dissonant, because they "wake us up" (Wells, 1980).
3 Proposed mapping of tones to colours
Contemporary systems for organising tones by harmonic relationships are usually two- or three-dimensional (Gatzsche et al., 2007, Bergstrom, Karahalios & Hart, 2007, Chew, 2000). Nearly all map the tones by a relationship of fifth (3:2) and third (5:4) (Parncutt, 1989). Our organisation of tones is a circle related to the circle of fifths where we can find all tones of a minor or major triad in a sequence called key spanning circle of thirds (Gatzsche et al., 2007). For mapping tones to colours we assign a colour wheel to the key spanning circle of thirds. We have to choose the direction of the colour wheel and initial alignment of the two circles. We chose white piano keys to have warm colours and black keys to have cold colours, but any other initial alignment would also be equal to the chosen one, so the initial alignment is not fixed and the user can change the associations by rotating and mirroring the colour wheel.
The upper right part of Figure 3 shows the basic or major colour of each tone. We can notice two rainbow-like sequences. Each tone appears twice in the key spanning circle of thirds, depending on its major and minor position. In this way, each tone has two different colours. The lower right part of Figure 3 shows the colour of relative minor added in a circle above the colour of the major. This gives us a colour arrangement where tones of the chord or tones belonging to same
scale have related colours. For example on Figure 4 we can see that C major triad is (magenta) red and that the c minor triad is blue.
We implemented this tone colouring model as a part of our multimedia application for visualising music with colour (Colour Visualization of Music). With an algorithm for calculating a common colour for any combination of concurrent tones (Ciuha, Klemenc & Solina, 2010), we could visualize musical harmony with colour and create coloured musical instruments that enable the performer to see the possibilities for forming musical harmonies with the help of colours. Composing of musical harmonies and music itself is thus transformed into a play, which gives a child or an adult a new entrance into music creation and also enriches experience of listening to music.
4 Visualization of fundamental expressive elements of music
4.1 From melody to composition
Visualization of a musical instrument or of a singing performance can clearly show the students if the melody is rising, falling or staying at the same height, additionally they can also observe the size of any jump or movement. The line that moves between the points (tones) of different heights is a very clear model of movement of the voice or the sound of an instrument (Figure 5). Visualization of the melody with a line enables a clear presentation of the whole and simplifies the integration of observed melody into the broader knowledge and understanding or comparison with other songs and compositions to search for similarities. Such knowledge exceeds the unconnected partial knowledge and is very important for permanent and long-term memory retention. Starting with a
Figure 5: An excerpt from Bizet's opera Carmen. On the lower side is the visualization; on the upper side is the line of melody.
melody, it is natural to continue into the study of composition, where we study how particular shapes change and connect themselves into bigger entities.
The two frames on Figure 5 denote repeating parts. We can observe two separate melodic movements - the melody and the accompaniment. In the visualization itself the melody can be seen as a movement that predominately resembles steps. This can be seen very clearly in monophonic compositions. Compositions with a complex harmony are usually based on polyphony, where more independent voices intertwine into a more complex harmonic entity. It is not always possible to determine an unanimous monophonic equivalent that would be equal to the psychological experience.
Melody is also a theme that can repeat itself more than once or even repeats itself constantly. In complex compositions we can observe various forms of repetitions with variations.
4.2 Time and rhythm
The repetition of musical elements, tones, themes and melodies represents the next dimension of music - time and rhythm. Like a picture or a statue exists in space, music exists in time. We commonly draw time as a line of events, similar to musical notation, writing or comic books. However, time in music is predominately cyclic. We can find a lot of concurrent movements, one inside another, and how to visualize them presents a challenge. We experience the repetition as rhythm and rhythm means movement, which can be observed in its purest form in African dances. Rhythm brings the listener into motion and enables us to live (beating of the heart and breathing), additionally it connects all the participants into a community. Music and rhythm is therefore even today a fundamental part of (religious) rituals. On the other hand, music and rhythm are sometimes also used to defeat one's will. This happened in concentration camps, where music forced the exhausted people to work or slaves on galleys to row according to the rhythm of drums.
When visualising rhythm we draw events as shapes in a timeline. Usually these sounds do not have qualities like tone height so we draw them as monochrome elements. Grey appears in the centre of the colour wheel and is also the result of visualizations for certain combinations of tones. Complex rhythms draw a complex shape and we can again observe rhythms inside rhythms and variations. Loudness of an event could be visualized as the size or degree of transparency.
In our visualization program the rhythm has its own plane, where every single rhythmic voice is represented in its own line (Figure 6 left). In melodic music tones also play the role of rhythm. They construct and limit the time, speed or movement, they change with breathing in or breathing out and with the two directions of a bow movement on a stringed instrument.
4.3 Harmony and timbre
Every voice or musical instrument produces its own pattern of fundamental tones with higher partial tones of different loudness. The recognition of these patterns of partial tones enables us to differentiate between particular musical instruments and voices. If the instruments are precisely mutually tuned, the performers can play with our aural perception and connect different musical instruments together in accordance with musical language. This emphasizes single tones or parts of the melody and joins them into bigger entities. Harmony can release feelings and melody creates movement and story. The whole range between perfect harmony and disharmony enables the whole range of feelings "from heaven to hell".
Evident is also the representation of modulation of harmony that is visualised as a colourful journey (Figure 6 right).
5 Project "Colour visualization of music"
Visualization of harmony was the main purpose of our visualization program (Ciuha, Klemenc & Solina, 2010). We tried to create a mathematical model that would behave as close as possible to the human aural perception. This task is not easy because we are not dealing only with passive reception but also with active cognition and recognition of meaningful entities based also on incomplete and missing information (Meyer, 1956, Parncutt, 1989). Ear functions as a frequency analyser and is very efficient at ordering sounds with different frequencies into bigger entities. If we perceive a non-complete pattern of frequencies, the aural perception system automatically fills in the missing parts of the whole. This happens when we hear only higher harmonic partials of a tone without hearing the basic frequency. In this case the aural system adds the missing basic frequency.
The aural system tries to order the information into the simplest possible form. This means that in the process of perception each part of information gets its final meaning based
Figure 6: Three-dimensional visualization of music with colours.
on the whole entity. In other words, the whole determines the contents of each part. Consequently the tones can have different interpretations in different music context. The same is true for the ratios of tone frequencies. For example the tritone can be interpreted and used in a harmonic way - as the ratio of 7:5 (as used by Debussy and Ravel) or as a dissonant interval 45:32 which appears in medieval music and is referred to as "Diabolus in musica" (Wells, 1980, Tramo, Cariani & Delgutte, 2001). We visualise the pure tritone as grey, because the two tones that form it are located on the opposite sides of the key spanning circle of thirds (Wells, 1980, Collopy, 2009).
Like our aural system joins the tones into a whole, the program analyses the tones and assigns an appropriate colours to the concurrent tones. The colour is calculated by representing the concurrent tones as appropriate vectors in a coloured key spanning circle of thirds and adds them together. For a detailed description of the algorithm refer to Ciuha, Klemenc & Solina (2010).
The next level of harmony analysis joins sequential temporal events. The results are represented as glowing colours around the tones and can show a presence of a key for that part of the musical piece. When there is no such key, the glowing is coloured grey or with unsaturated colours or is a sequence of very different glowing colours. On this level of analysis, all musical instruments and voices are processed together. If desired, the musical instruments can be separated and displayed on parallel planes. The visualization makes use of three spatial dimensions, which enables us to observe the visualized musical elements from different views, projections and perspectives (Figure 6).
We can group separate musical instruments by their timbre and separate them visually with patterns or even colours. This complies with the idea that different instruments have different colours. For example Wassily Kandinsky compared in his essay Concerning the Spiritual in Art (Kandinsky, 1910) the sound of a trumpet as yellow, the sound of a string instrument like a violin as green and a cello as deep blue (Collopy, 2009). However this kind of colour mapping excludes the mapping of harmony to colours.
The remaining challenges are the differentiation between major and its relative minor (for example C major and a minor) and display of functional harmony. What is the expressional difference between major and minor is still an unsolved question. Often is the music in minor keys connected with sad and
painful feelings. Leonard B. Meyer says in his work Emotion and Meaning in Music (Meyer, 1956) that there is no expres-sional difference between the major and the minor chord, but that the expressional difference is caused by usage of different scales and consequently of different melodic movements, augmented and suspended chords and also by slower tempo, different modulations and suspense (Parncutt, 1989).
Our program for music visualization (Ciuha, Klemenc & Solina, 2010) is in its current development phase still limited to tonal music. The analysis is made on music in MIDI format, which does not describe music as a sound waveform, but as a sequence of events (for example: each tone consists of two events in time - one for its start and one for its end).
Because the program is based on MIDI encoding it enables us to change the tempo of the music without distorting and changing of musical instruments (and their soundbanks). But the downside of MIDI is that the music is played by samples of sound of real instruments and is in consequence relatively lifeless compared to a real performance. In the next phase of the development of our program, direct audio signal could be used as input, followed by spectral analysis and finally by colour visualization. This is currently possible only if a third party program for conversion of audio signal into MIDI events is used. Finally, we come to a stage where we can evaluate our model of visualization with a series of experiments and measurements of physiological responses to aural-visual stimulus.
6 Coloured musical instruments
The next dimension of our project opens when a coloured musical instrument that is connected to the program is played. The program analyses the harmonic relationships during the performance in real-time and displays the results as colours on the musical instrument itself. For example, on a coloured piano the performer has keys coloured with 12 colours of a rainbow that represent the 12 major tonalities in the coloured key spanning circle of thirds (Figure 7 middle). By choosing and pressing a first key, each of the other keys colours itself in accordance with the possibility that the performer would press that key in the next moment. This means that all possibilities of playing and forming harmonies are displayed with colour. When the performer chooses and presses a second key the
Figure 7: Initial colours of the octave (middle) and colours during playing of different chords (left and right).
Figure 8: Coloured keyboard and coloured guitar.
process repeats itself and the results are new colours for each possible key.
Figure 7 displays virtual coloured octave during playing of chords. The performer playing on a coloured piano can choose which keys to press in every moment during his performance and his choices are not limited in any way. If he chooses vivid colours, he will stay in pure harmony and different colour nuances will lead him through different major and minor chords and tonalities. The choice of grey or unsaturated colours will bring him into dissonance or partial harmony. He can use disharmony to prevent his music to be one-dimensional or monotone.
During the performance the player can observe the computer screen or a projection with the visualization of his music and it is also possible to connect more performers together into a bigger joined performance. If they have very different musical skills, a musically inexperienced performer could offer coloured and musical cues on which the more experienced would build and improvise.
The coloured piano is currently in a prototype phase. The program can display one octave of a coloured piano. The resulting colours can be sent to a RGB LED computer keyboard (Figure 8) or a picture of the coloured piano keyboard can be projected onto real piano keys with the help of a LCD projector. Currently, we are also developing a system of RGB lit (velocity sensitive) key titles that would be placed over actual piano keys. The output of the velocity sensors would be sent to the computer program that would send back to the titles information which colours to display. A similar interface could be developed also for other musical instruments.
Another direction of the development of musical instruments would be the use of modern computer and communication technology, for example the aforementioned coloured or augmented standard keyboard. However even more interesting are modern tablet computers and mobile phones, music players and some game consoles with multi-touch displays. On
these devices the interface of a coloured instrument with all interactive elements can be displayed directly on the screen.
A coloured musical instrument could also be a part of a room or a concert hall which would lit up according to the music played (Ciuha, 2001).
7 Teaching with the help of multimedia materials
On the basis of coloured instruments and computer tools for visualization different programs and games can be developed. These could lead the student not only through learning of musical theory and practice but would enable him free creation, recording and modification of his own project. In addition a connection with other performers and the teacher is possible. From a technical point of view, there are no obstacles hindering the development of such applications. A number of musical games already exist that are derived from the idea of karaoke or include playing on computer equivalents of real instruments, for example guitars or drums. With the appearance and development of better multi-touch tablet computers a new era of teaching and learning is approaching. Interactive multimedia teaching materials can make most teaching subjects very interesting and exciting. Especially those that can convert the learning content into an experiment, adventure or game. In this case a child experiences the game as the purpose of his activity, but unconsciously acquires knowledge, which enables him to succeed in the game.
The project Colour Visualization of Music offers a possibility to make a step into an unexploited direction of music teaching that could empower children and adults to create music very naturally and not only by reproducing a musical score. The practice of drawing and painting is only rarely learned only by copying other pictures but mostly by creating artwork upon a theme, by experience, by understanding or emotion. New approaches to education could transform
consumerism and copy-paste practices into new creative experiences and schools could be transformed into a creative laboratories. It is only a question of vision.
Present day's curriculum encompasses art and music and others subjects and is developing in the direction of mutual interconnection of all learned content. Realisation of this aim today depends on communication between the teachers and on their openness, knowledge, desire and readiness to experiment in new forms of work. Teaching is an open process, which means that the teacher is no longer the one who knows the answer to every question and is in control the whole time. This can be an uncomfortable situation for many teachers.
A possible preparation for teaching music connected with colour and visual art could include a slideshow that would present basic ideas, together with audio, video and pictorial examples about the interconnection of the two art forms. Multimedia slides could be supported with actual experiments. When a fundamental understating and comprehension is achieved, the class can move on to actual demonstration of the visualization or at least play the recorded visualizations of different pieces and eventually move to music and visual art creation.
For the actual execution, optimal technical conditions are necessary besides having enough time: darkened classroom, LCD projector with vivid and saturated colours, a computer with powerful computer graphics, a good sound system and a connection to a digital piano or a coloured musical instrument or to an appropriate interface. The comprehension of coloured musical tools is achieved through practical demonstration. In parallel to the musical performance on a colour instrument, the remaining students in the class could draw, paint, dance, sing or in some other way create art on the basis of concurrent musical events. In this way the process of cognition deepens and connects itself with direct expression and with search for appropriate personal art equivalents. Or vice versa, the visual artworks or colour score can initiate musical exploration. Developing new creative practices can be a challenge for a motivated teacher and for his or her students.
The feeling of creativity is wonderful and intoxicating as self-realisation is the highest level in Maslow's hierarchy of human needs. A special quality of music created with coloured instruments is, that the often lost primeval happiness and enjoyment during creation of music can be found again. It happens too often that students forget the original purpose of musical education and direct their energy only into technique, repetition and skill, but forget the simple happiness of playing, listening, discovering and expressing themselves.
Such colour improvisation can be thought of as painting of music, or being like a painter, who instead of a brush and colour uses a colour piano. A musical walk through harmonies arises from that improvisation, which is free of limitations of melody and rhythm. In such music some qualities of nature like a sound of a waterfall or a singing bird appear. These can be monotone and mildly cyclical, but also variable and unexpected. These qualities are relatively rare in omnipresent pop music but can be found among jazz, improvised and modern serious music.
The last particularity of the teaching system is the relative absence of failure and consequently frustration because
a coloured instrument enables the realization of the ideal WYSIWYG (What You See Is What You Get). A failure may appear only as the result of a very superficial and disinterested approach. Such a state is not natural and is always acquired and thus can be unlearned with some help, understanding, guidance and trust. The system works if we use it and the use depends on the users themselves.
8 Conclusions: synthesis of art and science, play and project work as teaching model
The project Colour Visualization of Music is the result of an interdisciplinary approach in connecting science and art. This connection is not utilised enough on the societal level and in the context of the currently predominant scientific-technological approach. Different creative processes of science and art are based in the two specialised brain hemispheres. The left hemisphere is associated with symbols, logic and language and the right hemisphere is associated with recognition of patterns and faces, perceiving various relationships and with creativity, problem solving and intuition. The example of music and visual art shows us, that the right hemisphere can create new worlds of art, for which the science only later establishes explanations and deduces laws. Most of music and art theory was formed retrograde in this way. Thus art invents new rules and consequently meaning.
Teaching music with our visualization system is based on project work and direct experience. Project work means that the students understand the aim of the project and concrete objectives of the study - the practical project result. The knowledge that is the result of studying has a practical nature and is immediately used and tested during the learning project. Commonly, this knowledge interconnects several disciplines.
Direct experience is particularly strong in games. A game with its openness forces all participants into a repeating search for new creative solutions on the basis of rules that define admissible and forbidden moves. A related practice in science is an experiment, which is based on observation and attempt to understand the observed. Observed facts are ordered into patterns and patterns are in turn ordered into models. Results of experiments are compared with the forecasted results from the models.
The project Colour Visualization of Music can be used for observation and understanding of harmonies in music and understanding of music and music theory in general. On the other hand it could also open up new ways of experiencing music for deaf people. This, however, has not been evaluated yet. Another use of our music visualization program is a kind of a game, in which we create particular forms of music or music in general. There is no separation between theory and practice, so the project can connect the creative process of both brain hemispheres. Through observation we develop attention and enrich perception. This enables us to reach teaching objectives of stimulating curiosity, inquisitiveness, creativity and reach a creative surplus through a spiral-like development of the teaching process.
With the usage of an emancipatory approach we perceive the student or user as an equal to the mentor in the teaching process. This opens up a more efficient use of student's creative potential and stimulates activity based on a positive self-image. We offer a possibility of independent selection of teaching methods, which can be adapted to each student individually so that he can solve problems, make decisions and make reflections on the acquired learning experiences.
Creativity is essentially a skill for solving unprecedented problems and challenges, or finding better solutions for existing ones. Various art practices stimulate thinking outside of the box and in this way provide an effective development of latent creative potentials. Practicing art on regular basis can re-establish a balance between our logic and intuitive nature, which allows us to better know oneself and live a richer and more meaningful life.
Literature
Balsach, L. (1997). Application of virtual pitch theory in music analysis. Journal of New Music Research, 26(3): 244-265. DOI: 10.1080/09298219708570729 Benson, D. (2006). Music: A mathematical offering. Cambridge University Press.
Bergstrom, T., Karahalios, K. & Hart, J. C. (2007). Isochords: Visualizing structure in music, GI'07: Proceedings of Graphics Interface 2007, Montreal, Canada, May 2007, 297-304. DOI: 10.1145/1268517.1268565 Chew, E. (2000). Towards a Mathematical Model of Tonality. PhD thesis, Massachusetts Institute of Technology, Cambridge, MA. Ciuha, P. (2001). peter ciuha . the womb . statement, in 24th International Biennial of Graphic Arts. International Centre of Graphic Arts, Ljubljana, Slovenia, 2001, 99-102. Ciuha, P., Klemenc, B. & Solina, F. (2010). Visualization of concurrent tones in music with colours, Proceedings of ACM Multimedia 2010, Firenze, Italy, 25-29 October 2010, 16771680. DOI: 10.1145/1873951.1874320 Collopy, F. (2009). Playing (with) colour. Glimpse, 2(3): 62-67. Gatzsche, G., Mehnert, M., Gatzsche, D. & Brandenburg, K. (2007). A symmetry based approach for musical tonality analysis, 8th International Conference on Music Information Retrieval, ISMIR2007, Vienna, 2007, 207-210.
Huron, D. (2008). Asynchronous preparation of tonally fused intervals in polyphonic music. Empirical Musicology Review, 3(1): 11-21.
Meyer, L. B. (1956). Emotion and Meaning in Music. University of Chicago Press.
Parncutt, R. (1989). Harmony: A Psychoacoustical Approach, chapter 2. Springer-Verlag.
Tramo, M. J., Cariani, P. A. & Delgutte, B. (2001). Temporal coding of tonal harmony in the auditory nerve. Technical report, Harvard Medical School.
Wells, A. (1980). Music and visual color: A proposed correlation. Leonardo, 13: 101-107.
Bojan Klemenc is an assistant and a Ph.D. student at the Faculty of Computer and Information Science, University of Ljubljana. He graduated at the Faculty of Computer and Information Science in 2008. His research is focused on data visualization.
Peter Ciuha is a freelance Visual Artist and professor of Drawing, Painting and Printmaking at various schools and colleges. He received a B.A. and M.A. degrees in Painting and Printmaking from the Academy of Fine Arts and Design at Univeristy of Ljubljana, Slovenia in 1992 and 1997 respectively. He received two Art awards for Fractal prints and for interactive multimedia Fractal installation "Womb", in 1995 and 2001 at Ljubljana International Biennial of Graphic Art. He is developing new teaching metod for visual arts literacy and removing obstacles to creativity - "Five Worlds of Art". He researches connections between colour, musical harmony and mathematics of music and is developing interactive colour musical instruments.
Franc Solina is a professor of computer science at University of Ljubljana and Head of Computer Vision Laboratory at the Faculty of Computer and Information Science. He received a B.Sc. and a M.Sc. degree in Electrical Engineering from the University of Ljubljana, Slovenia in 1979 and 1982, respectively, and a Ph.D. degree in computer science from University of Pennsylvania in 1987. His research interests include range image interpretation, 3D shape reconstruction, panoramic imaging, and applications of computer vision in the arts.
Izobraževalne zmožnosti projekta Vizualizacija glasbe z barvami
Projekt Vizualizacija glasbe z barvami temelji na sistemu barvnih znakov, povezanih z glasbenimi toni. Znaki sorodnih barv se nahajajo na tonih, ki so med seboj v harmoniji ali sozvočju. V uvodu so predstavljene osnove, na katerih je osnovan sistem barvnih znakov - matematični model harmonije. Sledi razmislek o vidnem prikazu različnih izraznih elementov glasbe - melodiji, kompoziciji, ritmu in harmoniji. Te sorodnosti omogočajo razvoj računalniškega programa, ki uporabi te elemente za vizualizacijo ali prikaz barvnih in oblikovnih struktur glasbe. Program posnema človeško slušno zaznavo, v kateri so posamezni sestavni deli določeni s prepoznavanjem celote. Omogoča tudi razvoj orodij, ki lahko povečajo razumevanje med poslušanjem ali igranjem glasbe. Igranje glasbe pa lahko pridobi novo kvaliteto z uporabo interaktivnih barvnih glasbil, ki v povezavi s programom igralcu v vsakem trenutku z barvami prikažejo raznolike možnosti tvorjenja zvočnih harmonij. S tem spremenijo komponiranje glasbe v igro in privlačno barvno-zvočno potovanje. Tu se pojavi velik izziv za pedagogiko in metodiko, da osmisli in uporabi prihajajoča multimedijska barvna orodja. Ta bi lahko v veliki meri približala proces učenja igri, ki je otrokova najbolj naravna oblika učenja in delovanja. V umetniškem ustvarjanju pa lahko ponovno vzpostavimo ravnovesje med našo logično in intuitivno-kreativno naravo.
Ključne besede: vizualizacija, glasba, barve, učenje, ustvarjanje