English summary
Since time immemorial, music is part of the daily live of many people. Some find satisfaction in listening to music, others look for a more active form of musical involvement. One way to be actively involved in music is playing a (traditional) musical instrument. It allows people to express themselves by creating their own music or by performing music that is composed by others. But learning how to play a musical instrument often implies a long and intensive learning process. It therefore requires a lot of motivation, perseverance and even self-discipline.
Computer technology has also become part of our daily functioning. Moreover, ongoing technological developments (e.g. increasing computational power and expanding context through embedded computing) continuously change the way we interact with computers (Dourish, 2004). The traditional desktop computing is increasingly complemented and possibly even gradually replaced by a new breed of computational systems in which mouse and keyboard are exchanged for body and body movements as controller.
In the same way as technology has changed so many aspects of our daily activities, it has influenced the way we involve in music. New ways of interacting with the computer have introduced new possibilities for listening and playing. Indeed, we can listen to music anywhere, we make online play lists, share them through social networks and we retrieve music through all kinds of new ways of dealing with the computer (e.g. query by humming or by physically expressing the character of the music we look for). Furthermore, a variety of soft- and hardware applications have introduced new ways of composing and making music. These applications can replace the traditional instruments (e.g. Denis & Jouvelot, 2005). Why should one engage in the endeavour of learning how to play the bassoon, the cello or the violin if it is possible to more easily make music with new technologies such as the Wii® or the Kinect®? After all, the effort required for learning how to play traditional musical instruments contrasts sharply with the plug and play devices of today. However, technology can also play a more complementary role and, for example, support learning how to play a musical instrument. Indeed, on the one hand, music educators increasingly solicit these new technological possibilities in order to support music teaching and learning. On the other hand, some people believe that technology can replace the teacher.
In this thesis we combine the world of technology and the world of traditional musical instruments. We have developed an interactive computer system, called the Music Paint Machine, which is controlled by playing music on a (traditional) music instrument while moving on a coloured pressure mat. The system allows a musician to make a digital painting through the combination of sound and movement. Properties of the music (e.g. pitch, amplitude) and certain body movements (e.g. twisting the upper body, moving the feet) are mapped to properties of the painting (e.g. position on the digital drawing board, stroke size, colour). This interactive music system aims at supporting and enriching instrumental music instruction.
The investigations with the Music Paint Machine are framed within the embodied music cognition research paradigm (Leman, 2007). The theory of embodied music cognition asserts that the body mediates the cognitive processing of music. Understanding music and communicating this understanding is believed to rely on the process of translating music into a repertoire of corporeal experiences. Therefore, within the embodied music cognition paradigm, research focuses on the relationship between music, body and technology. Interactive music systems that integrate motion sensors allow the quantitative investigation of the bodily aspects of musical experience in an unobtrusive way. Increasingly, it is suggested that such systems also have an educational value. They appeal to the multimodal nature of musical experience and use body movement or the monitoring of these movements to develop musical skills. Not surprisingly, a growing number of research projects focuses on this educational value, trying to determine what type of systems can be designed to promote musical development.
It is within that specific context, namely music educational technology research, that this thesis is situated.
The presented research was guided by the following research question:
How and to what degree can an interactive music system contribute to the development of an embodied understanding of music when learning how to play a musical instrument?
Addressing this research question has lead to the development of three frameworks: a pedagogical, a technological, and an empirical framework.
In Part 1 of this thesis, the pedagogical framework is developed. First we describe the components of music instruction (Chapter 1). These components are the learner, the teacher, the interaction between the two and the musical material (Kennel, 2002). Next, we describe a number of characteristics that in the music educational and musicological literature are often attributed to traditional (instrumental) music instruction (Chapter 2). These characteristics are a master-apprentice model, the primacy of the score and an alienation from the learners’ daily reality. We elaborate on how these characteristics influence the different components of music instruction. Finally, we combine educational constructivism and the theory of embodied music cognition to formulate a possible alternative pedagogical approach (Chapter 3). In this approach, we emphasize the central and autonomous role of the learner, the construction of knowledge on the basis of concrete musical experiences, the importance of exploration and experimentation, and the role of the body for the development of musical abilities.
In Part 2 we develop the technological framework. We start with an overview of existing interactive music systems that aim at supporting instrumental music teaching and learning (Chapter 4, section 4.1). This overview is non-exhaustive but clearly shows that many interesting systems are being developed, in which visual feedback plays a prominent role. In most cases, the visual feedback is used to inform teachers and learners on different aspects of their playing (e.g. pitch/intonation, timbre). A growing number of systems also integrate sensors that enable motion tracking. As such, it becomes possible to also provide visual feedback on bodily aspects of music playing (e.g. posture, bowing gestures).
Next, we argue that these two features, namely visual feedback and motion tracking, can contribute to an embodied constructivist approach to instrumental music instruction (Chapter 4, sections 4.2 and 4.3). Interactive music systems are supposed to create a learning environment in which the subjective experience of the learner takes a central place. This way, learners can construct knowledge on the basis of their own experience. Visual feedback can support the construction of knowledge by influencing the cognitive processing of new information. By visually representing the effect of certain actions it is possible to accurately and unambiguously inform on the coupling between the action and the perceptual result of this action. As such, it becomes possible to correct or fine-tune the instrumental technique or musical expression. However, it is emphasized that visual feedback can also have a degrading effect on learning by introducing extra cognitive load or by creating a dependency on the feedback.
After the overview of existing systems and the elaboration on their potential to support an embodied constructivist approach, we present the interactive system that was developed throughout this research project (Chapter 5). This system, called the “Music Paint Machine”, allows musicians to make a digital painting by playing a (traditional) musical instrument while moving the body. Similar to many other systems, it is based on monitoring sound and movement and gives visual feedback on the playing. However, we believe that the Music Paint Machine differs from most systems in a number of ways. To begin with, the system goes beyond the mere provision of information as knowledge of performance or knowledge of results. Based on the combination of music, movement and visual output, it rather seeks to invite learners to explore and experiment with music, the musical instrument, the body and the visual representation of movement and sound (“painting”). The emphasis is no longer on a cognitive control of the music based on iconically or symbolically represented information, but on the physical, auditory and visual experience of the music. Moreover, the Music Paint Machine attributes an active role to body movement in the sense that movements are not only monitored but also serve to control the system in combination with the sound. This way, we hope, the Music Paint Machine appeals to the bodily processes that underlie musical meaning making. We argue that by intensifying a bodily involvement with the music the learner can develop a bodily motility or freedom that allows immersing into the music while playing. This view is supported by the principle of differential learning (Schöllhorn, 2000) and by the variability of practice hypothesis (R. A. Schmidt, 1975). Both argue in favour of variability when exercising a particular technique. Variability is seen as an alternative for repeatedly trying to maintain a posture or execute a movement in the same prescribed way. We also argue that developing this freedom contributes to the development of an optimal relationship with the instrument.
In addition to stimulating exploration and experimentation, the Music Paint Machine can be used in function of specific learning goals. The system’s mapping is so flexible that the translation of music and movement into the visualization can be adapted to the specific needs of a particular learning situation. As such, it is possible to introduce the learning content by starting with the concrete experience of interacting with the system. This experience can serve as the basis for the development of the symbolic or abstract knowledge related to that content.
In Part 3 of this thesis, we develop the experimental framework.
We start with an explanation of the rationale that underlies the development of the framework, namely the idea that the possible integration of the system in a naturalistic educational setting is the conditio sine qua non to arrive at a valid scientific investigation (Chapter 6). Accordingly, the research was guided by three objectives. The first objective was to adopt a pedagogy driven approach. We assert that the educational technology research needs to start from pedagogical-didactical concerns and goals, rather than celebrating the seemingly unlimited possibilities of diverse technologies (e.g. motion sensors) or software platforms. The second objective was to stay connected to the field of practice. We strongly belief that the Music Paint Machine and the research that was conducted with it, need to be relevant for daily instrumental music instruction. This implies that the knowledge and experience of teachers and learners are an important source of information for the development of the system. A third objective emerged throughout the research and concerns the focus of the empirical investigations. Gradually, it became clear that evaluating the effectiveness of an educational technology such as the Music Paint Machine couldn’t be solely based on testing learning outcomes. It also needs to focus on the way the technology has an impact on the different components of instruction (learner, teacher, the interaction between both, the musical material or learning content). It became clear that an educational technology is much more than an independent variable that is inserted in the instructional process in order to induce an amplicative effect on the basis of its features.
Before we describe the experiments with the Music Paint Machine, we present a theoretical investigation of the relationship between musician and musical instrument (Chapter 7). We argue that an optimal relationship between the two involves the incorporation of the instrument. In other words, musician and musical instrument merge into a unity. We unravel the processes that underlie such an optimal relationship. This way, we were able to define the basic components of an embodied interaction with music. These components are: an optimal experience, direct perception and skill-based playing. We define an optimal or “flow” experience, in which a musician is completely absorbed in the act of playing music, as the combination of the feeling of presence and a positive emotional condition. The concept of presence originates in the research on computer generated virtual reality. In this thesis, we elaborate the concept within the embodied music cognition framework.
The theoretical investigation of the relationship between musician and musical instrument provided the basis for the concept of the Music Paint Machine but also for the experiments with the system.
The first experiment was a user study that probed the participants’ personal experience with the system and their evaluation of its didactic potential (Chapter 8). We measured flow with the Flow State Scale (Jackson and Eklund, 2004) and presence with an in house designed questionnaire. This way, the measurement of presence was introduced in music research. Up until now, the concept of presence was only briefly referred to in the literature on music research (e.g. Leman, 2007). The results of this experiment suggested the system’s potential to elicit a flow experience. They also provided an empirical validation of the intrinsic relationship between flow and presence (Chapter 9). With regard to the evaluation of the didactic potential, the results indicated that both students and teachers value the system as a way to support musical development and learning how to play a musical instrument.
The second experiment was a longitudinal study in which children learned to play the clarineo with the instructional support of the Music Paint Machine (Chapter 10). In this study good practices were developed and the amplicative impact (focus on outcomes) of the system was tested on the basis of a non-equivalent control groups design. During nine months, twelve children received instruction on a weekly one-hour basis. Six children (two groups of three) received instruction with the Music Paint Machine, six children (two groups of three) received instruction without the system. The learning content was the same for the control and experimental groups. We investigated whether using the Music Paint Machine has an effect on the developmental music aptitude of the children. Music aptitude is a child’s potential for music achievement that can develop during the first nine years on the basis of the child’s musical experiences. According to Gordon (1986), the degree to which music aptitude develops also depends on the quality of formal music instruction. In our longitudinal study, we hypothesized that using an interactive music system that combines visual feedback and movement possibly contributes to the quality of instruction and thereby to the development of the children’s music aptitude. We did not find a significant difference between the control and treatment groups with regard to the dependent variable, music aptitude. However, this study made clear that it is necessary to focus on the transformative impact (focus on processes) of technology. On the basis of the weekly lessons, it was experienced by the researcher-teacher that using the system has a deep impact on the classroom events during instruction. It became clear that integrating the system in instruction changes the way the teacher and the students relate to each other and the way instruction is organized. We believe that investigating these transformations needs to be part of music educational technology research.
In Part 4 of this thesis, we first describe the contributions of the research tat was presented in this thesis. Next, in the general discussion, we critically discuss different aspects of the research. This is followed by an outlook on future research and by a final conclusion.
Since time immemorial, music is part of the daily live of many people. Some find satisfaction in listening to music, others look for a more active form of musical involvement. One way to be actively involved in music is playing a (traditional) musical instrument. It allows people to express themselves by creating their own music or by performing music that is composed by others. But learning how to play a musical instrument often implies a long and intensive learning process. It therefore requires a lot of motivation, perseverance and even self-discipline.
Computer technology has also become part of our daily functioning. Moreover, ongoing technological developments (e.g. increasing computational power and expanding context through embedded computing) continuously change the way we interact with computers (Dourish, 2004). The traditional desktop computing is increasingly complemented and possibly even gradually replaced by a new breed of computational systems in which mouse and keyboard are exchanged for body and body movements as controller.
In the same way as technology has changed so many aspects of our daily activities, it has influenced the way we involve in music. New ways of interacting with the computer have introduced new possibilities for listening and playing. Indeed, we can listen to music anywhere, we make online play lists, share them through social networks and we retrieve music through all kinds of new ways of dealing with the computer (e.g. query by humming or by physically expressing the character of the music we look for). Furthermore, a variety of soft- and hardware applications have introduced new ways of composing and making music. These applications can replace the traditional instruments (e.g. Denis & Jouvelot, 2005). Why should one engage in the endeavour of learning how to play the bassoon, the cello or the violin if it is possible to more easily make music with new technologies such as the Wii® or the Kinect®? After all, the effort required for learning how to play traditional musical instruments contrasts sharply with the plug and play devices of today. However, technology can also play a more complementary role and, for example, support learning how to play a musical instrument. Indeed, on the one hand, music educators increasingly solicit these new technological possibilities in order to support music teaching and learning. On the other hand, some people believe that technology can replace the teacher.
In this thesis we combine the world of technology and the world of traditional musical instruments. We have developed an interactive computer system, called the Music Paint Machine, which is controlled by playing music on a (traditional) music instrument while moving on a coloured pressure mat. The system allows a musician to make a digital painting through the combination of sound and movement. Properties of the music (e.g. pitch, amplitude) and certain body movements (e.g. twisting the upper body, moving the feet) are mapped to properties of the painting (e.g. position on the digital drawing board, stroke size, colour). This interactive music system aims at supporting and enriching instrumental music instruction.
The investigations with the Music Paint Machine are framed within the embodied music cognition research paradigm (Leman, 2007). The theory of embodied music cognition asserts that the body mediates the cognitive processing of music. Understanding music and communicating this understanding is believed to rely on the process of translating music into a repertoire of corporeal experiences. Therefore, within the embodied music cognition paradigm, research focuses on the relationship between music, body and technology. Interactive music systems that integrate motion sensors allow the quantitative investigation of the bodily aspects of musical experience in an unobtrusive way. Increasingly, it is suggested that such systems also have an educational value. They appeal to the multimodal nature of musical experience and use body movement or the monitoring of these movements to develop musical skills. Not surprisingly, a growing number of research projects focuses on this educational value, trying to determine what type of systems can be designed to promote musical development.
It is within that specific context, namely music educational technology research, that this thesis is situated.
The presented research was guided by the following research question:
How and to what degree can an interactive music system contribute to the development of an embodied understanding of music when learning how to play a musical instrument?
Addressing this research question has lead to the development of three frameworks: a pedagogical, a technological, and an empirical framework.
In Part 1 of this thesis, the pedagogical framework is developed. First we describe the components of music instruction (Chapter 1). These components are the learner, the teacher, the interaction between the two and the musical material (Kennel, 2002). Next, we describe a number of characteristics that in the music educational and musicological literature are often attributed to traditional (instrumental) music instruction (Chapter 2). These characteristics are a master-apprentice model, the primacy of the score and an alienation from the learners’ daily reality. We elaborate on how these characteristics influence the different components of music instruction. Finally, we combine educational constructivism and the theory of embodied music cognition to formulate a possible alternative pedagogical approach (Chapter 3). In this approach, we emphasize the central and autonomous role of the learner, the construction of knowledge on the basis of concrete musical experiences, the importance of exploration and experimentation, and the role of the body for the development of musical abilities.
In Part 2 we develop the technological framework. We start with an overview of existing interactive music systems that aim at supporting instrumental music teaching and learning (Chapter 4, section 4.1). This overview is non-exhaustive but clearly shows that many interesting systems are being developed, in which visual feedback plays a prominent role. In most cases, the visual feedback is used to inform teachers and learners on different aspects of their playing (e.g. pitch/intonation, timbre). A growing number of systems also integrate sensors that enable motion tracking. As such, it becomes possible to also provide visual feedback on bodily aspects of music playing (e.g. posture, bowing gestures).
Next, we argue that these two features, namely visual feedback and motion tracking, can contribute to an embodied constructivist approach to instrumental music instruction (Chapter 4, sections 4.2 and 4.3). Interactive music systems are supposed to create a learning environment in which the subjective experience of the learner takes a central place. This way, learners can construct knowledge on the basis of their own experience. Visual feedback can support the construction of knowledge by influencing the cognitive processing of new information. By visually representing the effect of certain actions it is possible to accurately and unambiguously inform on the coupling between the action and the perceptual result of this action. As such, it becomes possible to correct or fine-tune the instrumental technique or musical expression. However, it is emphasized that visual feedback can also have a degrading effect on learning by introducing extra cognitive load or by creating a dependency on the feedback.
After the overview of existing systems and the elaboration on their potential to support an embodied constructivist approach, we present the interactive system that was developed throughout this research project (Chapter 5). This system, called the “Music Paint Machine”, allows musicians to make a digital painting by playing a (traditional) musical instrument while moving the body. Similar to many other systems, it is based on monitoring sound and movement and gives visual feedback on the playing. However, we believe that the Music Paint Machine differs from most systems in a number of ways. To begin with, the system goes beyond the mere provision of information as knowledge of performance or knowledge of results. Based on the combination of music, movement and visual output, it rather seeks to invite learners to explore and experiment with music, the musical instrument, the body and the visual representation of movement and sound (“painting”). The emphasis is no longer on a cognitive control of the music based on iconically or symbolically represented information, but on the physical, auditory and visual experience of the music. Moreover, the Music Paint Machine attributes an active role to body movement in the sense that movements are not only monitored but also serve to control the system in combination with the sound. This way, we hope, the Music Paint Machine appeals to the bodily processes that underlie musical meaning making. We argue that by intensifying a bodily involvement with the music the learner can develop a bodily motility or freedom that allows immersing into the music while playing. This view is supported by the principle of differential learning (Schöllhorn, 2000) and by the variability of practice hypothesis (R. A. Schmidt, 1975). Both argue in favour of variability when exercising a particular technique. Variability is seen as an alternative for repeatedly trying to maintain a posture or execute a movement in the same prescribed way. We also argue that developing this freedom contributes to the development of an optimal relationship with the instrument.
In addition to stimulating exploration and experimentation, the Music Paint Machine can be used in function of specific learning goals. The system’s mapping is so flexible that the translation of music and movement into the visualization can be adapted to the specific needs of a particular learning situation. As such, it is possible to introduce the learning content by starting with the concrete experience of interacting with the system. This experience can serve as the basis for the development of the symbolic or abstract knowledge related to that content.
In Part 3 of this thesis, we develop the experimental framework.
We start with an explanation of the rationale that underlies the development of the framework, namely the idea that the possible integration of the system in a naturalistic educational setting is the conditio sine qua non to arrive at a valid scientific investigation (Chapter 6). Accordingly, the research was guided by three objectives. The first objective was to adopt a pedagogy driven approach. We assert that the educational technology research needs to start from pedagogical-didactical concerns and goals, rather than celebrating the seemingly unlimited possibilities of diverse technologies (e.g. motion sensors) or software platforms. The second objective was to stay connected to the field of practice. We strongly belief that the Music Paint Machine and the research that was conducted with it, need to be relevant for daily instrumental music instruction. This implies that the knowledge and experience of teachers and learners are an important source of information for the development of the system. A third objective emerged throughout the research and concerns the focus of the empirical investigations. Gradually, it became clear that evaluating the effectiveness of an educational technology such as the Music Paint Machine couldn’t be solely based on testing learning outcomes. It also needs to focus on the way the technology has an impact on the different components of instruction (learner, teacher, the interaction between both, the musical material or learning content). It became clear that an educational technology is much more than an independent variable that is inserted in the instructional process in order to induce an amplicative effect on the basis of its features.
Before we describe the experiments with the Music Paint Machine, we present a theoretical investigation of the relationship between musician and musical instrument (Chapter 7). We argue that an optimal relationship between the two involves the incorporation of the instrument. In other words, musician and musical instrument merge into a unity. We unravel the processes that underlie such an optimal relationship. This way, we were able to define the basic components of an embodied interaction with music. These components are: an optimal experience, direct perception and skill-based playing. We define an optimal or “flow” experience, in which a musician is completely absorbed in the act of playing music, as the combination of the feeling of presence and a positive emotional condition. The concept of presence originates in the research on computer generated virtual reality. In this thesis, we elaborate the concept within the embodied music cognition framework.
The theoretical investigation of the relationship between musician and musical instrument provided the basis for the concept of the Music Paint Machine but also for the experiments with the system.
The first experiment was a user study that probed the participants’ personal experience with the system and their evaluation of its didactic potential (Chapter 8). We measured flow with the Flow State Scale (Jackson and Eklund, 2004) and presence with an in house designed questionnaire. This way, the measurement of presence was introduced in music research. Up until now, the concept of presence was only briefly referred to in the literature on music research (e.g. Leman, 2007). The results of this experiment suggested the system’s potential to elicit a flow experience. They also provided an empirical validation of the intrinsic relationship between flow and presence (Chapter 9). With regard to the evaluation of the didactic potential, the results indicated that both students and teachers value the system as a way to support musical development and learning how to play a musical instrument.
The second experiment was a longitudinal study in which children learned to play the clarineo with the instructional support of the Music Paint Machine (Chapter 10). In this study good practices were developed and the amplicative impact (focus on outcomes) of the system was tested on the basis of a non-equivalent control groups design. During nine months, twelve children received instruction on a weekly one-hour basis. Six children (two groups of three) received instruction with the Music Paint Machine, six children (two groups of three) received instruction without the system. The learning content was the same for the control and experimental groups. We investigated whether using the Music Paint Machine has an effect on the developmental music aptitude of the children. Music aptitude is a child’s potential for music achievement that can develop during the first nine years on the basis of the child’s musical experiences. According to Gordon (1986), the degree to which music aptitude develops also depends on the quality of formal music instruction. In our longitudinal study, we hypothesized that using an interactive music system that combines visual feedback and movement possibly contributes to the quality of instruction and thereby to the development of the children’s music aptitude. We did not find a significant difference between the control and treatment groups with regard to the dependent variable, music aptitude. However, this study made clear that it is necessary to focus on the transformative impact (focus on processes) of technology. On the basis of the weekly lessons, it was experienced by the researcher-teacher that using the system has a deep impact on the classroom events during instruction. It became clear that integrating the system in instruction changes the way the teacher and the students relate to each other and the way instruction is organized. We believe that investigating these transformations needs to be part of music educational technology research.
In Part 4 of this thesis, we first describe the contributions of the research tat was presented in this thesis. Next, in the general discussion, we critically discuss different aspects of the research. This is followed by an outlook on future research and by a final conclusion.