Navigation – Plan du site

AccueilNuméros3VariaMusical Visual Vernacular. How th...

Varia

Musical Visual Vernacular. How the deaf people translate the sound vibrations into the sign language: An example from Italy

Anna Ambra Zaghetto
p. 273-298

Résumés

Le Visual Vernacular musical (VVm) est un nouveau style expressif répandu parmi les sourds italiens depuis 2008. À présent, on connaît seulement deux exemples du style VVm, représentés par deux travaux artistiques italiens. L’analyse des enregistrements vidéo des performances en VVm explique le procès d’évolution et de structuration/organisation de ce style expressif. Les données, recueillies entre 2010 et 2011, nous montrent que les performances en VVm se basent sur la connexion entre deux domaines sémantiques différents : le domaine linguistique (tiré de la langue signée) et le domaine musical/sonore. Le résultat des performances en VVm est fortement iconique et peut être défini comme une suite d’images (construites sur une base linguistique) qui sont également compréhensibles par un public, qu’il soit sourd ou entendant. Dans cette perspective, les signes linguistiques et musicaux sont combinés pour créer un nouvel espace sémantique où de nouvelles significations surgissent d’un substrat commun.

Haut de page

Texte intégral

Acknowledgements: This research is a part of the Thesis of A.A. Zaghetto discussed during July 2011 (University of Milano Bicocca) to obtain the degree in Anthropology (Dr.). The author wish to be grateful to: Prof. G. Iannàccaro, Prof. R. Carchio and Prof. V. Matera (University of Milano Bicocca); ENS Milano (Dr. V. Castelnuovo; A. De Pieri; M. Pasquotto; all deaf people of ENS); Ass. Maschera Viva, Torino (L. Daniele; L. Di Gioia); MAMbo, Bologna (F. Grilli art group, all deaf and hearing people); Dr. A. Mongia; Prof. G. Zaghetto; Prof. A. Gemelli; Dr. G. Frega.

Introduction

1The term Deaf Culture (DC) indicates a form of oral culture typical of a deaf community. DC describes the social beliefs, behaviors, art, literary traditions, history, values and shared institutions of communities that are affected by deafness and use sign languages to communicate (Lane, 2005). The definition of community itself is problematic and the term has been used in different ways (Senghas and Monaghan, 2002). Here, I adopt the term “Community” indicating the deaf people clustered as a social group and characterized by the use of the sign language (Senghas & Monaghan, 2002; Volterra, 1987/2004), according to the definition of Hymes and Duranti about the linguistic community (Hymes, 1971; Duranti, 2000).

2Deaf Communities develop an oral tradition based on their linguistic channel (sign language) that is a visual-gestual system (Stokoe, 1980; Sandler & Lillo-Martin, 2001; Volterra, 1987/2004). In Italy an Italian Deaf Community (IDC) is present (Volterra, 1987/2004). The members of IDC share, on the one hand a sign language system called Lingua Italiana dei Segni – LIS (Volterra, 1987/2004; Geraci et al., 2011), and on the other hand a common minority culture, Italian Deaf Culture (Zaccalà, 1997).

3Italian Deaf Culture is handed down through deaf generations and it is characterized by different demonstrations of “art” expression, like styles in LIS (Zuccalà, 1997; Volterra, 1987/2004; Russo-Cardona & Volterra, 2007). Today, six LIS styles are known (Zaghetto, 2012), and musical Visual Vernacular (VVm) is the more recent acquired style among the members of the IDC. VVm spreads in Italy from 2008; till now, only two examples are known: The Third Conversation (2008) and The Conversation (2010); both examples are two work-art created by the Italian artist F. Grilli (Grilli et al., 2011). The VVm technique is directly derived from Visual Vernacular (VV).

4VV is a new sign language style developed in Italy during 2005 (Zaghetto, 2012); it allows to realize 3D-narrations focused on details as Italian deaf people claim (data from direct interviews to deaf people; 2010/2011). VV compositions are comparable to mute-movies or pictures-sequences. The style is based on the use of classifiers (CL), facial expressions (FE) and embodiment technique (EM) supported by a specific motor plan. The motor plan leads to the creation of a peculiar rhythm of sign articulation, usually fast, that characterized every VV performance. The VV fast rhythm imposes the presentation at most of two ore three subjects for each 3D-narration.

5VVm develops from VV by the selection of the use of CLs and FE during the performances (Zaghetto, 2012). Music accompaniment is an essential element of the VVm performances, and the CL-sequences are always performed on music rhythm. In the two Grilli’s works a deaf performer directly interacts with the music vibration through the tactile perception: the performer translates in LIS his personal sensation of both sound vibration and artistic idea. The final performance results highly figurative and iconic, equally understandable by the deaf or the hearing audience.

6Data derived from direct interviews had shown that VVm performances are not improvisations: the deaf performer gets in contact with vibrations through an amplifier system (The Third Conversation) or by the wood of a sculpture (The Conversation), and he learns to understand the “quality” of the sound vibrations (direction, intensity, dinamycs) to transpose in a linguistic dominion his personal sensations. Still, VVm is almost unknown among the IDC members and it is not really defined in technical and performative rules.

7Data of this research were collected between 2010 and 2011 in four cities of the North Italy: Milano, Bologna, Torino and Verona. Data derived from video-recordings and from direct interviews. All the deaf people involved in the project are native signers (LIS), with profound deafness and without residual hearing, and memebers of the Italian Deaf Community (IDC). All of them had expressed formal approval to take part in this research, to be video-recorded during performances and direct interviews, and to use part of these data for pubblications.

8I propose a specific analysis of the work-art The Conversation (2010) by F. Grilli to describe the VVm structure and the performance organization.

Analysis: The Conversation (2010)

9The Third Conversation (2008) and The Conversation (2010) by F. Grilli are two works-art included in a cycle constituted by five works in which the artist explores new communication modalities together with a continuous relationship between body and perception (www.drodesera.it). In The Third Conversation the deaf performer receives the sound vibrations from an amplifier system (AS; pre-recorded score performance. In The Conversation the deaf performer gets in contact with the sound vibrations through the wood of a sculpture located in the center of the performative space; in this case, the AS is located below the sculpture and the music derives from the extemporaneous score execution by the musicians (see below).

Basic Informations

10The Conversation is the last of five works that whole constitute the art cycle realized by F. Grilli (Grilli et al., 2011). The artistic idea derives from the Odissea by Omero. The choosen passage from the Odissea is related to the encounter between Ulysses and the mermaids in the middle of the sea.

[…] In this passage Ulysses decides to protect the ear of his men (crew) from the mermaids voice; in this way, men momentarily become deaf. Instead, he decides to be tied to the watercraft mast without ear protection because he wants to listen the voice of the mermaids. This is the basic image of the work-art, but in The Conversation the situation is inverted: the deaf performer becomes the only deaf subject in a “sea” of hearing people […] (direct interview to F. Grilli; December 2010).

11The performative space had been organized on this idea, and it appears a closed space divided into two parts in which the performers (the deaf signer, N. Della Maggiora; and the two musicians, L. Bernard and F. Guerri) and the audience come into contact (Fig. 1). The Ulysse’s watercraft is represented by the wood sculpture (the watercraft prow overturn) located in the center of the performative space; the deaf performer must be barefeet on the sculpture under which an AS is present (Fig. 2). The two instruments, cello and double-bass, are located on the left side and on the right side of the sculpture. Around the sculpture many black balloons are present and the audience must hold the balloons between their hands to percive the sound vibrations like the deaf performer (tactile modality). The artist considers the music score as an integral part of the work-art; for this reason I can not show any parts of the score in this paper.

VVm performance structure

12The VVm is based on the specific use of CL and FE calibrated on the music rhythm and in The Conversation the deaf performer chooses seven CLs (basic hand shapes from LIS) from the typology grounded by Mazzoni (2008): CL5, CLB, CL1, CLO, CL5#, CLC and CLF (Tab. 1). The deaf signer modifies or adapts the basic CLs shapes (hand configuration and articulation; Fig. 3) in relation to expression necessities, and he creates specific CLs combinations (CL-sequences) for each part of the VVm performance.

13Each part of the resulting VVm performance correspondes to the eight parts of the music score (Tab. 2; Fig. 3), and the organization directly derives from the perception of both the sound vibrations and the music dynamics. Each CL-sequence is articulated (articulation rhythm, AR) in the respect of the score execution timing (Tab. 2). Moreover, the choice of the specific combinations of CLs is related to the “nature” of each correspondig score part (music dynamics) and the corresponding artistic idea (Tab. 2).

14The linguistic elements of every sign language are articulated inside a space located over the signer; this space is called neutral space, and in VVm style each CL-sequence is localized in a peculiar part of this space. The CL-sequences localization appears to be related to the pitch of the sound vibrations perceived by the deaf performer. Therefore, three main axis of articulation can be defined inside the neutral space (Fig. 4). I have called these three axis: head axis (H axis), sternum axis (S axis) and pelvis axis (P axis).

[…] I learn to understand which is the direction [spatiality] of the vibrations […] (direct interview to N. Della Maggiora; December 2010).

15Both the spatialization and the localization of linguistic elements (CLs) are organized on the music parameters (MP) like intensity, frequency, pitch and rhythm of the vibrations perceived by the deaf performer. MP can be consider as parameters of spatialization for the CL-sequences inside the neutral space. To clarify this fact I propose an entire transcription of the VVm performance of The Conversation (Fig. 5).

16The VVm transcription is usefull to show: 1) hoe the deaf performer organizes (spatialization) the CL-sequences around the three main performative axis (P, S, H); 2) how the CLs localization is exactly related to the pitch and the intensity of the sounds percived by the performer; 3) how a specific combination of CLs leads to the carachterization of each VVm performance section (Tab. 2). The VVm transcription derived from the direct analysis of the video-recorded VVm performance during the tryouts at MAMbo (Bologna, Italy; December 2010).

17Section A Moderato: the section is characterized by the use/connection of the three CLs: CL5, CLB and CL5#. The VVm performance start on t=20’’ and CL5 is the first CL used in the performance. The CL-sequences are articulated in the lower area of the neutral space till t=1’23’’ (the CLs localization corresponds to the low frequencies percived by the deaf performer. From t=1’24’’ to t=4’06’’ the CLs articulation is moved in the area around S axis and H axis in relation to the modified intensity and frequency of the sound vibrations. The AR corresponds to the score time of execution (Moderato).

18Section B Adagio: in this section new CLs are introduced: CLO, CL1; these CLs are connected to CLB nd CL5 introduced in the first section of the performance. In this section the CLB hand shape configuration is modified and the CL is articulated in the upper part of the neutral space, around the H axis; CLB appears “freezed” in its position during the articulation of the other elements. The CLB modified can be considered a single characteristic elements of the B part (and of some other parts of the performance; see below), and it is usually related to a strong intensity or quality of the sound vibrations. The Section B is connected to the Section C on t=5’06’’. In this section the AR is very slow in relation to the slow music rhythm (Adagio).

19Section C Adagio: this section is carachterized by the use of six CLs: CL1, CL5, CL5#, CLC, CLO and CLB. Cl1 is the main CL of the section and it is connected to the other CLs through repetitive structures related to the percussive rhythm of the music score. Here, the AR is slow in agreement with the imposed time of socre execution (Adagio).

20Section D Moderato/Vivace: in this section six CLs are used: CL1, CLO, CL5, CLB, CL5# and CL5. The two characteristic CLs of the first part of Section D are CL1 and CLO (till t=6’11’’); instead, the second part of the section is characterized by the use of CL5, CLB and CL5# (till t=7’07’’). The AR is moderately fast in relation to the music dynamic (Moderato – Vivace).

21Section E Moderato/Vivace: the use of CLB, CL5 and CL5#in connection to CLC and CLF characterized the organization of the Section e. The CL-sequences are articulated in the upper part of the neutral space (between S axis and H axis). The music rhythm is fast (Moderato/Vivace), and it influences the AR as in the previous parts of the performance. The section ends on t= 8’20’’

22Section F Adagio: in Section F four main CLs are used: CLB, CL5, CL5# and CLF. The modified CLB is proposed again, and it is articulated in the upper part of the neutral space (H axis) on t=8’21’’, t=8’48’’, t=9’03’’, t=9’19’’; it becomes a connection element between CL-sequences besed on the use of CLB, CL5 and CL5# articulated around the S axis. At the end of the F part (on t=9’26’’) a new CLs combination is proposed: CLAs + CL5. This combination will be proposed in the next parts of the performance in the alternative form CLAs + CLB. A slow music rhythm carachterized the Section F (Adagio), and the percussive structure of the score is presented again.

23Section G Veloce: the Section G starts with the presentation of the CLB single element that underlines, in numerous points, the percussive structure and carachter of the section (on t=9’28’’; t=9’34’’; t=9’51’’; t=9’58’’). The section can be devided into two part: the first part is carachterized by the articulation of the main CLs between S axis and H axis, and it ends on t=9’51’’; the second part is carachterized by the articulation of the CL-sequences between P and S axis. Both CL-sequences, CLAs + CL5 and CLAs + CLB, are used in this section, and these are organized in repetitive combinations in relation to the percussive structure of the music score. The AR derives from the score dynamics (Veloce).

24Section H — Adagio: the Section H is the last part of the entire performance. In this section all the CL-sequences used before are presented again, and the articulation/ localization inside the neutral space is related to the MP changes. In relation to the intensity and the pitch of the vibrations perceived by the deaf performer, the seven main CLs appears to be localized around all the three axis (H, S, P). This section appears more long then the other seven parts. CL-sequences are organized on a “music pedal”: the CLB modified element is proposed again (on t=10’05’’; t=11’56’’; t=12’45’’; t=12’51’’; t=12’59’’), and it becomes the percussive elements that underlines the short CLs combinations like CLAs + CL5 and CLAs + CLB. The end of the entire performance is realized articulating the modified CLB around the P axis corresponding to the low intensity and frequency of the sound vibrations perceived by the deaf performer. In spite of the slow music rhythm (Adagio) the AR of this section is more fast then the other sections that show the same rhythm.

Observations

25The visual result of the entire VVm performance of The Conversation is highly iconic and it appears to be understandable both by the deaf audience and the hearing audience (without a specific linguistic competence in LIS).

26The CLs used in the VVm performance are combined to create a sequence of images related, on the one hand, to the artistic idea of Grilli (see the list of images quoted in the text below), and on the other hand to the personal sound/vibration sensations of the deaf performer.

[…] waves — a big watercraft in the middle of the see — staggering on the waves — water — waves against the watercraft — calling mermaids — waves against the watercraft — fishes — looking mermaids below the water — wind trough hair — watercraft — waves […] (direct interview to the deaf performer, N. Della Maggiora; December 2010).

27Using a sign language system (LIS) the deaf performer transposes the artistic idea and his tactile sensations in a visual domain (in agreement with the visual-gestual channel of the sign language). In this process the performer modifies the hand shape configurations of each CL used to create new linguistic elements more iconic (some phonological changes occur; Zaghetto, data not shown). In fact, in spite of the linguistic structure, the final VVm result appears to be organized/ structurated in sequences of images.

28To create this sequence of images (linguistically structurated) the deaf performer learns to recognize the ‘quality’ of the sounds through the tactile channel: the performer speaks about a ‘direction’ of the sound vibrations, a changeable intensity (high or low), a dynamic of whole sounds (related to the eight different sections of the music score that he is able to distinguish), a ‘quality’ of sounds (data derived from direct interview to N. Della Maggiora). He learns to discriminate each of these parameters (MP) touching, time and time again, the surface of the wood sculpture with his hands or his feet; in this process the sound/ vibration input acquires a spatialization on the surface of the performer’s hands/ feet: as he asserts the input derived from high intensity vibrations is perceived/ localized in the anterior part of his hands/feet surface; instead, the input derived from low intensity vibrations is perceived/localized in the back part of his hands/ feet surface (Fig. 6a). The acquired tactile spatialization is transposed in a linguistic domain, and the localization of the CLs inside the neutral space reflects exactly the sound inputs ‘quality’ derived acquired through the tactile sensations (Fig. 6b; Fig. 4): the tactile input categorized as intense and ‘high’ is translated in the dynamic articulation of the CL-sequences in the upper part of the neutral space (around H and S axis); instead, the tactile input categorized as few intense and ‘low’ is translated in the dynamic articulation of the CL-sequences in the lower part of the neutral space (around P axis).

Discussion

29The relationship between music and sign language is difficult to address (Rossi, 2001; La Via, 2006; Mithen, 2005). If we consider music as something of heard, harmony based (with specific rules of sound organization), carachterized by the presence of a specific notation (Levitin, 2006), it is difficult to find, and to understand, a new perspective in which consider the possibility “to hear” music in a deaf condition.

30Deafness is a condition wherein the ability to detect certain frequencies of sound is completely or partially impaired (Wright, 1971). The hearing impairment is categorized by its type (conductive, sensorineural, or both), by its severity, and by the age of onset (Wright, 1971). Furthermore, a hearing impairment may exists in only one ear (unilateral) or in both ears (bilateral). This pathology is usually considered an unfitting condition to music/sound perception (Carré, 1997). However, the deaf people ‘hear’ sounds in a different way, through the sense of touch, perceiving vibrations (sound vibrations derived from a music source).

31The VVm is a new artistic technique in sign language acquired among the members of the Italian Deaf Community (IDC) during the last four years. Today, only two VVm’s examples are known: The Third Conversation (2008) and The Conversation (2010) by Grilli. In both artistic examples the deaf performer directly interacts with the music/sound vibrations: the tactile sensations derived from the perceptions of sounds are transposed in LIS together with the images derived from the artistic idea. This process represents something of new in the perspective of a Deaf Culture.

32In fact, usually, in the ‘oral educational method’ the deaf subjects are introduced to music during rehabilitation courses to spoken language: the music rhythm and the percussive stimuli induce the deaf subject to comprehend the words articulation rhythm during the oral communication. In this case, music becomes functional to the word’s pronunciation and to the sentence’s intonation/ inflection (Rossi, 2001; Zaghetto, data not shown).

33In situations different to the educational system, it is possibile to find deaf subjects really interested to ‘explore’ the music world through the participation to dance meeting/lessons: on the one hand, the deaf subjects learn to dance looking to the visual sequences of the dance steps performed by the hearing people; on the other hand, the deaf subjects ‘hear’ music putting their hands directly on the amplifier system or ‘listening’ to the tactile sensations produced in their abdomen by the low frequencies (Zaghetto, data not shown).

34In a VVm performance the direct interaction between the deaf performer and the sound vibrations is present, but a new fact happen: the transposition of the tactile sensations in the linguistic domain (sign language system).

35Music is structured on seven main basic elements, the music parameters (MP): the sound pitch, the rhythm, the music dynamics (slow or fast), the melody, the tone color, the intensity, the echo back as the distance perception from the sound source (Levitin, 2006). The organization of these elements leads to the creation of music compositions like the score performed in The Conversation. The VVm structure is created on the perfect coordination of both linguistic and music elements. In fact, the CLs from LIS are modified/adapted to reproduce both the artistic idea and the tactile sensation derived from the sound vibrations. The CLs localization inside the neutral space of articulation reproduces the MP perceived by the deaf performer troguht a spatialization of the tactile input (Fig. 4; Fig. 5; Fig. 6b).

36In The Conversation two players (cello and double-bass) performe the original score. From the spectrogram analysis (Audacity, Mac version) it is possible to establish the frequencies range perceived by the deaf performer: 8-350 Hz, low frequences(Fig. 7). Moreover, in many points of the music score the two instruments are overlapped and this fact increases the intensity of the vibrations perceived by the performer. As I have shown, the signer is able to feel and to discriminate the sounds frequences through the tactile modality (Fig. 6a and 6b); moreover, the signer is able to feel the music dynamics (rhythm changes) and to transpose these changes in the linguistic domain (AR, fast or slow). The deaf performer creates CL-sequences localized/articulated around three main axis inside the neutral space (H axis; S axis; P axis), and the localization corresponds to the sound pitch/intensity perceived (Fig. 6). The echo back (EB), another MP, increases the tactile sensations of the deaf performer: EB contributes to better define the ‘direction’ of the sound vibrations (direct interview to the deaf performer, N. Della Maggiora; December 2010). The four MP described become spatial localizers (SL) of the tactile sensations derived from the sound vibrations perception. The spatial localization is not related to the distance between the music source and the deaf performer because, in the VVm performance analysed, the medium of the final tactile perception is the AS located under the wood sculpture.

37The tactile input becomes the basic element to construct a sham/internal map (SM) of the sound vibrations: the deaf performer uses the MP to define, in his mind, the spatial localization of the whole vibrations perceived; then, the mental spatialization is transposed in a dynamic process of sign articulation (CL-sequences) inside the neutral space (Fig. 4; Fig. 8); the final spatialization of each CL-sequence reflects the MP perceived by the signer during the entire performance.

38In this process two concurrent events happen: on the one hand, the definition of a mental SM of the whole tactile sensations; on the other hand, the transposition of the SM defined into a linguistic space, the neutral space of signs articulation (Fig. 8). The double process characterizes the new VVm style and it leads to a highly iconic result understandable both by a deaf or a hearing audience.

39In fact, the VVm performance analyzed is based on the specific use of seven CLs from LIS repertory, but the basic hand shape of each CL is modified to obtain the higly iconic result described. The new hand shape configurations are combined in CL-sequences that appear more ‘visual’ then the usual LIS linguistic elments, and this fact leads to the creation of image-sequences (I-sequences) directly related to the MP, and these appear to be detached from the basic linguistic rules. So, linguistically, something of new happen, because the deaf performer is able to create a new way of communicatin understandable by different audiences and, from this perspective, a new semiological space can be defined.

In English, the term ‘space’ has had two major usages: denoting time or duration and denoting area or extension. In the former sense, the word has referred to short and long lapses of time between two definite events, to amounts of time contained in specified periods, to amounts of time somehow determined, or to any period or interval of time. In the latter sense, the term has generally denoted linear distance or interval between two or more points or objects, superficial extent or area, area sufficient for some purpose, extension from point in all directions, room available for something, and, finally, “continuous, unbounded, or unlimited extension in every direction, regarded as void of matter, or without reference to this” as well as “the immeasurable expanse in which the solar and stellar systems, nebulæ, etc., are situated; the stellar depths”, that is, the universe. […] in the context of semiosis, ‘space’ is indeed tied to the real existence of the participants of semiosis, both in terms of actual reality and phenomenal reality. [...] (Ojala, 2009: 198–199).

40In the VVm performance the ‘space’ notion is adapted to a linguistic transposition of the tactile input into a visual domain. During the transposition process two events of spatialization happen: the first event is related to the mental spatialization of the tactil inputs (vibrations) inside the performer’s mind; the second event is related to the transposition of the first spatialization inside the linguistic domain; in other words, the deaf performer translates the tactile input in linguistic elements sequences (CL-sequences) inside a ‘visual space’, the neutral space (Fig. 8).

41The classical musical space defined by Wellek forecasts a tripartition of the hearing space: 1) hearing space; 2) tonal space; 3) musical space (Ojala, 2009: 376– 377). The tripartition is based on the perception of pitch, spectrum, volume and other sound parameters, and Wellek describes, interestingly, the ‘tonal space’ as a three-dimensional schema (in the mind of the listener), and the ‘musical space’ as a subjective space, associated with personal feelings. The Wellek’s tripartition can be applied to a different modality of perception, like in deaf subjects.

[…] musical space as conceptual space in which the dynamic sound object structures are associated with other experiences thus “constituting a complex combined result” of a feeling-like subjective experience. […] (Ojala, 2009: 377).

42From a hearing perspective, the perception of pitch, spectrum and volume, leads to the elementary ‘spectral hearing’; however, from a deaf perspective, the same perception process leads to define a ‘tactile spectrum’ based on vibrations then translated in a personal internal SM (tactile spatialization; Fig. 6a and 6b). Therefore, hearing and deaf subjects define their internal SM in a different way: hearing subjects start from auditory stimuli; instead, deaf subjects start from tactile stimuli (vibrations). However, the source of the organization of the whole stimuli is shared by the two groups of subjects: music. Moreover, the two groups share also the dynamic of the spatial-embodiment (spatialization), but the final result is communicate through different channels: auditory channel for hearing people, and visual chiannel for deaf people.

43The mental imaginary derived from a sound sensation differs from a subject to another. In this respect, the participants in musical processes are indeed in a relationship with a world via their own situation (Tarasti, 1998b, 45). A hearing subject can respond to music stimulation performing a dance; instead, a deaf subject can transpose sensations into a visual-gestual linguistic channel, like in the VVm performance in The Conversation.

[…] Reception of musical communication must be just as multi-faceted and multi-dimensional — open to the point that the number of dimensions and facets may be constantly variable. […] the abstraction of sensory information from particular instances to a generalization of a sound object with an identity and “object constancy” capable of withstanding object manipulation (e.g. transposition, augmentation and diminution, inversion, even retroversion, fragmentation, and concatenation, etc.) is possible through object […], brought about by different kinds of habits of perception (e.g. stream segregation and auditory scene analysis, accustomed stylistic characteristics etc.). […] the perceptual construction of a sound object means the construction of an instance in dynamically established conceptual spaces, their domains, quality dimensions as well categories and their prototypes, which in turn depend on their conceivable practical bearings, according to the pragmatic maxim. […] In addition to the dynamic complexity of particular sound objects, the kaleidoscopic complexity is a result, […] , of the complexity of combinations of a number of sound objects, which can be related to each other in complex ways by being successive, simultaneous, or nested. […] (Ojala, 2009: 390–392).

44From a deaf perspective, the classical notion of music space is modified and it becomes possible to speak about a ‘vibration space’ percived and translated in linguistic sequences (CL-sequences): vibrations (like sounds) would correspond logically to the construction of object representation. A new semiological space is created/organized in The Conversation: the deaf performer connects linguistic elements to a real (tactile) perception of sounds, and according to Ojala (2009), action and perception are two aspects of musical semiosis, and it is impossible to think the music concept separated to the concept of the interaction with sounds. In the new semiological space the input is tactile and the output is visual (Fig. 9); the visual result (CL-sequences= I-sequences) appears understandable both by hearing and deaf audience, and the spatialization appear double and defined as a ‘function’ (Fig. 9). A new semantics has been created by the intersection of music, tactile sensations and language (sign language): the deaf performer becomes the bridge between two distinct domains, the linguistic domain and the music domain, and the tactile perception becomes the key element to comprehend the organization of a VVm performance.

Conclusion

45In this paper I describe the VVm style, a new expression technique known among Italian deaf people from 2008. Still, rules of VVm technique are not defined; however, the performances analysis had shown that VVm technique has an internal organization: linguistic elements (CLs) are organized and connected to the music elements (sound vibrations percived through the sense of touch). VVm can be consider a style in sign language (LIS) in which two different dominios (linguistic and musical) converge to develop a new semiological space of meanings. The VVm higly iconic (visual) result creates the sensation of a succession of images equally comprehensible by different types of audience (in this case: from observations, a deaf or hearing audience). The VVm performance represents an alternative way of communications from a deaf perspective, and a new style of expression in sign language based on the interaction of different semiological dominions.

Haut de page

Bibliographie

CarrÉ, Alain (1997), Quando la musica parla al silenzio. Proposta di musicoterapia per bambini audiolesi, Roma, Magi Edizioni.

Duranti, A. (2000), Antropologia del Linguaggio, Roma, Maltemi.

Geraci, Carlo, Battaglia, Katia, Cardinaletti, Anna, Cecchetto, Carlo, Donati, Caterina, Giudice, Serena & Mereghetti, Emiliano (2011), “The LIS Corpus Project: A discussion of sociolinguistic variation in the lexicon,” Sign Language Studies, 11(4), pp. 528–571.

Grilli, Francesca, Pietroiusti, Cesare & Riva, Caterina (2011), The Conversation, Produzioni Nero.

Hymes, Dell (1971), On Communicative Competence, Philadelphia, University of Pennsylvania Press.

Lane, Harlan (2005), “Ethnicity, Ethics and the Deaf-World,” J. Deaf Stud. Deaf Educ., 10(3), pp. 291–310.

La Via, Stefano (2006), Poesia per Musica e Musica per Poesia – Dai trovatori a Paolo Conte, Roma, Carocci Editore.

Levitin, Daniel J. (2006), This is Your Brainon Music – The Science of a Human Obsession, New York, Dutton Adult, Penguin.

Mazzoni, Laura (2008), Impersonamento nella Lingua dei Segni Italiana, Pisa, Edizioni Plus – Pisa University Press.

Mithen, Stephen (2005), The singing Neanderthals – The Origins of Music, Language, Mind and Body, London, Weidenfeld & Nicolson, The Orion Publishing Group Ltd.

Murray Schafer, Raymond (1977), The Tuning of the World, Toronto, McClelland and Stewart Limited.

Ojala, Juha (2009), Space in Musical Semiosis – An Abductive Theory of the Musical Composition Process, Imatra, Finland, International Semiotic Institute (ISI).

Rossi, Mario (2001), Dal Canto alla Parola, Milano, Franco Angeli Editore.

Russo-Cardona, Tommaso & Volterra, Virginia (2007), Le Lingue dei Segni – Storia e Semiotica, Roma, Carocci Editore.

Sandler, Wendy & Lillo-Martin, Diane (2001), “Natural Sign Languages,” in Aronoff, Mark & Rees-Miller, Janie (eds), Handbook of Linguistics, New York, Academic Press, pp. 533–562.

Senghas, Richard J. & Monaghan, Leila (2002), “Signs of their time: Deaf communities and the culture of language,” Annu. Rev. Anthropol., 31, pp. 69–97.

Stokoe, William (1980), “Sign language structure,” Annu. Rev. Anthropol., 9, pp. 365–390.

Tarasti, Eero (1998b), Signs as acts and events. An essay on musical situations, in Stefani, Gino, Tarasti, Eero and Marconi, Luca (eds), Musical signification. Between rhetoric and pragmatics. Proceedings of the 5th international congress on musical signification, pp. 39–62. See Tarasti 1996b. Imatra & Bologna : International Semiotics Institute & CLUEB.

Volterra, Virginia (1987/2004), La lingua dei segni italiana : La comunicazione visivo-gestuale dei sordi, Bologna, Il Mulino.

Wright, Mary Ingle (1971), The pathology of deafness – an Introduction, Manchester, The University Press, University of Manchester.

Zaghetto, Anna Ambra (2012), Italia Deaf Literature: new perspectives, in Conference Proceedings, VISUALIST 2012, (Istanbul, Turkey), 2002 March 8; I: 334340.

Zuccalà, Amir (1997), Cultura del Gesto e Cultura della Parola – Viaggio antropologico nel mondo dei sordi, Roma, Maltemi Editore.

Haut de page

Annexe

Tab. 1 Classifiers used in the VVm performance (The Conversation)

CL1

Thin and long shape; to indicate cylindrical and long objects

Animals (snake), humans (male and female), vehicles (rocket), different instruments (knife)

CL5

To indicate spherical objects; sometimes other shape of objects

Animals (elephant, …), vehicles (hot-air balloon), spherical objects (balloon, apple, …), robots, big objects (houses, churches, …)

CL5#

To indicate different object’s shape; to indicate grasping

People, sculpture, trees, columns, body’s parts, grasping of different objects (sheet, plate, …)

CLB

One of the most used CL; to indicate the shape of different objects

Vehicles (train, bus, …), objects (book, table, sheet, …), part of human body (foot, hand, …), instruments (large brush, …), surfaces (virtucal, flat, …)

CLC

To indicate cylindric shape or objects like holders; raising objects

Tubes, piles, field-glasses, bottles, glasses, legs, cups

CLF

Small objects

Round stones, buttons, coins, pastries, cups, eyes

CLO

Poor used; to indicate cylindrical or spherical objects, tridimensional

Glasses, robots

CLs typology from Mazzoni (2008). In the first column: the list of the basic CLs choosen for the VVm performance in The Conversation of F. Grilli. Second column: description of the references related to each CL type. Third column: examples of real objects indicated/described by the CL typology.

Tab. 2 VVm performance organization

SECTION

TIME

MUSIC DYNAMICS

CL-SEQUENCES

A

0’’ →

4’06’’

Moderato

iCL5/CLB/CL5#/CLB+CL5#/fCL5

B

4’07’’ →

5’09’’

Adagio (melodico)

iCLB/CL5/CL5#/fCL5

C

5’10’’ →

5’59’’

Adagio (melodico)

iCL1/CLB/CL5#/fCL5

D

6’00’ →

6’52’’

Moderato Vivace

iCL1/CLO/CL5/CLB/CL5#/fCL5

E

6’53’’ →

8’19’’

Moderato – Vivace (incalzando)

iCLB /CL5 r/CLB /CL5#/CLF/

var va var

CLC/CLAs+CL5/fCL5

F

8’20’ →

9’28’’

Adagio (melodico)

iCLB/CL5/CLF/CL5#/fCL5

G

9’29’’ →

10’19’’

Veloce (ritmato)

iCLB/CLAs/CL5#/CLAs+CL5/

CL1/CLAs+CLB/CL5/fCL5

H

10’20’’ →

13’44’’

Adagio (melodico)

iCLB/CL5/CL1/CLAs+CL5/

CLO+CL5/CLO/CL5#/fCL5

First column: the eight sections of the VVm performance and of the music score (capital letters, red color, from A to H). Second column: time lenght of each section. Third column: score music dynamics. Fourth column: CLsequences created by the deaf performer for each section of the VVm performance.

Fig. 1 The performative space in The Conversation (2010)

Fig. 1 The performative space in The Conversation (2010)

The performative space is divided into two part: A, here the wood sculpture is located in the center of the space; moreover, the performers (the deaf performer and the music players) stay in this psace during the entire performance; B, the space where the audience watch the performance.

Fig. 2 The interaction between the deaf performer and the sound vibrations in The Conversation (2010)

Fig. 2 The interaction between the deaf performer and the sound vibrations in The Conversation (2010)

The wood sculpture is located in the center of the performative space (brown lines); under the sculpture two amplifiers (amplifier system, AS) are located (blu color); the sound vibrations from the AS hit the sculpture (red arrow), and the deaf performer (barefeet on the top of the sculpture) percives the vibrations through the wood (tacitle modality).

Fig. 5 The VVm performance transcription

Fig. 5 The VVm performance transcription

The transcription derives from the direct analysis of the video-registration of the VVm performance (December 2010, MAMbo, Bologna, Italy). The transcription shows the relation between CL-sequences articulation (localization inside the neutral space of sign articulation) around the three main body axis and the music score timing (music performance). In this transcription does not appear the music score because the artist (F. Grilli) considers the score as integral part of the artistic work and it can not be reproduce.

Fig. 3 CL-sequences used in the VVm performance (The Conversation)

Fig. 3 CL-sequences used in the VVm performance (The Conversation)

In the figure: on the right of the graphic, the CLs list chosen by the deaf performer to realiza the VVm performance; the graphic shows the combinations of different CLs to carachterize the typology (“nature”) of each part of the performance (capital letters, from A to H) corresponding to the eight parts of the music score. CLB and CL5 – CL5# appear the most used CLs; in particular the CLB appears to be present in all parts of the performance.

Fig. 4 The three body axis of the signs articulation during the VVm performance

Fig. 4 The three body axis of the signs articulation during the VVm performance

In the figure: the three main axis are indicated (head axis H axis, sternum axis S axis, and pelvis axis P axis) in relation to a CL-sequence (example) takes from the transcription of the entire VVm performance of The Conversation. The CLs (CL5, CL5#, and CLB) are articulated around the three body axis corresponding to the pitch, the intensity and the dynamica of the sound vibrations (music example below the VVm transcription; music dynamnic is indicated with green arrows).

Fig. 6a The sound vibration spatialization on the performer’s hands/feet surface during a VVm performance

Fig. 6a The sound vibration spatialization on the performer’s hands/feet surface during a VVm performance

The sound input derived from high intensity vibrations is perceived/localized in the anterior part of his hands/feet surface; instead, the sound input derived from low intensity vibrations is perceived/localized in the back part of his hands/feet surface.

Fig. 6b The sound vibration spatialization on the performer’s hands/feet surface during a VVm performance

Fig. 6b The sound vibration spatialization on the performer’s hands/feet surface during a VVm performance

Parallelism between the spatialization of the sound inputs on the performer’s hands/ feet surface and the related CLs articulation around the three main axis inside the neutral space.

Fig. 7 The sound frequencies analysis during the score performance in The Conversation

Fig. 7 The sound frequencies analysis during the score performance in The Conversation

The analysis of the sound trace using Audacity (Mac version) shows that the sound frequencies produced by the two instruments (cello and double-bass) are included in the range 8 – 350 Hz (in the figure: Spectrogram log f; black line). These low frequencies are perceived by the deaf performer through the sense of touch.

Fig. 8 The spatialization process during the VVm performance

Fig. 8 The spatialization process during the VVm performance

CL-sequences are the final result of two spatialization events from the initial input (vibrations). First: tactile spatialization of the vibrations derived from the sound/music stimuli. This step leads to the creation, in the mind of the signer, of an internal sham map (SM) of the tactil sensations. Second: linguistic spatialization of the tactil stimuli (SM). This steps corresponds to a translation of the tactile stimuli into the visual-gestual channel use dby the signer to communicate. In the double process of spatialization the input is ‘tactile’, the output is ‘visual’. The result appears highly iconic (CL-sequences= I-sequences) and understandable by different types of audience.

Fig. 9 The semiological space in the VVm performance

Fig. 9 The semiological space in the VVm performance

The new semiological space in the VVm performance is created starting from a tactile input (vibrations). The double process of spatialization (tactile and linguistic spatialization) leads to create a visual output constituted by the CL-sequences that correspond to sequences of images (I-sequences). During the definition of the semiological space syntax and semantics are put on a second level of importance. The dynamic processs of spatialization acquires primary importance in the definition of the signifieds derived from the linguistic spatialization. Both, a hearing and a deaf audience can comprehend the final iconic result (I-sequences).

Haut de page

Table des illustrations

Titre Fig. 1 The performative space in The Conversation (2010)
Légende The performative space is divided into two part: A, here the wood sculpture is located in the center of the space; moreover, the performers (the deaf performer and the music players) stay in this psace during the entire performance; B, the space where the audience watch the performance.
URL http://journals.openedition.org/signata/docannexe/image/934/img-1.png
Fichier image/png, 5,9k
Titre Fig. 2 The interaction between the deaf performer and the sound vibrations in The Conversation (2010)
Légende The wood sculpture is located in the center of the performative space (brown lines); under the sculpture two amplifiers (amplifier system, AS) are located (blu color); the sound vibrations from the AS hit the sculpture (red arrow), and the deaf performer (barefeet on the top of the sculpture) percives the vibrations through the wood (tacitle modality).
URL http://journals.openedition.org/signata/docannexe/image/934/img-2.png
Fichier image/png, 8,8k
Titre Fig. 5 The VVm performance transcription
Légende The transcription derives from the direct analysis of the video-registration of the VVm performance (December 2010, MAMbo, Bologna, Italy). The transcription shows the relation between CL-sequences articulation (localization inside the neutral space of sign articulation) around the three main body axis and the music score timing (music performance). In this transcription does not appear the music score because the artist (F. Grilli) considers the score as integral part of the artistic work and it can not be reproduce.
URL http://journals.openedition.org/signata/docannexe/image/934/img-3.jpg
Fichier image/jpeg, 276k
Titre Fig. 3 CL-sequences used in the VVm performance (The Conversation)
Légende In the figure: on the right of the graphic, the CLs list chosen by the deaf performer to realiza the VVm performance; the graphic shows the combinations of different CLs to carachterize the typology (“nature”) of each part of the performance (capital letters, from A to H) corresponding to the eight parts of the music score. CLB and CL5 – CL5# appear the most used CLs; in particular the CLB appears to be present in all parts of the performance.
URL http://journals.openedition.org/signata/docannexe/image/934/img-4.jpg
Fichier image/jpeg, 60k
Titre Fig. 4 The three body axis of the signs articulation during the VVm performance
Légende In the figure: the three main axis are indicated (head axis H axis, sternum axis S axis, and pelvis axis P axis) in relation to a CL-sequence (example) takes from the transcription of the entire VVm performance of The Conversation. The CLs (CL5, CL5#, and CLB) are articulated around the three body axis corresponding to the pitch, the intensity and the dynamica of the sound vibrations (music example below the VVm transcription; music dynamnic is indicated with green arrows).
URL http://journals.openedition.org/signata/docannexe/image/934/img-5.jpg
Fichier image/jpeg, 48k
Titre Fig. 6a The sound vibration spatialization on the performer’s hands/feet surface during a VVm performance
Légende The sound input derived from high intensity vibrations is perceived/localized in the anterior part of his hands/feet surface; instead, the sound input derived from low intensity vibrations is perceived/localized in the back part of his hands/feet surface.
URL http://journals.openedition.org/signata/docannexe/image/934/img-6.jpg
Fichier image/jpeg, 36k
Titre Fig. 6b The sound vibration spatialization on the performer’s hands/feet surface during a VVm performance
Légende Parallelism between the spatialization of the sound inputs on the performer’s hands/ feet surface and the related CLs articulation around the three main axis inside the neutral space.
URL http://journals.openedition.org/signata/docannexe/image/934/img-7.jpg
Fichier image/jpeg, 40k
Titre Fig. 7 The sound frequencies analysis during the score performance in The Conversation
Légende The analysis of the sound trace using Audacity (Mac version) shows that the sound frequencies produced by the two instruments (cello and double-bass) are included in the range 8 – 350 Hz (in the figure: Spectrogram log f; black line). These low frequencies are perceived by the deaf performer through the sense of touch.
URL http://journals.openedition.org/signata/docannexe/image/934/img-8.jpg
Fichier image/jpeg, 76k
Titre Fig. 8 The spatialization process during the VVm performance
Légende CL-sequences are the final result of two spatialization events from the initial input (vibrations). First: tactile spatialization of the vibrations derived from the sound/music stimuli. This step leads to the creation, in the mind of the signer, of an internal sham map (SM) of the tactil sensations. Second: linguistic spatialization of the tactil stimuli (SM). This steps corresponds to a translation of the tactile stimuli into the visual-gestual channel use dby the signer to communicate. In the double process of spatialization the input is ‘tactile’, the output is ‘visual’. The result appears highly iconic (CL-sequences= I-sequences) and understandable by different types of audience.
URL http://journals.openedition.org/signata/docannexe/image/934/img-9.jpg
Fichier image/jpeg, 28k
Titre Fig. 9 The semiological space in the VVm performance
Légende The new semiological space in the VVm performance is created starting from a tactile input (vibrations). The double process of spatialization (tactile and linguistic spatialization) leads to create a visual output constituted by the CL-sequences that correspond to sequences of images (I-sequences). During the definition of the semiological space syntax and semantics are put on a second level of importance. The dynamic processs of spatialization acquires primary importance in the definition of the signifieds derived from the linguistic spatialization. Both, a hearing and a deaf audience can comprehend the final iconic result (I-sequences).
URL http://journals.openedition.org/signata/docannexe/image/934/img-10.jpg
Fichier image/jpeg, 32k
Haut de page

Pour citer cet article

Référence papier

Anna Ambra Zaghetto, « Musical Visual Vernacular. How the deaf people translate the sound vibrations into the sign language: An example from Italy »Signata, 3 | 2012, 273-298.

Référence électronique

Anna Ambra Zaghetto, « Musical Visual Vernacular. How the deaf people translate the sound vibrations into the sign language: An example from Italy »Signata [En ligne], 3 | 2012, mis en ligne le 30 septembre 2016, consulté le 28 mars 2024. URL : http://journals.openedition.org/signata/934 ; DOI : https://doi.org/10.4000/signata.934

Haut de page

Auteur

Anna Ambra Zaghetto

Née en 1978, Anna Ambra Zaghetto est professeur en musique (Conservatorio di Torino, 1999), docteur en sciences naturelles (Università Statale di Milano, 2008) et détentrice d’un master en anthropologie (Università di Milano Bicocca, 2011). Elle est actuellement en attente d’une bourse post-doctorale à l’EHESS (Paris). Ses intérêts de recherche sont : la relation entre la perception musicale et le langage, les incapacités dans la perception, la spatialisation, l’anthropologie de la musique et du langage, les modèles mathématiques dans la musique et le processus linguistique.

Haut de page

Droits d’auteur

CC-BY-4.0

Le texte seul est utilisable sous licence CC BY 4.0. Les autres éléments (illustrations, fichiers annexes importés) sont « Tous droits réservés », sauf mention contraire.

Haut de page
Rechercher dans OpenEdition Search

Vous allez être redirigé vers OpenEdition Search