Why Don’t We Discuss it?

How to Get the Evaluation of Music Mediation Activities out of its Procrustean Bed?

Irina Kirchberg

CHAPTER PDF

In Quebec and France, the implementation of music mediation activities depends on subsidies offered by arts councils, ministries of culture or social innovation, or donations from patrons. Evaluative practices have been appearing in the cultural sector since the 80s and 90s, with the development of project management practices accompanying the transformation of cultural policies now setting quantified targets. They seek to measure the ‘performance’ of cultural initiatives in order for them to be accountable to funding bodies (sponsors, patrons, etc.). The theme of evaluation is therefore systematically mentioned on the websites of organisations that promote local mediation activities. For example, Culture pour tous [Culture for all]¹, a well-known Quebec organisation, states:

Evaluation is a fascinating subject. In cultural mediation, as in any other field, evaluation helps to deepen activities, strengthen partnerships and even contribute to the achievement of objectives. (Pronovost and Harrison-Boisvert 2015, 1)²

In this context, is it possible to say that music mediation – a specific musical field of cultural mediation³ – finds itself, as Jean-Marie Lafortune puts it, “under the yoke of neoliberalism” (Lafortune 2019)? While the mediation process, in its “idealised” version, aims at “a project of political transformation” and lays “the foundations of an experimental democracy” (Faget 2022, 433), this mediation process is then reduced to a pragmaticism, a technicality, an efficiency that “manages” disputes in a short-term perspective when mediation is “tossed about by the gusts of the market” (ibid.). In short, is it possible to meet the evaluation demands of funding organisations (often governed by the laws of the market) without betraying the political and social project of music mediation? This question is particularly relevant in Quebec, since one of its specificities (especially in Montreal) is that the production of knowledge about cultural mediation is based on a partnership research mechanism.⁴ As co-direc­tor of the Étude Partenariale sur la Médiation de la Musique [Music Mediation Partnership Study] (EPMM)⁵ founded in 2018 and executive director of the Center for Research in Art and Social Engagement Artenso⁶, I can only emphasise how stimulating and fruitful this research cohabitation is, “enabling a plurality of individuals to contribute to the elaboration of collective decisions” (Kirchberg and Poirier 2024). However, it sometime leads me to question the position I adopt as a researcher in these collaborations. As Michel Sebillotte puts it, by committing to partners who have a problem, which may be a problem of evaluation, the researcher “participates in the transformation of reality by accepting that his research directly helps them to act in this reality” (Sebilotte 2007). This makes me wonder: To what extent do we want to conform to our partners’ expectations when it comes to evaluating music mediation activities? Answering this question means looking at the way in which the partners conceive the practice of evaluation and the concrete consequences of these representations (i.e. the performative effects of these representations).

Thus, is it possible to carry out an evaluation whose methods and conclusions are not amputated by the contingencies of the funders or the organism? Some researchers (Deslyper, Eloy, and Picaud 2022, 3) question the autonomy granted to researchers in this context, and the opportunity they are given to make explicit the preconceptions or biases of stakeholders – in short, to demonstrate critical distance from the partnership evaluation project. I propose instead a reflection that will provide a systematic presentation of certain types of reductionism at work in the common discourse on the evaluation of music mediation activities and which could affect the accomplishment of investigations in the field. To this end, I will analyse a few transcriptions of comments made by musicians, heads of music organisations, and representatives of professional artists’ rights organisations between 2019 and 2024 at training sessions, conferences, and consultations.⁷ By showing the limitations of this Procrustean bed,⁸ I will analyse drifts to which these reductionisms lead and, by contrast, present some key values of cultural mediation as it is conceptualised in Quebec. Being affected by ideological biases, the evaluation of music mediation activities would then be warped by a misunderstanding of the values conveyed by these practices. Music mediation would then become the Trojan horse of other interests. That is why I propose, in conclusion, to reflect on the solutions available for transforming evaluation from the Trojan horse it often becomes when impoverished, into a battle horse for the ideals of music mediation.

Additional Load and Assessment Standardisation

In the discussions I took part in, the task of evaluation is sometimes conceived as an exogenous to mediation activities. For the organisations, it is a matter of carrying out an evaluation task in addition to the many artistic, promotional, etc. tasks at the heart of their mission. This evaluation task is then considered as a satellite to the music mediation project, or as an overload, as can be seen in the following two quotes:

There’s a lack of resources, a lack of time. So very often, the extra reflection and participation required by evaluation seems like an extra burden. (General Director of a contemporary music organisation)

Would it be conceivable for those in the field, organisations with little money, to be equipped with standard evaluation tools? Given the constraints you all know about (a poorly resourced environment), that would be the solution. (Musician’s representative)

Given the many constraints already weighing down organisations, some of them already feel that they are showing goodwill in the face of what they see as a barrier to mediation. Since they agree to this evaluation, it should be possible to carry it out at a lower cost. This way of thinking opens the door to a form of assessment standardisation. This explains why, in the grey literature of cultural associations, generic evaluation guides are flourishing, proposing “generic inputs” (Jackson 2004), “uniform analysis grids” (Pronovost and Harrison-Boisvert 2015), and “pre-formatted reports” (Abbo et al. 2018). Researchers who always tend to grasp the specificity of their object may doubt the transferability of these standardised summative evaluation guides (i.e. those that ratify the end of the process of mediation) to capture theatrical, museum, or musical activities, regardless of the type of participation envisaged (Casemajor, Lamoureux, and Racine 2016). Such assessments run the risk of cutting out the singularities at work in mediation activities whose contexts and audiences are, by their very nature, disparate. This raises a series of questions: How can we fit the diversity of what happens during a mediation activity into the straitjacket of a standard guide demanded by a representative of a musicians’ guild, which places more value on results than on processes (which generate their own effects)? How can we propose innovative mediation activities that hybridise formats, contexts, and populations, if an analytical grid already locks or imprisons the horizon of possible results? We run the risk of witnessing the imposition of mediation activities whose objectives prioritise ease of evaluation over their alignment with genuine needs.⁹ It is important to keep in mind that music organisations and artists must manage to operate with reduced human and financial resources, but we must also understand that this leads to reductionisms such as this one, or the one that is about to follow.

Apologetic discourse and objectification of participants

Given their limited resources, organisations must seize every opportunity to grow and perpetuate their funding. In this context, evaluation is often reduced to an apologetic discourse. By this I mean that it consists of a simple discourse defending the value and benefits of mediation activities in the eyes of funders or in the public sphere: “Mediation is so closely linked to funding issues that everyone has a vested interest in saying it went well.” (A funded artist). This form of assessment is limited to verifying the effectiveness of the actions taken (Deslyper, Eloy, and Picaud 2022) and becomes a control tool. The evaluators then find themselves caught up in a logic of legitimisation that does not sit well with an academic approach. This problem is not new to sociologists interested in social work (Ion 2020). This is all the more problematic since William Beauchemin and Noemie Maignien point out that the interest of cultural mediation is that it is “worked from within by critical dynamics and ideals of social transformation” (Maignien and Beauchemin 2019). By becoming an apologetic or enchanted discourse, these evaluations present mediation as a miracle cure for the ills of society, rather than truly being able to report on the challenges and issues faced by the populations with whom mediators work. This leads directly to another common kind of reductionism, which is very clearly shown in the first excerpt from a report on the subject of art and social inclusion written by a consortium of experts: “Of course, it [the evaluation of cultural activities] also contributes to verifying the attainment of the objectives that have been set and their effectiveness in terms of the benefits expected from them” (Boisvert and Bracco 2023, 114). Here, evaluation is reduced to a tautological discourse that simply says the same thing over and over again, to reinforce itself. In short, in this situation, the evaluation only considers the effects of the mediation activities for which it was subsidised. The first risk of this is that the decision-makers alone decide which effects are important for the participants. The second risk is that some organisations focus their actions solely on the objectives already identified, to the detriment of their intrinsic mission.

Productivist Bias and Measuring versus Evaluation

This confusion between measuring and evaluating, between performance and effects, has led to a preference for quantitative rather than qualitative approaches to evaluating music mediation activities. The vocabulary used in the following excerpt is clear: Evaluating has become equated with measuring and verifying:

I represent the community and I have to do evaluations for my training courses, and we’re always going to try to bring that back on solid ground […] I understand that there are very broad and political reflections that have to be made, but the granting agencies have requirements for their accountability […] Are you able to measure what the granting agency is giving money for? Is it measurable? (Musician’s representative)

As Sylvie Pébrier points out, in this context, the subjective evaluation method, aimed at assessing or estimating a project, gives way to an evaluation method focused on determining or establishing the value of the action (Pébrier 2020). Between these two methods, the gap between a judgement to be made and a measure to be implemented comes to the fore. This is what I call a productivist bias. Evaluation becomes an administrative exercise, which at the end of the project or the year consists of submitting a balance sheet to reassure the grantmakers that the objectives set beforehand have been achieved. The lexical field used by the organisations’ representatives is highly instructive in this respect: Effects must be produced, and “impacts” (Quintas 2014) measured. As François Matarasso explains, this opens the door to several problems (Matarasso 2014; 2016; 2023). The term “impact” seems inappropriate in cultural mediation, since it is not necessarily the mediators’ vocation to produce a continuous effect on or change for the participants. If, as in physics, impact is the way something hits an object, can this notion be applied to what really happens in a mediation activity? During these activities, are participants passively subjected to the impact – or, to stretch the metaphor, the ‘shock’ – of the ‘culture’ that is shared with them in an activity? While the impact metaphor is flattering for politicians, it assumes a passive posture on the part of audiences and participants of mediation activities, thus denying their agency. The neoliberal turn to evaluation reflected in the vocabulary identified above is particularly problematic for cultural mediation, since its “main objective is to empower the individuals and communities who need it most, through a transfer of skills and responsibilities, [whereas] the dominant economic logic instead accentuates their dependence.” (Lafortune 2019, 222).

Conclusion: Making evaluation a subject for debate

Participatory research can help us – researchers, practitioners, and funders – “work together on the meaning” (Pébrier 2020) of our actions, and highlight the ethical objectives of music mediation activities in order to transform our way of thinking about and doing evaluations. One of the mediators on our research team puts it this way:

I think we also need to take inspiration from the scientific method when we’re doing an evaluation where we’re posing a hypothesis and we don’t really have the answer to our question at the outset […] We can end up with something that wasn’t planned and which is completely satisfactory, and we need to avoid settling for a grid that would be restrictive and that would cause us to miss out on these positive unforeseen events. (Head of Mediation Department)

Evaluation in music mediation should therefore be the pretext for “an ongoing exercise in reflection, integrated into the project” (Pébrier 2020, 12) On this subject, Jimmy Gamble points out: “The techniques that make for excellent evaluation in more static situations prove useless, or even harmful, in situations marked by uncertainty, when the rules of the game are constantly changing.” (Gamble 2008, 13) That is why it can be stated that the evaluation of music mediation activities should no longer be summative (i.e. judging the results), but evolutionary (i.e. adapting at the end of each activity to the issues encountered by the mediators and participants). Among other things, Pébrier (2020) also recommends that evaluation in music mediation should be participatory, to reflect on the ethics of evaluation, which is to encourage critical and logical thinking, and to recognise the abilities of each individual. Indeed, mediation’s mandate is “to be part of a horizontal relationship between the stakeholders that he or she brings together around an artistic work, in order to encourage each person’s ability to formulate his or her own judgment” (Gomez 2020). Evaluation should therefore become dialogical. It should open up a dialogue with participants on their expectations of the actions they take part in, and enable them to express their feelings throughout the activities. The conditions for voluntary, active, and inclusive participation (Zask 2011) should therefore be established for all those involved in an activity: artists, mediators, participants, and others. In this way, evaluation reinforces the identity of music mediation and reaffirms its political ambitions.

Finally, the replacement of “cultural” by “music” in the expression “music mediation” is intended to indicate that the mediation process takes on the character of music. With this in mind, shouldn’t the evaluation processes for these specific activities make it possible to grasp the specific effects of music in the context of these activities more effectively? Here, the full potential of art-based evaluation¹⁰ – a technique that uses metaphor as a means of exploring concepts and ideas that might otherwise be difficult or uncomfortable to communicate – remains to be explored.

Endnotes

1. See https://www.culturepourtous.ca/en/ (accessed April 10, 2025).↩︎

2. All translations by the author.↩︎

3. In the French-speaking world, authors use different terms to refer to music mediation activities: “médiation de la musique”, “médiation musicale” or “médiation culturelle de la musique”. I had the opportunity to clarify the nuances between these terms in the introduction to an issue of a magazine published in 2020 (Kirchberg 2020). Like Sylvie Pébrier, I note that these activities have the common aim of “[situating] music not so much as the finality of mediation, but rather as the means of an encounter that seeks to make culture together” (Pébrier 2020), and are therefore placed in the wake of the objectives of cultural mediation, but use music as a lever.↩︎

4. For example, the Observatoire des médiations culturelles, the Centre de recherches en art et engagement social and the Etude partenariale en médiation de la musique benefit from federal and provincial funding to support partnership research which “requires researchers and practitioners to work together, combining their knowledge, methods and resources to carry out research projects that are as relevant to the university community as they are to the practice environment”. (Sutton 2007, 6)↩︎

5. See https://epmm.p2m.oicrm.org/ (accessed April 10, 2025).↩︎

6. See https://artenso.ca/ (accessed April 10, 2025).↩︎

7. 2023 Orford Musique, Formation professionnelle à la médiation de la musique, MFort, Orford Musique, Conseil de la culture de l’Estrie, CQM, Guilde des musiciens. Subventionné par emploi Québec, January 5-8, 2023; 2021 EPMM workshop #3 Évaluer la médiation de la musique, December 3, 2021: https://epmm.p2m.oicrm.org/projet/atelier-epmm-3-evaluer-la-mediation-de-la-musique/ (Accessed April 10, 2025); 2019, Conseil Québécois de la musique, “Qu’est-ce que la médiation de la musique” and “Etats des lieux des pratiques de la médiation de la musique à l’ère du numérique”, Grands rendez-vous de la musique 2019, Montréal, June 5-6; 2018, Conseil Québécois de la Musique, “La médiation de la musique telle que vue par les diffuseurs et producteurs du Québec”, Montréal, September 20; 2018, Bourse rideau, “Pratiques exemplaires en développement de public”, Québec, February 12, 2018.↩︎

8. I am referring here to the Greek myth of Procrustes, a brigand who, under the pretext of welcoming strangers for the night, would lay them on a bed and cut off their limbs, or stretch them out, so that they corresponded exactly to the measurements of the bed. The expression “Procrustean bed” is therefore used to describe a narrow, awkward, tyrannical rule that is applied automatically.↩︎

9. For example, it will be easier to assess the evolution of musical skills than to describe the effects in terms of social cohesion or intercultural dialogue.↩︎

10. See http://www.ascevaluation.ca/Documents/Toolkits/GOAL_ArtBasedEvaluation.pdf (accessed April 10, 2025).↩︎

Bibliography

Abbo, Julie, Anne-France Bonnet, Pauline Chatin, and Louise Renaudin. 2018. “La boussole de l’Art Citoyen. renforcer l’impact de votre projet frâce à l’évaluation.” Fondation Daniel et Nina Carasso, Nueva vista. Accessed April 10, 2025. https://www.fondationcarasso.org/wp-content/uploads/2019/02/La-Boussole-de-lArt-Citoyen-WEB.pdf.

Boisvert, Daniel, and Marie-Anne Bracco. 2023. “La culture… un levier de transformation individuelle; sociale et collective!” Consortium National d’Expertise en Inclusion Sociale (CNEIS). Accessed April 10, 2025. https://www.cneis.ca/wp-content/uploads/Rapport-de-recherche-sur-les-activites-culturelles-inclusives-CNEIS-avril-2023.pdf.

Casemajor, Nathalie, Ève Lamoureux, and Daniele Racine. 2016. “Art participatif et médiation culturelle: typologie et enjeux des pratiques.” In Les mondes de la médiation culturelle. Volume 1: approches de la médiation, edited by Cécile Camart, François Mairesse, Cécile Prévost-Thomas, and Pauline Vessely, 171‑84. Paris: L’Harmattan.

Deslyper, Rémi, Florence Eloy, and Myrtille Picaud. 2022. “Cuisine (sociologique) et dépendances.” Geneses 127, no. 2: 129‑44.

Faget, Jacques. 2022. “Les vents contraires de la médiation.” In
L’art de pacifier nos conflits, edited by Imen Benharda, 339‑345. Toulouse: Érès.

Gamble, Jamie A. 2008. “Abc de l’évaluation évolutive. La fondation de la famille J. W. McConnell.” Accessed April 10, 2025. https://mcconnellfoundation.ca/wp-content/uploads/2017/07/A-Developmental-Evaluation-Primer-FR.pdf.

Gomez, Émilie. 2020. “Le maître ignorant. Cinq leçons sur l’émancipation intellectuelle, de Jacques Rancière.” Revue musicale OICRM 7, no. 2: 150‑57. Accessed April 10, 2025. DOI: https://doi.org/10.7202/1072423ar.

Ion, Jacques. 2020. “Libres propos d’un vieux chercheur ayant bourlingué dans les eaux du travail social.” In Le travail social en quête de légitimité, edited by Jean-Sébastien Alix, Michel Autès, and Éric Marlière, 141‑48. Presses de l’EHESP. Acccessed April 10, 2025.
https://www.cairn.info/le-travail-social-en-quete-de-legitimite–9782810908677-page-141.htm.

Jackson, Annabel. 2004. “Evaluation Toolkit for the Volontary and Community Arts in Northern Ireland.” The Arts Council of the Northern Ireland. Accessed April 10, 2025. https://www.nefconsulting.com/wp-content/uploads/2017/05/VoluntaryCommunityArtsEvalToolkit_000.pdf.

Kirchberg, Irina, and Christian Poirier. 2024. “Expérimenter des pratiques de cohabitation: échanges symétriques et pouvoirs asymétriques.” In Cohabiter. Imaginer les médiations culturelles au 21e siècle, edited by Eve Lamoureux, William Beauchemin, Nathalie Casemajor, and Louis Jacob. Presses Universitaires de Laval. Mondes culturels. Laval.

Lafortune, Jean-Marie. 2019. “Les dispositifs de médiation institutionnels de médiation culturelle sous le joug du néolibéralisme.” In L’essor de la vie culturelle au XXIe siècle: perspectives France-Québec, edited by Maryse Paquin, Jean-Marie Lafortune, Myriam Lemonchois, and Martin Lussier, 213-224. Montréal: Presse de l’Université de Montréal.

Maignien, Noémie, and William-Jacomo Beauchemin. 2019. “Expérimenter les hybridations entre recherche-création et médiation: pour une rencontre entre institutions culturelles et marges.” Animation, territoires et pratiques socioculturelles (Revue ATPS), no. 16 (décembre): 95‑111.

Matarasso, François. 2014. “A Three Pipe Problem.” Accessed April 10, 2025. https://arestlessart.com/wp-content/uploads/2023/11/matarasso-2014-a-three-pipe-problem.pdf.

Matarasso, François. 2014. 2016. “Good Enough?” Accessed April 10, 2025. https://arestlessart.com/2016/04/03/good-enough/.

Matarasso, François. 2014. 2023. Music and Social Change. Accessed April 10, 2025. https://arestlessart.com/writing/old-words/music-and-social-change/.

Pébrier, Sylvie. 2020. “Evaluer la médiation culturelle de la musique. Le travail en commun du sens.” Revue musicale de
l’OICRM 7, no. 2.

Pronovost, Marc, and Catherine Harrison-Boisvert. 2015. “Guide sur l’évaluation de projets en médiation culturelle.” Montréal: Culture pour tous. Accessed April 10, 2025. https://www.culturepourtous.ca/professionnels-de-la-culture/mediation-culturelle/wp-content/uploads/sites/6/2015/09/Guide_Evaluation_projets_CPT_mai2015.pdf.

Quintas, Eva, and Marc Pronovost. 2014. Mise en oeuvre de projets de médiation culturelle. Accessed April 10, 2025. https://www.culturepourtous.ca/professionnels-de-la-culture/mediation-culturelle/wp-content/uploads/sites/6/2015/05/mise_en_oeuvres_projets_mediation.pdf.

Sebilotte, Michel. 2007. “L’analyse des pratiques: Réflexions épistémologiques pour l’agir du chercheur.” In La recherche participative: multiples regards, edited by Marta Anadon, 13‑20. Québec: Presses de l’Université du Québec.

Sutton, Louise. 2007. “La recherche partenariale: Le modèle de l’ARUC-ÉS et du RQRP-ÉS.” L’Alliance de recherche universités-communautés en économie sociale et le Réseau québécois de recherche partenariale en économie sociale. Accessed April 10, 2025. https://chairerp.uqam.ca/fichier/document/Publications/La_recherche_partenariale_Le_mod%C3%A8le_de_lARUC-%C3%89S_et_du_RQRP-%C3%89S.pdf.

Zask, Joëlle. 2011. Participer: essai sur les formes démocratiques de la parti­cipation. Les voies du politique. Lormont: Le Bord de l’eau.