Towards the end of the 19th century, and in the wake of Wagner’s achievement in Tristan und Isolde and Parsifal, the musical language which had been common property of Western composers since the Renaissance, underwent a crisis.

What we now know as tonality, which is the system of keys and scales, and the harmonic progressions, which had been accepted by audiences since at least the end of the Middle Ages, entered a kind of flux. Keys were no longer stable; dissonances began to resolve onto other dissonances (as in the Prelude to Tristan und Isolde), new harmonies began to insert themselves into the old sequences, and the scale expanded from eight notes to the twelve-note chromatic scale, using notes at random from other keys, and constructing sinuous melodic lines that seemed more adapted to dark and solitary emotions than to the cheerful day-light exuberance of choral song.

The crisis deepened during the first quarter of the 20th century, as a result of two striking innovations. The first was that of Debussy, anticipated by Liszt, who began to use the whole-tone scale (the scale without semitones). This scale, emphasizing each note equally, and being without a dominant, is directionless and lacks the dynamic tension of the traditional major and minor modalities. From the whole-tone scale new harmonies emerge – static, indolent, yet somehow not at rest. Debussy combined this scale with post-Wagnerian harmonies, in music which was guided entirely by his own sensitive ear, and by none of the rules of classical harmony, not even those followed and stretched by Wagner. Ravel followed suit, and in due course the French composers were to influence Bartók, Stravinsky and Janácek, all of whom borrowed the whole-tone language when they needed it, meanwhile inventing with the ear.

The second innovation, yet more subversive, was the introduction of entirely atonal melodies and harmonies by Schoenberg, who also, in his vocal setting Pierrot Lunaire, used Sprachgesang – a kind of insinuating sing-song, in which words are deftly stuck onto the musical line, rather than being sung to a melody of their own. It was impossible to dismiss Schoenberg’s innovations as the work of a second-rate composer trying to disguise his incompetence. In Gurrelieder, Verklärte Nacht, and Pelléas et Mélisande he had shown total mastery of tonality and of late romantic harmony, and these great works remain part of the repertoire today. But by the time of the Piano Pieces op. 11 he was writing music which to many people no longer made sense, with melodic lines that began and ended nowhere, and harmonies that seemed to bear no relation to the principal voice. At the same time, Schoenberg’s atonal pieces were meticulously composed, according to schemes that involved the intricate relation of phrases and thematic ideas.

In due course this meticulousness led to an obsession with structure and the quasi-mathematical idiom of twelve-tone serialism, in which the linear relations of tonal music were entirely replaced by a permutational grammar. The result, in the hands of a musical genius like Schoenberg, was intriguing, often (as in the unfinished opera Moses und Aron, and The Survivor from Warsaw) genuinely moving. Schoenberg’s pupils Alban Berg and Anton Webern developed the idiom, the one in a romantic and quasi-tonal direction, the other towards a refined pointillistic style that is uniquely evocative. But it should be remembered that all these experiments were begun at a time when Mahler was composing tonal symphonies, with great arched melodies in the high romantic tradition, and using modernist harmonies only as rhetorical gestures within a strongly diatonic style. And in England Vaughan Williams and Holst were working in a similar way, treating dissonances as by-ways within an all-including tonal logic.

A concert-goer in the early 1930s would have been faced with two completely different musics – one (Vaughan Williams, Holst, Sibelius, Walton, Strauss, Busoni) remaining within the bounds of the tonal language, the other (Schoenberg and his school) consciously departing from the old language, and often striking a deliberately defiant pose towards it. Somewhere in between those two musics hovered the great eclectic geniuses, Stravinsky, Bartók and Prokoviev. And meanwhile the polemics abounded, some dismissing the tonal idiom as reactionary and exhausted, some attacking the modernists as nonsensical and deliberately insulting to the good bourgeois audiences who paid for their self-indulgence.

As we know the contest between tonality and atonality continued throughout the 20th century. The first was popular, the second, on the whole, popular only with the elites. But it was the elites who controlled things, and who directed the state subsidies to the music that they preferred – or at least, that they pretended to prefer. From the time (1959) when the modernist critic Sir William Glock took over the musical direction of BBC’s Third Programme, only the second kind of contemporary music was broadcast over the airwaves in Britain. Composers like Vaughan Williams were marginalised, and experimental voices given an airing in proportion to their cacophonousness. During the 1950s there also grew up in Darmstadt a wholly new pedagogy of music, under the aegis of Karlheinz Stockhausen. Composition, as taught by Stockhausen, consisted in total randomness of inspiration combined with a meticulous mathematisation of the score, to produce music which makes little or no sense to the ear, but which fascinates the eye when spelled out on the page.

Stockhausen’s own works – bulbous monstrosities which make maximum demands on the listener’s attention and give next to nothing in return – received and still receive extensive, usually state-subsidised performances all across the world. His older Austrian contemporary, Gottfried von Einem, who was at the time writing powerful operas in a tonal idiom influenced by Stravinsky and Prokoviev, was in comparison ignored, not because his music is trivial, but because he was perceived to be out of touch with a musical culture determined to clear away the dangerous vestiges of the romantic worldview.

It is no longer accepted as proof of a low-brow musical sensibility to wish to reconnect with that the romantic tradition. It is now permissible to like Sibelius and Vaughan Williams, and to believe that they are superior to Stockhausen and Boulez. It is permissible to reject the notion that tonality was made irrelevant by the atonal school, and to recognise that some of the greatest works in the tonal tradition were composed in the middle of the 20th century: Rachmaninov’s Rhapsody on a Theme of Paganini, for example, Strauss’s Four Last Songs, Britten’s War Requiem, the later symphonies of Shostakovitch and Vaughan Williams, Aaron Copland’s Clarinet Concerto and Appalachian Spring. Some of these – the Rachmaninov and the Strauss – could be seen as extracting unexploited remainders from the tonal tradition. Others – Britten (and not in this work only) and Copland – could be seen as actively engaged in renewing the tonal tradition, drawing out new kinds of melodic line and novel harmonic sequences.

The idea put about by the Marxist critic Theodor Adorno, that tonality was by then nothing but the exhausted remainder of a dead tradition, was definitely disproven by post-war music. By the 1950s it was atonality and not tonality that was exhausted. The radical modernist idiom was kept alive by Darmstadt and the musical establishment, by the system of official patronage and by the fact that real musical education, which used to be a household requirement, had been effectively destroyed by the invention of broadcasting and recording. Without a musical education it is not easy to feel confident in saying, of Stockhausen, that the Emperor has no clothes.

Meanwhile the real modernist experiments – those which drew freely on the tonal tradition and on the eclectic spirit of Western civilisation – gained acceptance from the ordinary concert-goer. Works like the Turangalila Symphony of Messiaen, the remarkable Star-Child oratorio by George Crumb, and the triple concerto of Michael Tippett entered the repertoire without any need for the critical hype and institutional support enjoyed by Stockhausen and Boulez.

But there is another reason for the brief ascendancy of radical modernism, and one that bears heavily on the future of Western music. During the course of the 20th century a wholly new kind of popular music emerged. Nobody can say, in retrospect, that the waltzes and polkas of Strauss or the operettas of Léhar and Offenbach belong to another language and another culture from the symphonies of Brahms or the music dramas of Wagner. Strauss (father and son), Léhar, Offenbach are now counted in the ‘classical’ repertoire, just as much as Wagner, Brahms and the other Strauss. And the distinction between popular entertainment and high art is internal to their repertoire: the Overture to Der Fledermaus and the Hungarian Dances of Brahms surely stand side by side.

Only in the 20th century did popular and serious music finally divide, and the principal reason for this was jazz – itself an intrusion into Western culture from a place outside it. The origin of this remarkable idiom is veiled in obscurity, though it is evident that it absorbed, along the way, both the syncopated rhythms of African drum music, the blues notes that come from attempting to unite the pentatonic and the diatonic scales, and the chord grammar of the Negro spirituals. The jazz idiom showed a remarkable ability to develop, so that an entirely new harmonic language grew from it, and soon became the foundation of a new kind of popular song and dance. It was this quintessentially American idiom that most got up the nose of Adorno, and which served as his proof that tonality was destined to degenerate into short-breathed melodies and repetitious sequences. For that was all that Adorno found in the jazz of his day.

It is true that improvisation around a ‘jazz standard’ is a very different thing from the far-ranged musical thinking that we find in the concert-hall. A work that returns constantly to the same source for refreshment, and goes on ‘forever’ precisely because it goes on only for a moment is a very different thing from the symphony which develops thematic material, extracting a continuous musical narrative. But Ravel, Gershwin and Stravinsky showed how to incorporate jazz melodic lines and even jazz harmonic sequences into symphonic works that had some of the long-distance complexity of the classical tradition. Meanwhile there emerged a new form of popular music, on the edge of jazz, but reaching into the world of folk melody and light opera. This was the idiom of the Broadway Musical and the American Song Book. Brilliant musicians like Cole Porter, Hoagy Carmichael and Richard Rodgers became household names, with songs that our parents knew by heart, and which defined a new kind of taste.

This was music to be sung around the house, which normalised the emotions of ordinary people as they endeavoured to cope with the new world of machines, gadgets, social mobility, fast romance and easy divorce. Thus began the great fracture in the world of music between ‘pop’ and ‘classical’, in which it became ever more important for the critics to side with the classical tradition, and to find something that distinguished modern composers in that tradition from the ‘easy listening’ and ‘light music’ that filled the suburban bathroom.

For a while, therefore, there was an added motive for composers to take the path of radical modernism, and so to give proof that they belonged to the great tradition of serious musical thinking. A composer like Boulez or Fernyhough, ensconed in the madhouse of IRCAM in Paris, could be ‘bounded in a nutshell and count himself king of infinite space’, as Hamlet put it. Insulated from the vulgar world of musical enjoyment, sending out musical spells into the electronic ether, the composer began to live in a world of his own.

That it should be Boulez who received the accolades and not Maurice Duruflé or Henri Dutilleux is explained by the enormous publicity value of difficulty, when difficulty is subsidised by the state. The modernists had succeeded in persuading the official bodies that they were keeping alive the flame of high art amid the grotesqueries of an increasingly degenerate pop culture. And for a while, following the transformation of rhythm and blues into a universal idiom of song and dance by Chuck Berry, The Beatles and The Rolling Stones, it seemed as though they were right. What did this new popular music have to do even with the comparatively refined language and domestic charm of the Broadway musical, still less with the symphonic and operative traditions?

But then the whole thing collapsed. Impassable divides have an ability to survive in the old hierachical culture of Europe; but they don’t last for long in America. Composers like Steve Reich and Philip Glass had no desire to separate from their hippy friends, or to lose the one benefit that makes the life of a composer worthwhile, namely an audience. There emerged the new idiom of minimalism, in which the harmonic complexities of the modernists and those of the great jazz musicians like Monk, Tatum and Peterson were both rejected in favour of simple tonal triads, often repeated ad nauseam on mesmeric instruments like the marimbas.

The result, to my ear utterly empty and the best argument for Boulez that I have yet encountered, succeeded in entering the repertoire and gaining a young and enthusiastic audience. The explanation is simple. This music uses the devices of pop: regular and mechanical rhythm, fragmented and constantly repeated melodic lines, and a small repertoirs of chords constantly repeated. It has in effect joined the world of ‘easy listening’.

Whether Reich and Glass entitle us to talk of a new and ‘postmodern’ idiom in the world of serious music I doubt. For this is not serious music, but a kind of musical void. Listening to Glass’s opera Ekhnaton, for instance, you will be tempted to agree with Adorno, that the musical idiom (let’s not speak of the drama) is utterly exhausted. But then along came John Adams, whose mastery of orchestration and knowledge of real tonal harmony began to redeem the minimalist idiom, and to bring it properly into the concert hall. And other American composers followed suit – Torke, Del Tredici, Corigliano – writing ‘tonal music with attitude’, inserting advanced harmonic episodes into structures that make thematic and rhythmical sense. In Britain a new wave of tonal composers has also emerged, some of them – like James Macmillan, Oliver Knussen and David Matthews beginning as radical modernists – but all moving along the path mapped out by the great Benjamin Britten, the path out of the sterile desert where no melody ever grows, into a place of song and dance.

Such composers learned the lesson taught (however clumsily) by Reich and Glass, which is that music is nothing without an audience, and that the audience must be discovered among young people whose ears have been muddled by the ostinato rhythms and empty chord sequences of pop. To offer serious music to such an audience you must also attract their attention. And this cannot be done without melody and rhythm that connect with their own bodily perceptions.

Is it working? I don’t know. The catastrophic effect of modern pop – think of youjr own Death Metal Group Meshuggah, or the obscene Lady Gaga – is felt not only in the ears but in the soul of its devotees. And it is difficult to write fresh and tuneful music for a burnt out soul. Still, we have no other path to tread, unless it is the one that reaches back into that sound-proof bunker beneath the Centre Pompidou, where the spells are still being brewed and sent with undiminished malice across the airways of Europe.