Almost as remarkable as John Cage's actual music is how varied it is. Nothing obvious connects The Wonderful Widow of Eighteen Springs, a beautifully simple four-note piece for soprano and piano lid, to Four Solos for voice, a late work in which four singers independently do a series of strange things at the same time. But somehow all the works connect; each exudes the same sense of serendipitous serenity; they stand alongside each other much like the four soloists in Four Solos, totally distinct and yet together too.

The vocal group Exaudi are exemplary performers of anything they do, and this was a typically spotless rendition of all the Cage pieces, sounded in St. Leonard's Church, Shoreditch to beautiful effect. At the other concerts I've attended here, it's always taken ensembles a few minutes or movements to adjust their balance to the acoustic, but there were no such problems for Exaudi, despite their beginning with a performance of Four2 (1990) in which singers were stationed in various places around the church, presumably impairing communication between them. The layout meant that most of the audience seemed to have no idea where the piece's occasional very low bass notes were coming from, and the many turning heads and confused giggles contributed to an eminently Cagean atmosphere.

The Wonderful Widow of Eighteen Springs (1942) and A Flower (1950) were performed by soprano Amy Moore with the ensemble's director James Weeks tapping away on a piano lid. This is earlier Cage, and the works have an ecstatic simplicity about them, with the most minimal of vocal lines, made up of just four notes and the occasional slide. Moore sang the pieces with a direct, pure tone, clear and presentational but a little expressive too. Though I couldn't make out the words of The Wonderful Widow from where I sat, they were taken from James Joyce's Finnegans Wake so it probably didn't make much difference, and this piece blended seamlessly into the vowels-only A Flower.

Moore's impressions of a pigeon and a wild duck in A Flower set the tone for Four Solos for voice, which followed. Exaudi's performance of this virtuosic and rather silly piece was hugely impressive; the four singers stood calmly in a line, looking quite serious whenever they weren't screaming manically into their microphones, impersonating a fly, coughing theatrically, reciting prose, or singing snippets of opera or light jazz. There were nominally four distinct pieces – numbered 93 to 96, following on from the Song Books, which go up to 92 – but they mostly all blended together into one huge, confusing, hilarious whole. That said, the last number did seem to hint at a sort of pan-dimensional polyphony, with some correspondence (possibly) emerging between the parts. But this may well just have been my ears, getting used to the style and searching for patterns. It's hard to accept that chance is as deeply engrained in this composition as it really is.

The concert's final piece was different again: Hymns and Variations takes two 18th-century American hymn tunes by William Billings and removes a diverse selection of notes, while lengthening some of those which remain. All sense of the original hymns' flow is lost, but what is gained is a remarkable, soft and wonderful collection of sounds, occasionally reminiscent in harmony and gesture of the music it's based on but never less than weirdly compelling in its own right. What is truly radical about this piece is the number of different versions of it that would have been just as beautiful; as ever with Cage, the composition is more a question asked than an answer, and the answer which emerges in performance speaks for itself.